Oct 30 21:49:07.438937 ip-10-0-136-210 systemd[1]: Starting Kubernetes Kubelet...
Oct 30 21:49:07.894627 ip-10-0-136-210 kubenswrapper[2576]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 21:49:07.894627 ip-10-0-136-210 kubenswrapper[2576]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Oct 30 21:49:07.894627 ip-10-0-136-210 kubenswrapper[2576]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 21:49:07.894627 ip-10-0-136-210 kubenswrapper[2576]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 21:49:07.901492 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.901398    2576 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:49:07.901492 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.901486    2576 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:49:07.901492 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.901490    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:49:07.901492 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.901494    2576 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:49:07.901492 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.901497    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:49:07.901492 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.901499    2576 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:49:07.901492 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.901504    2576 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:49:07.901873 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.901507    2576 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:49:07.901873 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.901510    2576 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:49:07.901873 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.901513    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:49:07.901873 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.901516    2576 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:49:07.901873 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.901519    2576 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:49:07.901873 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.901522    2576 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:49:07.901873 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.901525    2576 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:49:07.901873 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.901528    2576 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:49:07.901873 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.901530    2576 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:49:07.901873 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.901534    2576 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:49:07.901873 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.901539    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:49:07.901873 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.901542    2576 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:49:07.901873 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.901544    2576 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:49:07.901873 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.901547    2576 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:49:07.901873 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.901549    2576 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:49:07.901873 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.901552    2576 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:49:07.901873 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.901555    2576 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:49:07.901873 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.901558    2576 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:49:07.901873 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.901561    2576 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:49:07.902590 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.901563    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:49:07.902590 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.901566    2576 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:49:07.902590 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.901568    2576 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:49:07.902590 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.901571    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:49:07.902590 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.901574    2576 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:49:07.902590 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.901577    2576 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:49:07.902590 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.901580    2576 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:49:07.902590 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.901583    2576 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:49:07.902590 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.901585    2576 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:49:07.902590 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.901588    2576 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:49:07.902590 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.901591    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:49:07.902590 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.901593    2576 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:49:07.902590 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.901597    2576 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:49:07.902590 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.901600    2576 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:49:07.902590 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.901603    2576 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:49:07.902590 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.901606    2576 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:49:07.902590 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.901608    2576 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:49:07.902590 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.901611    2576 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:49:07.902590 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.901614    2576 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:49:07.902590 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.901616    2576 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:49:07.903100 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.901619    2576 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:49:07.903100 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.901621    2576 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:49:07.903100 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.901624    2576 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:49:07.903100 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.901627    2576 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:49:07.903100 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.901630    2576 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:49:07.903100 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.901632    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:49:07.903100 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.901635    2576 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:49:07.903100 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.901638    2576 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:49:07.903100 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.901641    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:49:07.903100 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.901644    2576 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:49:07.903100 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.901646    2576 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:49:07.903100 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.901649    2576 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:49:07.903100 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.901651    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:49:07.903100 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.901671    2576 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:49:07.903100 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.901674    2576 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:49:07.903100 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.901677    2576 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:49:07.903100 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.901679    2576 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:49:07.903100 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.901682    2576 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:49:07.903100 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.901685    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:49:07.903100 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.901687    2576 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:49:07.903567 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.901690    2576 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:49:07.903567 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.901692    2576 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:49:07.903567 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.901695    2576 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:49:07.903567 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.901698    2576 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:49:07.903567 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.901701    2576 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:49:07.903567 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.901706    2576 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:49:07.903567 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.901710    2576 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:49:07.903567 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.901713    2576 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:49:07.903567 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.901716    2576 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:49:07.903567 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.901719    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:49:07.903567 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.901721    2576 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:49:07.903567 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.901724    2576 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:49:07.903567 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.901727    2576 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:49:07.903567 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.901729    2576 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:49:07.903567 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.901732    2576 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:49:07.903567 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.901735    2576 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:49:07.903567 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.901737    2576 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:49:07.903567 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.901742    2576 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:49:07.903567 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.901745    2576 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:49:07.904078 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.901748    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:49:07.904078 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.901750    2576 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:49:07.904078 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.901753    2576 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:49:07.904078 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.901755    2576 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:49:07.904078 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.901759    2576 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:49:07.904078 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.901762    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:49:07.904078 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.901765    2576 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:49:07.904078 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.901768    2576 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:49:07.904078 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.901776    2576 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:49:07.904078 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.901779    2576 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:49:07.904078 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.901782    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:49:07.904078 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.901784    2576 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:49:07.904078 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.901787    2576 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:49:07.904078 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.901790    2576 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:49:07.904078 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.901792    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:49:07.904078 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.901795    2576 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:49:07.904078 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.901798    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:49:07.904078 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.901803    2576 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:49:07.904078 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.901806    2576 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:49:07.904078 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.901809    2576 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:49:07.904569 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.901812    2576 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:49:07.904569 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.903338    2576 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:49:07.904569 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.903347    2576 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:49:07.904569 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.903350    2576 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:49:07.904569 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.903353    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:49:07.904569 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.903356    2576 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:49:07.904569 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.903360    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:49:07.904569 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.903363    2576 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:49:07.904569 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.903365    2576 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:49:07.904569 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.903368    2576 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:49:07.904569 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.903371    2576 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:49:07.904569 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.903374    2576 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:49:07.904569 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.903377    2576 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:49:07.904569 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.903380    2576 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:49:07.904569 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.903382    2576 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:49:07.904569 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.903385    2576 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:49:07.904569 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.903388    2576 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:49:07.904569 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.903391    2576 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:49:07.904569 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.903394    2576 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:49:07.904569 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.903397    2576 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:49:07.905108 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.903399    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:49:07.905108 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.903402    2576 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:49:07.905108 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.903405    2576 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:49:07.905108 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.903407    2576 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:49:07.905108 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.903410    2576 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:49:07.905108 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.903414    2576 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:49:07.905108 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.903417    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:49:07.905108 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.903421    2576 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:49:07.905108 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.903424    2576 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:49:07.905108 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.903429    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:49:07.905108 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.903432    2576 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:49:07.905108 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.903434    2576 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:49:07.905108 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.903437    2576 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:49:07.905108 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.903440    2576 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:49:07.905108 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.903443    2576 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:49:07.905108 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.903446    2576 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:49:07.905108 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.903448    2576 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:49:07.905108 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.903451    2576 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:49:07.905108 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.903453    2576 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:49:07.905558 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.903456    2576 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:49:07.905558 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.903458    2576 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:49:07.905558 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.903461    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:49:07.905558 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.903464    2576 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:49:07.905558 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.903466    2576 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:49:07.905558 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.903470    2576 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:49:07.905558 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.903472    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:49:07.905558 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.903475    2576 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:49:07.905558 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.903477    2576 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:49:07.905558 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.903482    2576 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:49:07.905558 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.903484    2576 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:49:07.905558 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.903487    2576 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:49:07.905558 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.903490    2576 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:49:07.905558 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.903492    2576 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:49:07.905558 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.903495    2576 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:49:07.905558 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.903497    2576 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:49:07.905558 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.903500    2576 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:49:07.905558 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.903502    2576 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:49:07.905558 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.903505    2576 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:49:07.905558 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.903507    2576 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:49:07.906043 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.903510    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:49:07.906043 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.903513    2576 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:49:07.906043 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.903517    2576 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:49:07.906043 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.903519    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:49:07.906043 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.903522    2576 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:49:07.906043 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.903525    2576 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:49:07.906043 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.903528    2576 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:49:07.906043 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.903530    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:49:07.906043 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.903533    2576 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:49:07.906043 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.903535    2576 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:49:07.906043 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.903538    2576 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:49:07.906043 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.903541    2576 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:49:07.906043 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.903543    2576 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:49:07.906043 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.903546    2576 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:49:07.906043 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.903548    2576 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:49:07.906043 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.903551    2576 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:49:07.906043 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.903553    2576 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:49:07.906043 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.903556    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:49:07.906043 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.903561    2576 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:49:07.906043 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.903563    2576 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:49:07.906540 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.903566    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:49:07.906540 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.903569    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:49:07.906540 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.903572    2576 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:49:07.906540 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.903575    2576 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:49:07.906540 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.903577    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:49:07.906540 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.903580    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:49:07.906540 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.903583    2576 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:49:07.906540 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.903585    2576 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:49:07.906540 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.903588    2576 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:49:07.906540 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.903590    2576 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:49:07.906540 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.903593    2576 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:49:07.906540 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.903595    2576 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:49:07.906540 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.903598    2576 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:49:07.906540 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.903600    2576 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:49:07.906540 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.903604    2576 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:49:07.906540 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.903606    2576 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:49:07.906540 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.903609    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:49:07.906540 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.903613    2576 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:49:07.906540 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.903616    2576 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:49:07.907007 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.903620    2576 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:49:07.907007 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.903623    2576 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:49:07.907007 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.903626    2576 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:49:07.907007 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.903629    2576 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:49:07.907007 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.903631    2576 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:49:07.907007 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.903634    2576 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:49:07.907007 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.903636    2576 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:49:07.907007 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.903639    2576 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:49:07.907007 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.903641    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:49:07.907007 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.904201    2576 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:49:07.907007 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.904207    2576 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:49:07.907007 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.904212    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:49:07.907007 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.904215    2576 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:49:07.907007 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.904217    2576 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:49:07.907007 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.904220    2576 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:49:07.907007 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.904223    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:49:07.907007 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.904226    2576 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:49:07.907007 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.904228    2576 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:49:07.907007 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.904231    2576 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:49:07.907456 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.904233    2576 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:49:07.907456 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.904236    2576 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:49:07.907456 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.904238    2576 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:49:07.907456 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.904241    2576 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:49:07.907456 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.904244    2576 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:49:07.907456 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.904246    2576 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:49:07.907456 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.904249    2576 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:49:07.907456 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.904251    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:49:07.907456 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.904254    2576 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:49:07.907456 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.904257    2576 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:49:07.907456 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.904260    2576 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:49:07.907456 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.904262    2576 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:49:07.907456 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.904265    2576 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:49:07.907456 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.904268    2576 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:49:07.907456 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.904271    2576 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:49:07.907456 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.904273    2576 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:49:07.907456 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.904276    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:49:07.907456 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.904278    2576 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:49:07.907456 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.904281    2576 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:49:07.907456 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.904284    2576 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:49:07.907959 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.904287    2576 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:49:07.907959 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.904289    2576 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:49:07.907959 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.904292    2576 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:49:07.907959 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.904295    2576 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:49:07.907959 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.904298    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:49:07.907959 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.904301    2576 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:49:07.907959 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.904304    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:49:07.907959 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.904306    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:49:07.907959 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.904309    2576 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:49:07.907959 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.904312    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:49:07.907959 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.904314    2576 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:49:07.907959 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.904317    2576 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:49:07.907959 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.904319    2576 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:49:07.907959 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.904322    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:49:07.907959 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.904324    2576 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:49:07.907959 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.904327    2576 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:49:07.907959 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.904329    2576 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:49:07.907959 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.904332    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:49:07.907959 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.904334    2576 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:49:07.908418 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.904337    2576 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:49:07.908418 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.904340    2576 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:49:07.908418 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.904343    2576 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:49:07.908418 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.904345    2576 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:49:07.908418 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.904348    2576 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:49:07.908418 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.904350    2576 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:49:07.908418 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.904353    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:49:07.908418 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.904355    2576 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:49:07.908418 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.904358    2576 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:49:07.908418 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.904360    2576 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:49:07.908418 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.904363    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:49:07.908418 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.904365    2576 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:49:07.908418 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.904368    2576 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:49:07.908418 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.904371    2576 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:49:07.908418 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.904374    2576 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:49:07.908418 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.904376    2576 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:49:07.908418 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.904378    2576 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:49:07.908418 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.904381    2576 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:49:07.908418 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.904384    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:49:07.908891 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.904387    2576 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:49:07.908891 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.904390    2576 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:49:07.908891 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.904392    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:49:07.908891 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.904395    2576 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:49:07.908891 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.904398    2576 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:49:07.908891 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.904400    2576 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:49:07.908891 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.904404    2576 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:49:07.908891 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.904408    2576 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:49:07.908891 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.904411    2576 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:49:07.908891 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.904414    2576 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:49:07.908891 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.904417    2576 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:49:07.908891 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.904420    2576 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:49:07.908891 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.904423    2576 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:49:07.908891 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.904425    2576 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:49:07.908891 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.904428    2576 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:49:07.908891 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.904431    2576 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:49:07.908891 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.904433    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:49:07.908891 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.904436    2576 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:49:07.908891 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.904438    2576 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:49:07.908891 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.904441    2576 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:49:07.909352 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.904443    2576 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:49:07.909352 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.904446    2576 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:49:07.909352 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.904448    2576 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:49:07.909352 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.904451    2576 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:49:07.909352 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.904453    2576 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:49:07.909352 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.904456    2576 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:49:07.909352 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.904459    2576 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:49:07.909352 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.904461    2576 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:49:07.909352 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.904464    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:49:07.909352 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.904466    2576 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:49:07.909352 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.904469    2576 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:49:07.909352 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.904471    2576 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:49:07.909352 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.904474    2576 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:49:07.909352 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.904477    2576 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:49:07.909352 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.904480    2576 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:49:07.909352 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.904484    2576 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:49:07.909352 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.904488    2576 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:49:07.909352 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.904490    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:49:07.909352 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:07.905525    2576 flags.go:64] FLAG: --address="0.0.0.0"
Oct 30 21:49:07.909352 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:07.905537    2576 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Oct 30 21:49:07.909836 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:07.905544    2576 flags.go:64] FLAG: --anonymous-auth="true"
Oct 30 21:49:07.909836 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:07.905548    2576 flags.go:64] FLAG: --application-metrics-count-limit="100"
Oct 30 21:49:07.909836 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:07.905553    2576 flags.go:64] FLAG: --authentication-token-webhook="false"
Oct 30 21:49:07.909836 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:07.905556    2576 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Oct 30 21:49:07.909836 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:07.905563    2576 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Oct 30 21:49:07.909836 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:07.905568    2576 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Oct 30 21:49:07.909836 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:07.905571    2576 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Oct 30 21:49:07.909836 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:07.905574    2576 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Oct 30 21:49:07.909836 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:07.905578    2576 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Oct 30 21:49:07.909836 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:07.905581    2576 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Oct 30 21:49:07.909836 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:07.905585    2576 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Oct 30 21:49:07.909836 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:07.905588    2576 flags.go:64] FLAG: --cgroup-root=""
Oct 30 21:49:07.909836 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:07.905591    2576 flags.go:64] FLAG: --cgroups-per-qos="true"
Oct 30 21:49:07.909836 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:07.905594    2576 flags.go:64] FLAG: --client-ca-file=""
Oct 30 21:49:07.909836 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:07.905597    2576 flags.go:64] FLAG: --cloud-provider="external"
Oct 30 21:49:07.909836 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:07.905600    2576 flags.go:64] FLAG: --cluster-dns="[]"
Oct 30 21:49:07.909836 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:07.905606    2576 flags.go:64] FLAG: --cluster-domain=""
Oct 30 21:49:07.909836 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:07.905609    2576 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Oct 30 21:49:07.909836 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:07.905612    2576 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Oct 30 21:49:07.909836 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:07.905616    2576 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Oct 30 21:49:07.909836 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:07.905619    2576 flags.go:64] FLAG: --container-log-max-files="5"
Oct 30 21:49:07.909836 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:07.905623    2576 flags.go:64] FLAG: --container-log-max-size="10Mi"
Oct 30 21:49:07.909836 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:07.905626    2576 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Oct 30 21:49:07.910361 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:07.905630    2576 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Oct 30 21:49:07.910361 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:07.905633    2576 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Oct 30 21:49:07.910361 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:07.905636    2576 flags.go:64] FLAG: --contention-profiling="false"
Oct 30 21:49:07.910361 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:07.905640    2576 flags.go:64] FLAG: --cpu-cfs-quota="true"
Oct 30 21:49:07.910361 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:07.905643    2576 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Oct 30 21:49:07.910361 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:07.905646    2576 flags.go:64] FLAG: --cpu-manager-policy="none"
Oct 30 21:49:07.910361 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:07.905649    2576 flags.go:64] FLAG: --cpu-manager-policy-options=""
Oct 30 21:49:07.910361 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:07.905665    2576 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Oct 30 21:49:07.910361 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:07.905671    2576 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Oct 30 21:49:07.910361 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:07.905674    2576 flags.go:64] FLAG: --enable-debugging-handlers="true"
Oct 30 21:49:07.910361 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:07.905678    2576 flags.go:64] FLAG: --enable-load-reader="false"
Oct 30 21:49:07.910361 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:07.905681    2576 flags.go:64] FLAG: --enable-server="true"
Oct 30 21:49:07.910361 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:07.905684    2576 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Oct 30 21:49:07.910361 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:07.905691    2576 flags.go:64] FLAG: --event-burst="100"
Oct 30 21:49:07.910361 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:07.905694    2576 flags.go:64] FLAG: --event-qps="50"
Oct 30 21:49:07.910361 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:07.905697    2576 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Oct 30 21:49:07.910361 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:07.905701    2576 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Oct 30 21:49:07.910361 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:07.905704    2576 flags.go:64] FLAG: --eviction-hard=""
Oct 30 21:49:07.910361 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:07.905713    2576 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Oct 30 21:49:07.910361 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:07.905717    2576 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Oct 30 21:49:07.910361 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:07.905720    2576 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Oct 30 21:49:07.910361 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:07.905723    2576 flags.go:64] FLAG: --eviction-soft=""
Oct 30 21:49:07.910361 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:07.905726    2576 flags.go:64] FLAG: --eviction-soft-grace-period=""
Oct 30 21:49:07.910361 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:07.905745    2576 flags.go:64] FLAG: --exit-on-lock-contention="false"
Oct 30 21:49:07.910361 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:07.905749    2576 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Oct 30 21:49:07.911055 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:07.905753    2576 flags.go:64] FLAG: --experimental-mounter-path=""
Oct 30 21:49:07.911055 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:07.905756    2576 flags.go:64] FLAG: --fail-cgroupv1="true"
Oct 30 21:49:07.911055 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:07.905759    2576 flags.go:64] FLAG: --fail-swap-on="true"
Oct 30 21:49:07.911055 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:07.905763    2576 flags.go:64] FLAG: --feature-gates=""
Oct 30 21:49:07.911055 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:07.905767    2576 flags.go:64] FLAG: --file-check-frequency="20s"
Oct 30 21:49:07.911055 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:07.905771    2576 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Oct 30 21:49:07.911055 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:07.905774    2576 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Oct 30 21:49:07.911055 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:07.905777    2576 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Oct 30 21:49:07.911055 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:07.905780    2576 flags.go:64] FLAG: --healthz-port="10248"
Oct 30 21:49:07.911055 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:07.905784    2576 flags.go:64] FLAG: --help="false"
Oct 30 21:49:07.911055 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:07.905787    2576 flags.go:64] FLAG: --hostname-override="ip-10-0-136-210.us-west-2.compute.internal"
Oct 30 21:49:07.911055 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:07.905791    2576 flags.go:64] FLAG: --housekeeping-interval="10s"
Oct 30 21:49:07.911055 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:07.905794    2576 flags.go:64] FLAG: --http-check-frequency="20s"
Oct 30 21:49:07.911055 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:07.905797    2576 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Oct 30 21:49:07.911055 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:07.905800    2576 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Oct 30 21:49:07.911055 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:07.905804    2576 flags.go:64] FLAG: --image-gc-high-threshold="85"
Oct 30 21:49:07.911055 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:07.905807    2576 flags.go:64] FLAG: --image-gc-low-threshold="80"
Oct 30 21:49:07.911055 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:07.905810    2576 flags.go:64] FLAG: --image-service-endpoint=""
Oct 30 21:49:07.911055 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:07.905813    2576 flags.go:64] FLAG: --kernel-memcg-notification="false"
Oct 30 21:49:07.911055 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:07.905816    2576 flags.go:64] FLAG: --kube-api-burst="100"
Oct 30 21:49:07.911055 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:07.905821    2576 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Oct 30 21:49:07.911055 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:07.905824    2576 flags.go:64] FLAG: --kube-api-qps="50"
Oct 30 21:49:07.911055 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:07.905827    2576 flags.go:64] FLAG: --kube-reserved=""
Oct 30 21:49:07.911055 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:07.905830    2576 flags.go:64] FLAG: --kube-reserved-cgroup=""
Oct 30 21:49:07.911643 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:07.905833    2576 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Oct 30 21:49:07.911643 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:07.905836    2576 flags.go:64] FLAG: --kubelet-cgroups=""
Oct 30 21:49:07.911643 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:07.905839    2576 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Oct 30 21:49:07.911643 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:07.905842    2576 flags.go:64] FLAG: --lock-file=""
Oct 30 21:49:07.911643 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:07.905845    2576 flags.go:64] FLAG: --log-cadvisor-usage="false"
Oct 30 21:49:07.911643 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:07.905848    2576 flags.go:64] FLAG: --log-flush-frequency="5s"
Oct 30 21:49:07.911643 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:07.905851    2576 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Oct 30 21:49:07.911643 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:07.905857    2576 flags.go:64] FLAG: --log-json-split-stream="false"
Oct 30 21:49:07.911643 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:07.905861    2576 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Oct 30 21:49:07.911643 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:07.905865    2576 flags.go:64] FLAG: --log-text-split-stream="false"
Oct 30 21:49:07.911643 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:07.905868    2576 flags.go:64] FLAG: --logging-format="text"
Oct 30 21:49:07.911643 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:07.905871    2576 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Oct 30 21:49:07.911643 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:07.905875    2576 flags.go:64] FLAG: --make-iptables-util-chains="true"
Oct 30 21:49:07.911643 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:07.905878    2576 flags.go:64] FLAG: --manifest-url=""
Oct 30 21:49:07.911643 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:07.905881    2576 flags.go:64] FLAG: --manifest-url-header=""
Oct 30 21:49:07.911643 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:07.905886    2576 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Oct 30 21:49:07.911643 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:07.905889    2576 flags.go:64] FLAG: --max-open-files="1000000"
Oct 30 21:49:07.911643 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:07.905893    2576 flags.go:64] FLAG: --max-pods="110"
Oct 30 21:49:07.911643 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:07.905896    2576 flags.go:64] FLAG: --maximum-dead-containers="-1"
Oct 30 21:49:07.911643 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:07.905899    2576 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Oct 30 21:49:07.911643 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:07.905903    2576 flags.go:64] FLAG: --memory-manager-policy="None"
Oct 30 21:49:07.911643 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:07.905906    2576 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Oct 30 21:49:07.911643 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:07.905909    2576 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Oct 30 21:49:07.911643 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:07.905912    2576 flags.go:64] FLAG: --node-ip="0.0.0.0"
Oct 30 21:49:07.911643 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:07.905915    2576 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Oct 30 21:49:07.912695 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:07.905922    2576 flags.go:64] FLAG: --node-status-max-images="50"
Oct 30 21:49:07.912695 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:07.905925    2576 flags.go:64] FLAG: --node-status-update-frequency="10s"
Oct 30 21:49:07.912695 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:07.905928    2576 flags.go:64] FLAG: --oom-score-adj="-999"
Oct 30 21:49:07.912695 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:07.905933    2576 flags.go:64] FLAG: --pod-cidr=""
Oct 30 21:49:07.912695 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:07.905936    2576 flags.go:64] FLAG: --pod-manifest-path=""
Oct 30 21:49:07.912695 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:07.905939    2576 flags.go:64] FLAG: --pod-max-pids="-1"
Oct 30 21:49:07.912695 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:07.905942    2576 flags.go:64] FLAG: --pods-per-core="0"
Oct 30 21:49:07.912695 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:07.905946    2576 flags.go:64] FLAG: --port="10250"
Oct 30 21:49:07.912695 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:07.905949    2576 flags.go:64] FLAG: --protect-kernel-defaults="false"
Oct 30 21:49:07.912695 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:07.905951    2576 flags.go:64] FLAG: --provider-id="aws:///us-west-2a/i-0d3218457a2c9bb70"
Oct 30 21:49:07.912695 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:07.905955    2576 flags.go:64] FLAG: --qos-reserved=""
Oct 30 21:49:07.912695 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:07.905958    2576 flags.go:64] FLAG: --read-only-port="10255"
Oct 30 21:49:07.912695 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:07.905961    2576 flags.go:64] FLAG: --register-node="true"
Oct 30 21:49:07.912695 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:07.905964    2576 flags.go:64] FLAG: --register-with-taints=""
Oct 30 21:49:07.912695 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:07.905968    2576 flags.go:64] FLAG: --registry-burst="10"
Oct 30 21:49:07.912695 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:07.905972    2576 flags.go:64] FLAG: --registry-qps="5"
Oct 30 21:49:07.912695 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:07.905975    2576 flags.go:64] FLAG: --reserved-cpus=""
Oct 30 21:49:07.912695 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:07.905978    2576 flags.go:64] FLAG: --reserved-memory=""
Oct 30 21:49:07.912695 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:07.905986    2576 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Oct 30 21:49:07.912695 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:07.905989    2576 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Oct 30 21:49:07.912695 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:07.905992    2576 flags.go:64] FLAG: --rotate-certificates="false"
Oct 30 21:49:07.912695 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:07.905995    2576 flags.go:64] FLAG: --rotate-server-certificates="false"
Oct 30 21:49:07.912695 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:07.905998    2576 flags.go:64] FLAG: --runonce="false"
Oct 30 21:49:07.912695 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:07.906001    2576 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Oct 30 21:49:07.912695 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:07.906004    2576 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Oct 30 21:49:07.912695 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:07.906007    2576 flags.go:64] FLAG: --seccomp-default="false"
Oct 30 21:49:07.912695 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:07.906010    2576 flags.go:64] FLAG: --serialize-image-pulls="true"
Oct 30 21:49:07.913873 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:07.906013    2576 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Oct 30 21:49:07.913873 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:07.906018    2576 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Oct 30 21:49:07.913873 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:07.906021    2576 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Oct 30 21:49:07.913873 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:07.906024    2576 flags.go:64] FLAG: --storage-driver-password="root"
Oct 30 21:49:07.913873 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:07.906027    2576 flags.go:64] FLAG: --storage-driver-secure="false"
Oct 30 21:49:07.913873 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:07.906030    2576 flags.go:64] FLAG: --storage-driver-table="stats"
Oct 30 21:49:07.913873 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:07.906033    2576 flags.go:64] FLAG: --storage-driver-user="root"
Oct 30 21:49:07.913873 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:07.906036    2576 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Oct 30 21:49:07.913873 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:07.906040    2576 flags.go:64] FLAG: --sync-frequency="1m0s"
Oct 30 21:49:07.913873 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:07.906044    2576 flags.go:64] FLAG: --system-cgroups=""
Oct 30 21:49:07.913873 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:07.906047    2576 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Oct 30 21:49:07.913873 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:07.906052    2576 flags.go:64] FLAG: --system-reserved-cgroup=""
Oct 30 21:49:07.913873 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:07.906055    2576 flags.go:64] FLAG: --tls-cert-file=""
Oct 30 21:49:07.913873 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:07.906058    2576 flags.go:64] FLAG: --tls-cipher-suites="[]"
Oct 30 21:49:07.913873 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:07.906063    2576 flags.go:64] FLAG: --tls-min-version=""
Oct 30 21:49:07.913873 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:07.906066    2576 flags.go:64] FLAG: --tls-private-key-file=""
Oct 30 21:49:07.913873 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:07.906069    2576 flags.go:64] FLAG: --topology-manager-policy="none"
Oct 30 21:49:07.913873 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:07.906072    2576 flags.go:64] FLAG: --topology-manager-policy-options=""
Oct 30 21:49:07.913873 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:07.906075    2576 flags.go:64] FLAG: --topology-manager-scope="container"
Oct 30 21:49:07.913873 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:07.906078    2576 flags.go:64] FLAG: --v="2"
Oct 30 21:49:07.913873 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:07.906084    2576 flags.go:64] FLAG: --version="false"
Oct 30 21:49:07.913873 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:07.906088    2576 flags.go:64] FLAG: --vmodule=""
Oct 30 21:49:07.913873 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:07.906093    2576 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Oct 30 21:49:07.913873 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:07.906096    2576 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Oct 30 21:49:07.913873 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906235    2576 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:49:07.915014 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906239    2576 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:49:07.915014 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906243    2576 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:49:07.915014 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906245    2576 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:49:07.915014 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906248    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:49:07.915014 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906251    2576 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:49:07.915014 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906254    2576 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:49:07.915014 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906257    2576 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:49:07.915014 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906259    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:49:07.915014 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906262    2576 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:49:07.915014 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906266    2576 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:49:07.915014 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906268    2576 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:49:07.915014 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906271    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:49:07.915014 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906273    2576 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:49:07.915014 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906276    2576 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:49:07.915014 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906279    2576 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:49:07.915014 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906283    2576 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:49:07.915014 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906285    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:49:07.915014 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906288    2576 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:49:07.915014 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906290    2576 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:49:07.915950 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906293    2576 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:49:07.915950 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906296    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:49:07.915950 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906299    2576 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:49:07.915950 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906301    2576 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:49:07.915950 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906304    2576 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:49:07.915950 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906307    2576 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:49:07.915950 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906309    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:49:07.915950 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906312    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:49:07.915950 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906316    2576 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:49:07.915950 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906319    2576 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:49:07.915950 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906321    2576 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:49:07.915950 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906324    2576 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:49:07.915950 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906326    2576 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:49:07.915950 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906331    2576 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:49:07.915950 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906334    2576 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:49:07.915950 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906338    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:49:07.915950 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906341    2576 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:49:07.915950 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906344    2576 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:49:07.915950 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906347    2576 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:49:07.916677 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906349    2576 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:49:07.916677 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906352    2576 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:49:07.916677 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906355    2576 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:49:07.916677 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906358    2576 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:49:07.916677 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906361    2576 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:49:07.916677 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906364    2576 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:49:07.916677 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906366    2576 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:49:07.916677 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906369    2576 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:49:07.916677 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906371    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:49:07.916677 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906375    2576 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:49:07.916677 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906378    2576 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:49:07.916677 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906381    2576 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:49:07.916677 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906384    2576 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:49:07.916677 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906386    2576 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:49:07.916677 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906389    2576 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:49:07.916677 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906392    2576 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:49:07.916677 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906394    2576 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:49:07.916677 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906397    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:49:07.916677 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906400    2576 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:49:07.916677 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906403    2576 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:49:07.917366 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906406    2576 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:49:07.917366 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906409    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:49:07.917366 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906412    2576 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:49:07.917366 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906415    2576 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:49:07.917366 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906418    2576 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:49:07.917366 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906420    2576 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:49:07.917366 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906423    2576 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:49:07.917366 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906426    2576 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:49:07.917366 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906428    2576 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:49:07.917366 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906431    2576 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:49:07.917366 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906433    2576 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:49:07.917366 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906436    2576 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:49:07.917366 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906439    2576 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:49:07.917366 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906441    2576 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:49:07.917366 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906444    2576 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:49:07.917366 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906446    2576 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:49:07.917366 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906449    2576 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:49:07.917366 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906452    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:49:07.917366 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906455    2576 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:49:07.917366 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906457    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:49:07.918270 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906460    2576 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:49:07.918270 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906463    2576 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:49:07.918270 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906466    2576 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:49:07.918270 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906469    2576 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:49:07.918270 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906471    2576 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:49:07.918270 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906474    2576 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:49:07.918270 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906477    2576 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:49:07.918270 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906479    2576 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:49:07.918270 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906482    2576 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:49:07.918270 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906485    2576 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:49:07.918270 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906487    2576 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:49:07.918270 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906491    2576 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:49:07.918270 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906495    2576 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:49:07.918270 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906500    2576 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:49:07.918270 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906502    2576 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:49:07.918270 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906505    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:49:07.918270 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906508    2576 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:49:07.918270 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906510    2576 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:49:07.918270 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906513    2576 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:49:07.918270 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906516    2576 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:49:07.918830 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906518    2576 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:49:07.918830 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906521    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:49:07.918830 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906524    2576 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:49:07.918830 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906526    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:49:07.918830 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906529    2576 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:49:07.918830 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906531    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:49:07.918830 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906534    2576 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:49:07.918830 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:07.906561    2576 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 21:49:07.918830 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906602    2576 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:49:07.918830 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906606    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:49:07.918830 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906610    2576 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:49:07.918830 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906613    2576 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:49:07.918830 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906616    2576 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:49:07.918830 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906619    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:49:07.918830 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906623    2576 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:49:07.919307 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906626    2576 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:49:07.919307 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906629    2576 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:49:07.919307 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906631    2576 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:49:07.919307 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906634    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:49:07.919307 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906636    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:49:07.919307 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906639    2576 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:49:07.919307 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906642    2576 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:49:07.919307 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906644    2576 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:49:07.919307 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906647    2576 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:49:07.919307 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906649    2576 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:49:07.919307 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906670    2576 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:49:07.919307 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906675    2576 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:49:07.919307 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906677    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:49:07.919307 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906680    2576 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:49:07.919307 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906683    2576 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:49:07.919307 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906686    2576 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:49:07.919307 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906688    2576 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:49:07.919307 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906691    2576 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:49:07.919307 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906693    2576 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:49:07.919779 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906696    2576 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:49:07.919779 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906698    2576 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:49:07.919779 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906701    2576 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:49:07.919779 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906703    2576 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:49:07.919779 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906706    2576 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:49:07.919779 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906708    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:49:07.919779 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906711    2576 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:49:07.919779 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906714    2576 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:49:07.919779 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906717    2576 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:49:07.919779 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906720    2576 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:49:07.919779 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906722    2576 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:49:07.919779 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906725    2576 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:49:07.919779 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906727    2576 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:49:07.919779 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906730    2576 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:49:07.919779 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906733    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:49:07.919779 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906735    2576 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:49:07.919779 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906738    2576 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:49:07.919779 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906741    2576 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:49:07.919779 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906743    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:49:07.919779 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906746    2576 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:49:07.920310 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906748    2576 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:49:07.920310 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906751    2576 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:49:07.920310 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906754    2576 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:49:07.920310 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906756    2576 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:49:07.920310 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906759    2576 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:49:07.920310 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906761    2576 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:49:07.920310 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906765    2576 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:49:07.920310 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906767    2576 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:49:07.920310 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906770    2576 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:49:07.920310 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906773    2576 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:49:07.920310 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906775    2576 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:49:07.920310 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906778    2576 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:49:07.920310 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906780    2576 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:49:07.920310 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906783    2576 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:49:07.920310 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906785    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:49:07.920310 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906788    2576 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:49:07.920310 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906790    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:49:07.920310 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906793    2576 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:49:07.920310 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906796    2576 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:49:07.920310 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906798    2576 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:49:07.920955 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906801    2576 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:49:07.920955 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906803    2576 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:49:07.920955 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906806    2576 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:49:07.920955 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906809    2576 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:49:07.920955 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906811    2576 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:49:07.920955 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906814    2576 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:49:07.920955 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906817    2576 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:49:07.920955 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906819    2576 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:49:07.920955 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906823    2576 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:49:07.920955 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906827    2576 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:49:07.920955 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906830    2576 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:49:07.920955 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906833    2576 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:49:07.920955 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906836    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:49:07.920955 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906839    2576 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:49:07.920955 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906841    2576 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:49:07.920955 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906844    2576 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:49:07.920955 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906846    2576 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:49:07.920955 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906849    2576 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:49:07.920955 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906852    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:49:07.920955 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906855    2576 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:49:07.921430 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906857    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:49:07.921430 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906860    2576 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:49:07.921430 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906862    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:49:07.921430 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906865    2576 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:49:07.921430 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906868    2576 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:49:07.921430 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906870    2576 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:49:07.921430 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906873    2576 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:49:07.921430 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906875    2576 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:49:07.921430 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906878    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:49:07.921430 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906880    2576 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:49:07.921430 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906884    2576 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:49:07.921430 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906886    2576 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:49:07.921430 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906889    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:49:07.921430 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906893    2576 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:49:07.921430 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906895    2576 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:49:07.921430 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906898    2576 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:49:07.921430 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906901    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:49:07.921430 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906904    2576 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:49:07.921430 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906906    2576 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:49:07.921886 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.906909    2576 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:49:07.921886 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:07.914248    2576 server.go:525] "Kubelet version" kubeletVersion="v1.35.9"
Oct 30 21:49:07.921886 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:07.914267    2576 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Oct 30 21:49:07.921886 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:07.914292    2576 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Oct 30 21:49:07.921886 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:07.914299    2576 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Oct 30 21:49:07.921886 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.914384    2576 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:49:07.921886 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.914391    2576 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:49:07.921886 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.914397    2576 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:49:07.921886 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.914402    2576 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:49:07.921886 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.914406    2576 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:49:07.921886 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.914412    2576 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:49:07.921886 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.914416    2576 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:49:07.921886 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.914421    2576 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:49:07.921886 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.914425    2576 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:49:07.921886 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.914430    2576 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:49:07.921886 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.914434    2576 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:49:07.921886 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.914438    2576 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:49:07.921886 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.914443    2576 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:49:07.921886 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.914449    2576 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:49:07.922639 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.914456    2576 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:49:07.922639 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.914460    2576 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:49:07.922639 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.914465    2576 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:49:07.922639 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.914469    2576 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:49:07.922639 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.914473    2576 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:49:07.922639 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.914478    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:49:07.922639 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.914483    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:49:07.922639 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.914487    2576 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:49:07.922639 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.914492    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:49:07.922639 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.914496    2576 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:49:07.922639 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.914501    2576 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:49:07.922639 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.914506    2576 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:49:07.922639 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.914510    2576 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:49:07.922639 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.914515    2576 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:49:07.922639 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.914519    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:49:07.922639 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.914525    2576 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:49:07.922639 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.914530    2576 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:49:07.922639 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.914534    2576 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:49:07.922639 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.914538    2576 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:49:07.922639 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.914543    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:49:07.923479 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.914548    2576 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:49:07.923479 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.914552    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:49:07.923479 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.914557    2576 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:49:07.923479 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.914561    2576 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:49:07.923479 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.914566    2576 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:49:07.923479 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.914571    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:49:07.923479 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.914575    2576 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:49:07.923479 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.914580    2576 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:49:07.923479 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.914584    2576 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:49:07.923479 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.914588    2576 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:49:07.923479 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.914593    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:49:07.923479 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.914599    2576 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:49:07.923479 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.914605    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:49:07.923479 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.914610    2576 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:49:07.923479 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.914614    2576 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:49:07.923479 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.914619    2576 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:49:07.923479 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.914623    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:49:07.923479 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.914628    2576 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:49:07.923479 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.914632    2576 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:49:07.924284 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.914637    2576 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:49:07.924284 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.914641    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:49:07.924284 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.914646    2576 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:49:07.924284 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.914650    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:49:07.924284 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.914672    2576 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:49:07.924284 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.914677    2576 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:49:07.924284 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.914681    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:49:07.924284 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.914686    2576 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:49:07.924284 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.914690    2576 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:49:07.924284 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.914696    2576 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:49:07.924284 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.914700    2576 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:49:07.924284 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.914705    2576 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:49:07.924284 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.914709    2576 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:49:07.924284 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.914713    2576 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:49:07.924284 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.914718    2576 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:49:07.924284 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.914722    2576 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:49:07.924284 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.914726    2576 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:49:07.924284 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.914731    2576 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:49:07.924284 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.914735    2576 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:49:07.924284 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.914740    2576 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:49:07.924989 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.914744    2576 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:49:07.924989 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.914749    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:49:07.924989 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.914754    2576 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:49:07.924989 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.914758    2576 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:49:07.924989 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.914762    2576 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:49:07.924989 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.914767    2576 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:49:07.924989 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.914771    2576 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:49:07.924989 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.914776    2576 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:49:07.924989 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.914782    2576 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:49:07.924989 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.914786    2576 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:49:07.924989 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.914791    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:49:07.924989 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.914796    2576 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:49:07.924989 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.914800    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:49:07.924989 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.914804    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:49:07.924989 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.914809    2576 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:49:07.924989 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.914813    2576 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:49:07.924989 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.914818    2576 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:49:07.924989 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.914823    2576 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:49:07.924989 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.914827    2576 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:49:07.925481 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.914832    2576 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:49:07.925481 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.914836    2576 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:49:07.925481 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.914840    2576 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:49:07.925481 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.914845    2576 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:49:07.925481 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.914850    2576 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:49:07.925481 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.914855    2576 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:49:07.925481 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.914859    2576 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:49:07.925481 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.914863    2576 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:49:07.925481 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.914868    2576 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:49:07.925481 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.914872    2576 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:49:07.925481 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.914876    2576 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:49:07.925481 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.914880    2576 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:49:07.925481 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.914884    2576 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:49:07.925481 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.914889    2576 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:49:07.925481 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:07.914929    2576 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 21:49:07.925481 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.915149    2576 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:49:07.925884 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.915157    2576 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:49:07.925884 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.915162    2576 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:49:07.925884 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.915166    2576 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:49:07.925884 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.915171    2576 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:49:07.925884 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.915176    2576 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:49:07.925884 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.915181    2576 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:49:07.925884 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.915185    2576 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:49:07.925884 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.915189    2576 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:49:07.925884 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.915194    2576 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:49:07.925884 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.915199    2576 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:49:07.925884 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.915204    2576 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:49:07.925884 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.915208    2576 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:49:07.925884 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.915212    2576 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:49:07.925884 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.915217    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:49:07.925884 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.915221    2576 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:49:07.925884 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.915225    2576 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:49:07.925884 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.915232    2576 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:49:07.925884 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.915238    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:49:07.925884 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.915243    2576 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:49:07.925884 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.915248    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:49:07.926350 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.915254    2576 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:49:07.926350 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.915258    2576 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:49:07.926350 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.915262    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:49:07.926350 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.915267    2576 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:49:07.926350 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.915271    2576 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:49:07.926350 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.915275    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:49:07.926350 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.915280    2576 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:49:07.926350 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.915284    2576 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:49:07.926350 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.915288    2576 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:49:07.926350 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.915292    2576 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:49:07.926350 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.915297    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:49:07.926350 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.915301    2576 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:49:07.926350 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.915305    2576 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:49:07.926350 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.915310    2576 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:49:07.926350 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.915314    2576 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:49:07.926350 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.915318    2576 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:49:07.926350 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.915323    2576 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:49:07.926350 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.915328    2576 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:49:07.926350 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.915332    2576 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:49:07.926350 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.915336    2576 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:49:07.926873 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.915341    2576 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:49:07.926873 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.915346    2576 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:49:07.926873 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.915350    2576 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:49:07.926873 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.915354    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:49:07.926873 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.915359    2576 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:49:07.926873 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.915363    2576 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:49:07.926873 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.915367    2576 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:49:07.926873 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.915371    2576 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:49:07.926873 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.915375    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:49:07.926873 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.915380    2576 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:49:07.926873 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.915385    2576 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:49:07.926873 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.915389    2576 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:49:07.926873 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.915393    2576 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:49:07.926873 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.915398    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:49:07.926873 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.915402    2576 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:49:07.926873 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.915407    2576 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:49:07.926873 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.915411    2576 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:49:07.926873 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.915415    2576 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:49:07.926873 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.915419    2576 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:49:07.926873 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.915424    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:49:07.927339 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.915428    2576 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:49:07.927339 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.915432    2576 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:49:07.927339 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.915436    2576 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:49:07.927339 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.915441    2576 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:49:07.927339 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.915445    2576 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:49:07.927339 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.915449    2576 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:49:07.927339 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.915453    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:49:07.927339 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.915458    2576 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:49:07.927339 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.915462    2576 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:49:07.927339 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.915467    2576 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:49:07.927339 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.915471    2576 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:49:07.927339 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.915476    2576 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:49:07.927339 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.915481    2576 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:49:07.927339 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.915485    2576 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:49:07.927339 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.915489    2576 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:49:07.927339 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.915493    2576 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:49:07.927339 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.915497    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:49:07.927339 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.915502    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:49:07.927339 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.915506    2576 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:49:07.927339 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.915511    2576 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:49:07.927874 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.915515    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:49:07.927874 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.915519    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:49:07.927874 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.915524    2576 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:49:07.927874 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.915528    2576 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:49:07.927874 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.915535    2576 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:49:07.927874 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.915540    2576 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:49:07.927874 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.915546    2576 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:49:07.927874 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.915551    2576 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:49:07.927874 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.915555    2576 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:49:07.927874 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.915560    2576 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:49:07.927874 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.915564    2576 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:49:07.927874 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.915568    2576 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:49:07.927874 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.915573    2576 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:49:07.927874 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.915577    2576 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:49:07.927874 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.915582    2576 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:49:07.927874 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.915586    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:49:07.927874 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.915590    2576 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:49:07.927874 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.915594    2576 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:49:07.927874 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.915599    2576 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:49:07.928337 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.915603    2576 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:49:07.928337 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.915608    2576 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:49:07.928337 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.915612    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:49:07.928337 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.915616    2576 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:49:07.928337 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.915620    2576 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:49:07.928337 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.915625    2576 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:49:07.928337 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:07.915721    2576 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 21:49:07.928337 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.915776    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:49:07.928337 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.915782    2576 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:49:07.928337 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.915787    2576 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:49:07.928337 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.915792    2576 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:49:07.928337 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.915796    2576 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:49:07.928337 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.915801    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:49:07.928337 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.915805    2576 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:49:07.928337 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.915809    2576 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:49:07.928337 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.915813    2576 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:49:07.928826 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.915818    2576 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:49:07.928826 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.915822    2576 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:49:07.928826 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.915826    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:49:07.928826 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.915830    2576 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:49:07.928826 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.915835    2576 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:49:07.928826 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.915840    2576 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:49:07.928826 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.915844    2576 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:49:07.928826 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.915848    2576 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:49:07.928826 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.915852    2576 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:49:07.928826 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.915857    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:49:07.928826 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.915861    2576 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:49:07.928826 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.915865    2576 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:49:07.928826 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.915870    2576 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:49:07.928826 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.915874    2576 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:49:07.928826 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.915878    2576 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:49:07.928826 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.915883    2576 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:49:07.928826 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.915887    2576 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:49:07.928826 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.915891    2576 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:49:07.928826 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.915895    2576 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:49:07.928826 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.915899    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:49:07.929290 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.915903    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:49:07.929290 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.915908    2576 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:49:07.929290 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.915915    2576 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:49:07.929290 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.915920    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:49:07.929290 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.915925    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:49:07.929290 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.915929    2576 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:49:07.929290 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.915933    2576 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:49:07.929290 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.915940    2576 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:49:07.929290 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.915945    2576 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:49:07.929290 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.915949    2576 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:49:07.929290 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.915954    2576 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:49:07.929290 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.915958    2576 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:49:07.929290 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.915962    2576 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:49:07.929290 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.915966    2576 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:49:07.929290 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.915970    2576 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:49:07.929290 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.915974    2576 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:49:07.929290 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.915979    2576 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:49:07.929290 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.915984    2576 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:49:07.929842 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.915989    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:49:07.929842 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.915993    2576 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:49:07.929842 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.915997    2576 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:49:07.929842 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.916002    2576 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:49:07.929842 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.916014    2576 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:49:07.929842 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.916019    2576 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:49:07.929842 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.916024    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:49:07.929842 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.916028    2576 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:49:07.929842 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.916032    2576 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:49:07.929842 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.916037    2576 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:49:07.929842 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.916041    2576 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:49:07.929842 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.916045    2576 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:49:07.929842 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.916049    2576 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:49:07.929842 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.916054    2576 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:49:07.929842 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.916058    2576 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:49:07.929842 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.916062    2576 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:49:07.929842 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.916069    2576 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:49:07.929842 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.916074    2576 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:49:07.929842 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.916078    2576 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:49:07.929842 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.916084    2576 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:49:07.930311 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.916088    2576 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:49:07.930311 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.916093    2576 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:49:07.930311 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.916097    2576 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:49:07.930311 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.916102    2576 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:49:07.930311 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.916106    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:49:07.930311 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.916110    2576 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:49:07.930311 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.916115    2576 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:49:07.930311 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.916121    2576 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:49:07.930311 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.916128    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:49:07.930311 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.916133    2576 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:49:07.930311 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.916137    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:49:07.930311 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.916141    2576 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:49:07.930311 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.916151    2576 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:49:07.930311 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.916155    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:49:07.930311 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.916160    2576 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:49:07.930311 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.916165    2576 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:49:07.930311 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.916169    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:49:07.930311 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.916173    2576 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:49:07.930311 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.916178    2576 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:49:07.930845 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.916182    2576 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:49:07.930845 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.916187    2576 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:49:07.930845 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.916191    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:49:07.930845 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.916195    2576 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:49:07.930845 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.916199    2576 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:49:07.930845 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.916203    2576 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:49:07.930845 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.916207    2576 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:49:07.930845 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.916212    2576 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:49:07.930845 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.916216    2576 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:49:07.930845 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.916222    2576 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:49:07.930845 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.916227    2576 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:49:07.930845 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.916231    2576 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:49:07.930845 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.916237    2576 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:49:07.930845 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.916241    2576 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:49:07.930845 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.916246    2576 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:49:07.930845 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.916250    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:49:07.930845 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.916254    2576 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:49:07.930845 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.916259    2576 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:49:07.930845 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.916263    2576 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:49:07.930845 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:07.916267    2576 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:49:07.931437 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:07.916358    2576 server.go:957] "Client rotation is on, will bootstrap in background"
Oct 30 21:49:07.931437 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:07.918846    2576 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Oct 30 21:49:07.931437 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:07.919921    2576 server.go:1014] "Starting client certificate rotation"
Oct 30 21:49:07.931437 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:07.920031    2576 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Oct 30 21:49:07.931437 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:07.920069    2576 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Oct 30 21:49:07.931437 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:07.922526    2576 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 30 21:49:07.931437 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:07.924686    2576 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 30 21:49:07.938606 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:07.938588    2576 log.go:25] "Validated CRI v1 runtime API"
Oct 30 21:49:08.031727 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:08.031701    2576 log.go:25] "Validated CRI v1 image API"
Oct 30 21:49:08.032646 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:08.032618    2576 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Oct 30 21:49:08.035694 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:08.035651    2576 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 825034c1-cd56-4c9c-ad44-16d53b1e630b:/dev/nvme0n1p4 c7fc0637-553e-4526-a80b-df66fd1465c4:/dev/nvme0n1p3]
Oct 30 21:49:08.035694 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:08.035685    2576 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0}]
Oct 30 21:49:08.042786 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:08.042664    2576 manager.go:217] Machine: {Timestamp:2026-10-30 21:49:08.040552981 +0000 UTC m=+0.472501239 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3101515 MemoryCapacity:33164337152 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec26b90f21bde3a07705d9f8f834ca89 SystemUUID:ec26b90f-21bd-e3a0-7705-d9f8f834ca89 BootID:c916d274-bf09-4d8a-a59c-d198f21ad112 Filesystems:[{Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16582168576 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16582168576 Type:vfs Inodes:1048576 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:06:ff:cf:f6:cd:09 Speed:0 Mtu:9001} {Name:ens5 MacAddress:06:ff:cf:f6:cd:09 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:36:ea:c8:db:7a:c5 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164337152 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Oct 30 21:49:08.042786 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:08.042781    2576 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Oct 30 21:49:08.042893 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:08.042860    2576 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Oct 30 21:49:08.044008 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:08.043975    2576 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Oct 30 21:49:08.044172 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:08.044014    2576 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-136-210.us-west-2.compute.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Oct 30 21:49:08.044827 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:08.044816    2576 topology_manager.go:143] "Creating topology manager with none policy"
Oct 30 21:49:08.044857 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:08.044829    2576 container_manager_linux.go:311] "Creating device plugin manager"
Oct 30 21:49:08.044857 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:08.044848    2576 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 30 21:49:08.044931 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:08.044862    2576 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 30 21:49:08.045701 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:08.045691    2576 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager"
Oct 30 21:49:08.049486 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:08.049475    2576 state_mem.go:41] "Initialized" logger="CPUManager state memory"
Oct 30 21:49:08.049603 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:08.049594    2576 server.go:1235] "Using root directory" path="/var/lib/kubelet"
Oct 30 21:49:08.050225 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:08.050215    2576 kubelet.go:493] "Attempting to sync node with API server"
Oct 30 21:49:08.050261 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:08.050232    2576 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests"
Oct 30 21:49:08.050261 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:08.050249    2576 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Oct 30 21:49:08.050314 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:08.050263    2576 kubelet.go:405] "Adding apiserver pod source"
Oct 30 21:49:08.050314 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:08.050275    2576 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Oct 30 21:49:08.050697 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:08.050685    2576 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 30 21:49:08.053816 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:08.053798    2576 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1"
Oct 30 21:49:08.059390 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:08.059376    2576 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 30 21:49:08.060227 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:08.060215    2576 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Oct 30 21:49:08.060275 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:08.060246    2576 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Oct 30 21:49:08.060420 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:08.060409    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Oct 30 21:49:08.060453 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:08.060426    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Oct 30 21:49:08.060453 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:08.060432    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Oct 30 21:49:08.060453 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:08.060438    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Oct 30 21:49:08.060453 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:08.060443    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Oct 30 21:49:08.060453 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:08.060449    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Oct 30 21:49:08.061111 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:08.061102    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Oct 30 21:49:08.061141 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:08.061112    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Oct 30 21:49:08.061141 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:08.061121    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Oct 30 21:49:08.061141 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:08.061127    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Oct 30 21:49:08.061141 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:08.061142    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Oct 30 21:49:08.061334 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:08.061151    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Oct 30 21:49:08.062401 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:08.062391    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Oct 30 21:49:08.062439 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:08.062403    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Oct 30 21:49:08.066359 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:08.066336    2576 server.go:1263] "Started kubelet"
Oct 30 21:49:08.066470 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:08.066377    2576 server.go:182] "Starting to listen" address="0.0.0.0" port=10250
Oct 30 21:49:08.066470 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:08.066385    2576 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Oct 30 21:49:08.066470 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:08.066463    2576 server_v1.go:49] "podresources" method="list" useActivePods=true
Oct 30 21:49:08.067156 ip-10-0-136-210 systemd[1]: Started Kubernetes Kubelet.
Oct 30 21:49:08.067604 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:08.067566    2576 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Oct 30 21:49:08.071241 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:08.071224    2576 server.go:317] "Adding debug handlers to kubelet server"
Oct 30 21:49:08.074249 ip-10-0-136-210 kubenswrapper[2576]: E1030 21:49:08.074229    2576 kubelet.go:1679] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Oct 30 21:49:08.075446 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:08.075433    2576 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Oct 30 21:49:08.076704 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:08.076690    2576 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer"
Oct 30 21:49:08.076858 ip-10-0-136-210 kubenswrapper[2576]: E1030 21:49:08.076834    2576 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-136-210.us-west-2.compute.internal\" not found"
Oct 30 21:49:08.077427 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:08.077408    2576 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Oct 30 21:49:08.077526 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:08.077440    2576 volume_manager.go:309] "The desired_state_of_world populator starts"
Oct 30 21:49:08.077526 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:08.077457    2576 volume_manager.go:311] "Starting Kubelet Volume Manager"
Oct 30 21:49:08.077601 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:08.077526    2576 reconstruct.go:97] "Volume reconstruction finished"
Oct 30 21:49:08.077601 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:08.077537    2576 reconciler.go:29] "Reconciler: start to sync state"
Oct 30 21:49:08.077713 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:08.077701    2576 factory.go:153] Registering CRI-O factory
Oct 30 21:49:08.077752 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:08.077716    2576 factory.go:223] Registration of the crio container factory successfully
Oct 30 21:49:08.077784 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:08.077772    2576 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Oct 30 21:49:08.077784 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:08.077779    2576 factory.go:55] Registering systemd factory
Oct 30 21:49:08.077784 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:08.077784    2576 factory.go:223] Registration of the systemd container factory successfully
Oct 30 21:49:08.077877 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:08.077797    2576 factory.go:103] Registering Raw factory
Oct 30 21:49:08.077877 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:08.077812    2576 manager.go:1199] Started watching for new ooms in manager
Oct 30 21:49:08.078982 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:08.078968    2576 manager.go:319] Starting recovery of all containers
Oct 30 21:49:08.089968 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:08.089953    2576 manager.go:324] Recovery completed
Oct 30 21:49:08.091528 ip-10-0-136-210 kubenswrapper[2576]: E1030 21:49:08.091505    2576 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory
Oct 30 21:49:08.094719 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:08.094707    2576 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 21:49:08.096354 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:08.096334    2576 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-210.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 21:49:08.096405 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:08.096367    2576 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-210.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 21:49:08.096405 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:08.096378    2576 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-210.us-west-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 21:49:08.096896 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:08.096882    2576 cpu_manager.go:226] "Starting" policy="none"
Oct 30 21:49:08.096956 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:08.096896    2576 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s"
Oct 30 21:49:08.096956 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:08.096905    2576 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 21:49:08.096956 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:08.096923    2576 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory"
Oct 30 21:49:08.097873 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:08.097859    2576 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-210.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 21:49:08.097928 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:08.097884    2576 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-210.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 21:49:08.097928 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:08.097895    2576 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-210.us-west-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 21:49:08.099133 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:08.099117    2576 policy_none.go:50] "Start"
Oct 30 21:49:08.099937 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:08.099923    2576 memory_manager.go:187] "Starting memorymanager" policy="None"
Oct 30 21:49:08.100029 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:08.099939    2576 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Oct 30 21:49:08.101971 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:08.101959    2576 policy_none.go:44] "Start"
Oct 30 21:49:08.143237 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:08.143211    2576 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"}
Oct 30 21:49:08.145604 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:08.145553    2576 manager.go:341] "Starting Device Plugin manager"
Oct 30 21:49:08.146154 ip-10-0-136-210 kubenswrapper[2576]: E1030 21:49:08.146137    2576 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Oct 30 21:49:08.146211 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:08.146158    2576 server.go:87] "Starting device plugin registration server"
Oct 30 21:49:08.146456 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:08.146443    2576 eviction_manager.go:194] "Eviction manager: starting control loop"
Oct 30 21:49:08.146496 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:08.146459    2576 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Oct 30 21:49:08.146667 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:08.146636    2576 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Oct 30 21:49:08.146747 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:08.146725    2576 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts"
Oct 30 21:49:08.146747 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:08.146732    2576 plugin_manager.go:121] "Starting Kubelet Plugin Manager"
Oct 30 21:49:08.147083 ip-10-0-136-210 kubenswrapper[2576]: E1030 21:49:08.147051    2576 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Oct 30 21:49:08.147149 ip-10-0-136-210 kubenswrapper[2576]: E1030 21:49:08.147111    2576 eviction_manager.go:297] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-136-210.us-west-2.compute.internal\" not found"
Oct 30 21:49:08.188409 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:08.188393    2576 reflector.go:446] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Oct 30 21:49:08.190248 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:08.188430    2576 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-j88xv"
Oct 30 21:49:08.202861 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:08.202812    2576 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-136-210.us-west-2.compute.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Oct 30 21:49:08.202861 ip-10-0-136-210 kubenswrapper[2576]: E1030 21:49:08.202817    2576 controller.go:201] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-136-210.us-west-2.compute.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Oct 30 21:49:08.204765 ip-10-0-136-210 kubenswrapper[2576]: E1030 21:49:08.202832    2576 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-210.us-west-2.compute.internal.18e36d05bebf8549  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-210.us-west-2.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-136-210.us-west-2.compute.internal,},FirstTimestamp:2026-10-30 21:49:08.066075977 +0000 UTC m=+0.498024236,LastTimestamp:2026-10-30 21:49:08.066075977 +0000 UTC m=+0.498024236,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-210.us-west-2.compute.internal,}"
Oct 30 21:49:08.207175 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:08.207148    2576 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Oct 30 21:49:08.208378 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:08.208362    2576 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Oct 30 21:49:08.208452 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:08.208388    2576 status_manager.go:249] "Starting to sync pod status with apiserver"
Oct 30 21:49:08.208452 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:08.208412    2576 kubelet.go:2521] "Starting kubelet main sync loop"
Oct 30 21:49:08.208526 ip-10-0-136-210 kubenswrapper[2576]: E1030 21:49:08.208452    2576 kubelet.go:2545] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Oct 30 21:49:08.247092 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:08.246929    2576 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 21:49:08.248140 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:08.248125    2576 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-210.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 21:49:08.248207 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:08.248153    2576 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-210.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 21:49:08.248207 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:08.248164    2576 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-210.us-west-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 21:49:08.248207 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:08.248188    2576 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-136-210.us-west-2.compute.internal"
Oct 30 21:49:08.254024 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:08.254010    2576 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 21:49:08.308739 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:08.308710    2576 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-210.us-west-2.compute.internal","kube-system/kube-apiserver-proxy-ip-10-0-136-210.us-west-2.compute.internal"]
Oct 30 21:49:08.308865 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:08.308772    2576 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 21:49:08.310280 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:08.310260    2576 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-210.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 21:49:08.310361 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:08.310292    2576 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-210.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 21:49:08.310361 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:08.310303    2576 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-210.us-west-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 21:49:08.312482 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:08.312469    2576 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 21:49:08.312623 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:08.312609    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-210.us-west-2.compute.internal"
Oct 30 21:49:08.312673 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:08.312641    2576 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 21:49:08.314799 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:08.314785    2576 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-210.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 21:49:08.314799 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:08.314795    2576 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-210.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 21:49:08.314911 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:08.314811    2576 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-210.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 21:49:08.314911 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:08.314823    2576 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-210.us-west-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 21:49:08.314911 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:08.314813    2576 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-210.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 21:49:08.314911 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:08.314892    2576 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-210.us-west-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 21:49:08.316939 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:08.316923    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-210.us-west-2.compute.internal"
Oct 30 21:49:08.316988 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:08.316956    2576 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 21:49:08.317705 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:08.317690    2576 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-210.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 21:49:08.317772 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:08.317716    2576 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-210.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 21:49:08.317772 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:08.317726    2576 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-210.us-west-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 21:49:08.343965 ip-10-0-136-210 kubenswrapper[2576]: E1030 21:49:08.343947    2576 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-136-210.us-west-2.compute.internal\" not found" node="ip-10-0-136-210.us-west-2.compute.internal"
Oct 30 21:49:08.346121 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:08.346092    2576 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 21:49:08.346121 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:08.346102    2576 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 21:49:08.346744 ip-10-0-136-210 kubenswrapper[2576]: E1030 21:49:08.346728    2576 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-136-210.us-west-2.compute.internal\" not found" node="ip-10-0-136-210.us-west-2.compute.internal"
Oct 30 21:49:08.346959 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:08.346940    2576 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-136-210.us-west-2.compute.internal"
Oct 30 21:49:08.347018 ip-10-0-136-210 kubenswrapper[2576]: E1030 21:49:08.346967    2576 kubelet_node_status.go:524] "Error updating node status, will retry" err="error getting node \"ip-10-0-136-210.us-west-2.compute.internal\": node \"ip-10-0-136-210.us-west-2.compute.internal\" not found"
Oct 30 21:49:08.347079 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:08.347060    2576 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 21:49:08.377307 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:08.377289    2576 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-210.us-west-2.compute.internal"
Oct 30 21:49:08.448827 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:08.448757    2576 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 21:49:08.450815 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:08.450800    2576 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-210.us-west-2.compute.internal"
Oct 30 21:49:08.478451 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:08.478429    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/d91a9843014f769d901fa6aba545e397-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-210.us-west-2.compute.internal\" (UID: \"d91a9843014f769d901fa6aba545e397\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-210.us-west-2.compute.internal"
Oct 30 21:49:08.478531 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:08.478459    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d91a9843014f769d901fa6aba545e397-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-210.us-west-2.compute.internal\" (UID: \"d91a9843014f769d901fa6aba545e397\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-210.us-west-2.compute.internal"
Oct 30 21:49:08.478531 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:08.478480    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/4492449458f4d650448405f5044e8d24-config\") pod \"kube-apiserver-proxy-ip-10-0-136-210.us-west-2.compute.internal\" (UID: \"4492449458f4d650448405f5044e8d24\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-210.us-west-2.compute.internal"
Oct 30 21:49:08.519431 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:08.519408    2576 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 21:49:08.578648 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:08.578612    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/d91a9843014f769d901fa6aba545e397-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-210.us-west-2.compute.internal\" (UID: \"d91a9843014f769d901fa6aba545e397\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-210.us-west-2.compute.internal"
Oct 30 21:49:08.578648 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:08.578647    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d91a9843014f769d901fa6aba545e397-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-210.us-west-2.compute.internal\" (UID: \"d91a9843014f769d901fa6aba545e397\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-210.us-west-2.compute.internal"
Oct 30 21:49:08.578863 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:08.578685    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/4492449458f4d650448405f5044e8d24-config\") pod \"kube-apiserver-proxy-ip-10-0-136-210.us-west-2.compute.internal\" (UID: \"4492449458f4d650448405f5044e8d24\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-210.us-west-2.compute.internal"
Oct 30 21:49:08.578863 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:08.578740    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/4492449458f4d650448405f5044e8d24-config\") pod \"kube-apiserver-proxy-ip-10-0-136-210.us-west-2.compute.internal\" (UID: \"4492449458f4d650448405f5044e8d24\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-210.us-west-2.compute.internal"
Oct 30 21:49:08.578863 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:08.578740    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d91a9843014f769d901fa6aba545e397-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-210.us-west-2.compute.internal\" (UID: \"d91a9843014f769d901fa6aba545e397\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-210.us-west-2.compute.internal"
Oct 30 21:49:08.578863 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:08.578739    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/d91a9843014f769d901fa6aba545e397-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-210.us-west-2.compute.internal\" (UID: \"d91a9843014f769d901fa6aba545e397\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-210.us-west-2.compute.internal"
Oct 30 21:49:08.650636 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:08.650591    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-210.us-west-2.compute.internal"
Oct 30 21:49:08.650837 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:08.650772    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-210.us-west-2.compute.internal"
Oct 30 21:49:08.921537 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:08.921503    2576 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Oct 30 21:49:08.922079 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:08.921645    2576 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received"
Oct 30 21:49:08.922079 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:08.921690    2576 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received"
Oct 30 21:49:08.922079 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:08.921734    2576 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received"
Oct 30 21:49:08.922079 ip-10-0-136-210 kubenswrapper[2576]: E1030 21:49:08.921736    2576 event.go:368] "Unable to write event (may retry after sleeping)" err="Patch \"https://a8a1717a8546b4318bd8725a26f164b5-359068c9b1489aca.elb.us-east-2.amazonaws.com:6443/api/v1/namespaces/default/events/ip-10-0-136-210.us-west-2.compute.internal.18e36d05c08df675\": read tcp 10.0.136.210:59194->18.227.113.39:6443: use of closed network connection" event="&Event{ObjectMeta:{ip-10-0-136-210.us-west-2.compute.internal.18e36d05c08df675  default   5252 0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-210.us-west-2.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-136-210.us-west-2.compute.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-136-210.us-west-2.compute.internal,},FirstTimestamp:2026-10-30 21:49:08 +0000 UTC,LastTimestamp:2026-10-30 21:49:08.248168381 +0000 UTC m=+0.680116640,Count:3,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-210.us-west-2.compute.internal,}"
Oct 30 21:49:09.050857 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.050823    2576 apiserver.go:52] "Watching apiserver"
Oct 30 21:49:09.076169 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.076141    2576 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Oct 30 21:49:09.192630 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.192543    2576 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-10-29 21:44:08 +0000 UTC" deadline="2028-05-17 21:15:54.671889892 +0000 UTC"
Oct 30 21:49:09.192630 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.192576    2576 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13559h26m45.479316585s"
Oct 30 21:49:09.237402 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.237373    2576 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Oct 30 21:49:09.237836 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.237817    2576 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-210.us-west-2.compute.internal","openshift-multus/multus-v8j9s","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cqxms","openshift-multus/network-metrics-daemon-vmrkz","openshift-cluster-node-tuning-operator/tuned-ldvhf","openshift-dns/node-resolver-krtnr","openshift-network-operator/iptables-alerter-l9d98","openshift-ovn-kubernetes/ovnkube-node-wrxww","kube-system/global-pull-secret-syncer-68jrf","kube-system/konnectivity-agent-5mw6n","kube-system/kube-apiserver-proxy-ip-10-0-136-210.us-west-2.compute.internal","openshift-image-registry/node-ca-xkrx5","openshift-multus/multus-additional-cni-plugins-kggmc","openshift-network-diagnostics/network-check-target-jmbng"]
Oct 30 21:49:09.240543 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.240519    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-v8j9s"
Oct 30 21:49:09.243798 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.243480    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cqxms"
Oct 30 21:49:09.245948 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.245927    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vmrkz"
Oct 30 21:49:09.246057 ip-10-0-136-210 kubenswrapper[2576]: E1030 21:49:09.245990    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vmrkz" podUID="52eb01f8-e86a-4458-8875-639394a87b30"
Oct 30 21:49:09.248272 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.248257    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-ldvhf"
Oct 30 21:49:09.250393 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.250378    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-krtnr"
Oct 30 21:49:09.252651 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.252633    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-l9d98"
Oct 30 21:49:09.255144 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.255125    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-wrxww"
Oct 30 21:49:09.257459 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.257444    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-68jrf"
Oct 30 21:49:09.257535 ip-10-0-136-210 kubenswrapper[2576]: E1030 21:49:09.257507    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-68jrf" podUID="e810bcd1-78da-409f-b86e-08c47031939e"
Oct 30 21:49:09.259810 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.259792    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-5mw6n"
Oct 30 21:49:09.262106 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.262089    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-xkrx5"
Oct 30 21:49:09.264537 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.264522    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-kggmc"
Oct 30 21:49:09.266920 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.266902    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jmbng"
Oct 30 21:49:09.266993 ip-10-0-136-210 kubenswrapper[2576]: E1030 21:49:09.266964    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jmbng" podUID="73c208eb-ffc0-46f2-8bd4-05ade5adb529"
Oct 30 21:49:09.278792 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.278774    2576 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Oct 30 21:49:09.282400 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.282373    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/c8b13e55-b3c7-4656-9845-16749704265f-host-run-k8s-cni-cncf-io\") pod \"multus-v8j9s\" (UID: \"c8b13e55-b3c7-4656-9845-16749704265f\") " pod="openshift-multus/multus-v8j9s"
Oct 30 21:49:09.282496 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.282413    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c8b13e55-b3c7-4656-9845-16749704265f-host-run-netns\") pod \"multus-v8j9s\" (UID: \"c8b13e55-b3c7-4656-9845-16749704265f\") " pod="openshift-multus/multus-v8j9s"
Oct 30 21:49:09.282496 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.282440    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d5e5ac6e-d37d-457d-9ca7-93065f5925ff-var-lib-openvswitch\") pod \"ovnkube-node-wrxww\" (UID: \"d5e5ac6e-d37d-457d-9ca7-93065f5925ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrxww"
Oct 30 21:49:09.282593 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.282492    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d7c355fa-2141-463e-8cbc-bae3c44a16f1-device-dir\") pod \"aws-ebs-csi-driver-node-cqxms\" (UID: \"d7c355fa-2141-463e-8cbc-bae3c44a16f1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cqxms"
Oct 30 21:49:09.282593 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.282527    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8f2f95c4-21b2-477b-a2cb-7c8590ec340d-sys\") pod \"tuned-ldvhf\" (UID: \"8f2f95c4-21b2-477b-a2cb-7c8590ec340d\") " pod="openshift-cluster-node-tuning-operator/tuned-ldvhf"
Oct 30 21:49:09.282593 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.282555    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d7c355fa-2141-463e-8cbc-bae3c44a16f1-socket-dir\") pod \"aws-ebs-csi-driver-node-cqxms\" (UID: \"d7c355fa-2141-463e-8cbc-bae3c44a16f1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cqxms"
Oct 30 21:49:09.282768 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.282600    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8f2f95c4-21b2-477b-a2cb-7c8590ec340d-lib-modules\") pod \"tuned-ldvhf\" (UID: \"8f2f95c4-21b2-477b-a2cb-7c8590ec340d\") " pod="openshift-cluster-node-tuning-operator/tuned-ldvhf"
Oct 30 21:49:09.282768 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.282641    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/52eb01f8-e86a-4458-8875-639394a87b30-metrics-certs\") pod \"network-metrics-daemon-vmrkz\" (UID: \"52eb01f8-e86a-4458-8875-639394a87b30\") " pod="openshift-multus/network-metrics-daemon-vmrkz"
Oct 30 21:49:09.282768 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.282683    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/8f2f95c4-21b2-477b-a2cb-7c8590ec340d-run\") pod \"tuned-ldvhf\" (UID: \"8f2f95c4-21b2-477b-a2cb-7c8590ec340d\") " pod="openshift-cluster-node-tuning-operator/tuned-ldvhf"
Oct 30 21:49:09.282768 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.282745    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-td6rr\" (UniqueName: \"kubernetes.io/projected/0ab70415-9f1d-4988-a228-df6ed621f0d8-kube-api-access-td6rr\") pod \"iptables-alerter-l9d98\" (UID: \"0ab70415-9f1d-4988-a228-df6ed621f0d8\") " pod="openshift-network-operator/iptables-alerter-l9d98"
Oct 30 21:49:09.282918 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.282782    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/d5e5ac6e-d37d-457d-9ca7-93065f5925ff-ovnkube-config\") pod \"ovnkube-node-wrxww\" (UID: \"d5e5ac6e-d37d-457d-9ca7-93065f5925ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrxww"
Oct 30 21:49:09.282918 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.282809    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/d5e5ac6e-d37d-457d-9ca7-93065f5925ff-env-overrides\") pod \"ovnkube-node-wrxww\" (UID: \"d5e5ac6e-d37d-457d-9ca7-93065f5925ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrxww"
Oct 30 21:49:09.282918 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.282826    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/dc220f0e-8a21-4d7f-96b3-e7055c242f42-cni-binary-copy\") pod \"multus-additional-cni-plugins-kggmc\" (UID: \"dc220f0e-8a21-4d7f-96b3-e7055c242f42\") " pod="openshift-multus/multus-additional-cni-plugins-kggmc"
Oct 30 21:49:09.282918 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.282845    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/dc220f0e-8a21-4d7f-96b3-e7055c242f42-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-kggmc\" (UID: \"dc220f0e-8a21-4d7f-96b3-e7055c242f42\") " pod="openshift-multus/multus-additional-cni-plugins-kggmc"
Oct 30 21:49:09.282918 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.282869    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/c8b13e55-b3c7-4656-9845-16749704265f-host-run-multus-certs\") pod \"multus-v8j9s\" (UID: \"c8b13e55-b3c7-4656-9845-16749704265f\") " pod="openshift-multus/multus-v8j9s"
Oct 30 21:49:09.282918 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.282899    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d7c355fa-2141-463e-8cbc-bae3c44a16f1-etc-selinux\") pod \"aws-ebs-csi-driver-node-cqxms\" (UID: \"d7c355fa-2141-463e-8cbc-bae3c44a16f1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cqxms"
Oct 30 21:49:09.283206 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.282929    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d7c355fa-2141-463e-8cbc-bae3c44a16f1-sys-fs\") pod \"aws-ebs-csi-driver-node-cqxms\" (UID: \"d7c355fa-2141-463e-8cbc-bae3c44a16f1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cqxms"
Oct 30 21:49:09.283206 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.282969    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vg89l\" (UniqueName: \"kubernetes.io/projected/d7c355fa-2141-463e-8cbc-bae3c44a16f1-kube-api-access-vg89l\") pod \"aws-ebs-csi-driver-node-cqxms\" (UID: \"d7c355fa-2141-463e-8cbc-bae3c44a16f1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cqxms"
Oct 30 21:49:09.283206 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.282994    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/c8b13e55-b3c7-4656-9845-16749704265f-multus-daemon-config\") pod \"multus-v8j9s\" (UID: \"c8b13e55-b3c7-4656-9845-16749704265f\") " pod="openshift-multus/multus-v8j9s"
Oct 30 21:49:09.283206 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.283020    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f69b6a2e-d33f-4cc8-8e0a-598b20948f38-agent-certs\") pod \"konnectivity-agent-5mw6n\" (UID: \"f69b6a2e-d33f-4cc8-8e0a-598b20948f38\") " pod="kube-system/konnectivity-agent-5mw6n"
Oct 30 21:49:09.283206 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.283056    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/8f2f95c4-21b2-477b-a2cb-7c8590ec340d-etc-sysctl-conf\") pod \"tuned-ldvhf\" (UID: \"8f2f95c4-21b2-477b-a2cb-7c8590ec340d\") " pod="openshift-cluster-node-tuning-operator/tuned-ldvhf"
Oct 30 21:49:09.283206 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.283097    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t7ldj\" (UniqueName: \"kubernetes.io/projected/8f2f95c4-21b2-477b-a2cb-7c8590ec340d-kube-api-access-t7ldj\") pod \"tuned-ldvhf\" (UID: \"8f2f95c4-21b2-477b-a2cb-7c8590ec340d\") " pod="openshift-cluster-node-tuning-operator/tuned-ldvhf"
Oct 30 21:49:09.283206 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.283137    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c8b13e55-b3c7-4656-9845-16749704265f-cni-binary-copy\") pod \"multus-v8j9s\" (UID: \"c8b13e55-b3c7-4656-9845-16749704265f\") " pod="openshift-multus/multus-v8j9s"
Oct 30 21:49:09.283206 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.283163    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/d5e5ac6e-d37d-457d-9ca7-93065f5925ff-run-ovn\") pod \"ovnkube-node-wrxww\" (UID: \"d5e5ac6e-d37d-457d-9ca7-93065f5925ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrxww"
Oct 30 21:49:09.283206 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.283205    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z2p79\" (UniqueName: \"kubernetes.io/projected/d5e5ac6e-d37d-457d-9ca7-93065f5925ff-kube-api-access-z2p79\") pod \"ovnkube-node-wrxww\" (UID: \"d5e5ac6e-d37d-457d-9ca7-93065f5925ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrxww"
Oct 30 21:49:09.283556 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.283231    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8f2f95c4-21b2-477b-a2cb-7c8590ec340d-var-lib-kubelet\") pod \"tuned-ldvhf\" (UID: \"8f2f95c4-21b2-477b-a2cb-7c8590ec340d\") " pod="openshift-cluster-node-tuning-operator/tuned-ldvhf"
Oct 30 21:49:09.283556 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.283271    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e810bcd1-78da-409f-b86e-08c47031939e-kubelet-config\") pod \"global-pull-secret-syncer-68jrf\" (UID: \"e810bcd1-78da-409f-b86e-08c47031939e\") " pod="kube-system/global-pull-secret-syncer-68jrf"
Oct 30 21:49:09.283556 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.283303    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/dc220f0e-8a21-4d7f-96b3-e7055c242f42-cnibin\") pod \"multus-additional-cni-plugins-kggmc\" (UID: \"dc220f0e-8a21-4d7f-96b3-e7055c242f42\") " pod="openshift-multus/multus-additional-cni-plugins-kggmc"
Oct 30 21:49:09.283556 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.283322    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/dc220f0e-8a21-4d7f-96b3-e7055c242f42-tuning-conf-dir\") pod \"multus-additional-cni-plugins-kggmc\" (UID: \"dc220f0e-8a21-4d7f-96b3-e7055c242f42\") " pod="openshift-multus/multus-additional-cni-plugins-kggmc"
Oct 30 21:49:09.283556 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.283352    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fcjg4\" (UniqueName: \"kubernetes.io/projected/73c208eb-ffc0-46f2-8bd4-05ade5adb529-kube-api-access-fcjg4\") pod \"network-check-target-jmbng\" (UID: \"73c208eb-ffc0-46f2-8bd4-05ade5adb529\") " pod="openshift-network-diagnostics/network-check-target-jmbng"
Oct 30 21:49:09.283556 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.283371    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8f2f95c4-21b2-477b-a2cb-7c8590ec340d-tmp\") pod \"tuned-ldvhf\" (UID: \"8f2f95c4-21b2-477b-a2cb-7c8590ec340d\") " pod="openshift-cluster-node-tuning-operator/tuned-ldvhf"
Oct 30 21:49:09.283556 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.283392    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d7c355fa-2141-463e-8cbc-bae3c44a16f1-registration-dir\") pod \"aws-ebs-csi-driver-node-cqxms\" (UID: \"d7c355fa-2141-463e-8cbc-bae3c44a16f1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cqxms"
Oct 30 21:49:09.283556 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.283408    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/8f2f95c4-21b2-477b-a2cb-7c8590ec340d-etc-sysctl-d\") pod \"tuned-ldvhf\" (UID: \"8f2f95c4-21b2-477b-a2cb-7c8590ec340d\") " pod="openshift-cluster-node-tuning-operator/tuned-ldvhf"
Oct 30 21:49:09.283556 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.283461    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9krk9\" (UniqueName: \"kubernetes.io/projected/db4b06a3-45e5-4d45-9d7a-407d117d9454-kube-api-access-9krk9\") pod \"node-resolver-krtnr\" (UID: \"db4b06a3-45e5-4d45-9d7a-407d117d9454\") " pod="openshift-dns/node-resolver-krtnr"
Oct 30 21:49:09.283556 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.283510    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/04465eda-e6db-42d3-8d5f-c9be668038a9-host\") pod \"node-ca-xkrx5\" (UID: \"04465eda-e6db-42d3-8d5f-c9be668038a9\") " pod="openshift-image-registry/node-ca-xkrx5"
Oct 30 21:49:09.283556 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.283535    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v4d5j\" (UniqueName: \"kubernetes.io/projected/c8b13e55-b3c7-4656-9845-16749704265f-kube-api-access-v4d5j\") pod \"multus-v8j9s\" (UID: \"c8b13e55-b3c7-4656-9845-16749704265f\") " pod="openshift-multus/multus-v8j9s"
Oct 30 21:49:09.283990 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.283575    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/0ab70415-9f1d-4988-a228-df6ed621f0d8-iptables-alerter-script\") pod \"iptables-alerter-l9d98\" (UID: \"0ab70415-9f1d-4988-a228-df6ed621f0d8\") " pod="openshift-network-operator/iptables-alerter-l9d98"
Oct 30 21:49:09.283990 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.283594    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dc220f0e-8a21-4d7f-96b3-e7055c242f42-system-cni-dir\") pod \"multus-additional-cni-plugins-kggmc\" (UID: \"dc220f0e-8a21-4d7f-96b3-e7055c242f42\") " pod="openshift-multus/multus-additional-cni-plugins-kggmc"
Oct 30 21:49:09.283990 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.283623    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f69b6a2e-d33f-4cc8-8e0a-598b20948f38-konnectivity-ca\") pod \"konnectivity-agent-5mw6n\" (UID: \"f69b6a2e-d33f-4cc8-8e0a-598b20948f38\") " pod="kube-system/konnectivity-agent-5mw6n"
Oct 30 21:49:09.283990 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.283650    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/8f2f95c4-21b2-477b-a2cb-7c8590ec340d-etc-tuned\") pod \"tuned-ldvhf\" (UID: \"8f2f95c4-21b2-477b-a2cb-7c8590ec340d\") " pod="openshift-cluster-node-tuning-operator/tuned-ldvhf"
Oct 30 21:49:09.283990 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.283718    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/c8b13e55-b3c7-4656-9845-16749704265f-host-var-lib-cni-multus\") pod \"multus-v8j9s\" (UID: \"c8b13e55-b3c7-4656-9845-16749704265f\") " pod="openshift-multus/multus-v8j9s"
Oct 30 21:49:09.283990 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.283741    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d5e5ac6e-d37d-457d-9ca7-93065f5925ff-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-wrxww\" (UID: \"d5e5ac6e-d37d-457d-9ca7-93065f5925ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrxww"
Oct 30 21:49:09.283990 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.283784    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d7c355fa-2141-463e-8cbc-bae3c44a16f1-kubelet-dir\") pod \"aws-ebs-csi-driver-node-cqxms\" (UID: \"d7c355fa-2141-463e-8cbc-bae3c44a16f1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cqxms"
Oct 30 21:49:09.283990 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.283799    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/8f2f95c4-21b2-477b-a2cb-7c8590ec340d-etc-modprobe-d\") pod \"tuned-ldvhf\" (UID: \"8f2f95c4-21b2-477b-a2cb-7c8590ec340d\") " pod="openshift-cluster-node-tuning-operator/tuned-ldvhf"
Oct 30 21:49:09.283990 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.283850    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/8f2f95c4-21b2-477b-a2cb-7c8590ec340d-etc-systemd\") pod \"tuned-ldvhf\" (UID: \"8f2f95c4-21b2-477b-a2cb-7c8590ec340d\") " pod="openshift-cluster-node-tuning-operator/tuned-ldvhf"
Oct 30 21:49:09.283990 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.283869    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c8b13e55-b3c7-4656-9845-16749704265f-cnibin\") pod \"multus-v8j9s\" (UID: \"c8b13e55-b3c7-4656-9845-16749704265f\") " pod="openshift-multus/multus-v8j9s"
Oct 30 21:49:09.283990 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.283884    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d5e5ac6e-d37d-457d-9ca7-93065f5925ff-etc-openvswitch\") pod \"ovnkube-node-wrxww\" (UID: \"d5e5ac6e-d37d-457d-9ca7-93065f5925ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrxww"
Oct 30 21:49:09.283990 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.283899    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/d5e5ac6e-d37d-457d-9ca7-93065f5925ff-log-socket\") pod \"ovnkube-node-wrxww\" (UID: \"d5e5ac6e-d37d-457d-9ca7-93065f5925ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrxww"
Oct 30 21:49:09.283990 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.283914    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e810bcd1-78da-409f-b86e-08c47031939e-dbus\") pod \"global-pull-secret-syncer-68jrf\" (UID: \"e810bcd1-78da-409f-b86e-08c47031939e\") " pod="kube-system/global-pull-secret-syncer-68jrf"
Oct 30 21:49:09.283990 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.283959    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e810bcd1-78da-409f-b86e-08c47031939e-original-pull-secret\") pod \"global-pull-secret-syncer-68jrf\" (UID: \"e810bcd1-78da-409f-b86e-08c47031939e\") " pod="kube-system/global-pull-secret-syncer-68jrf"
Oct 30 21:49:09.283990 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.283997    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/04465eda-e6db-42d3-8d5f-c9be668038a9-serviceca\") pod \"node-ca-xkrx5\" (UID: \"04465eda-e6db-42d3-8d5f-c9be668038a9\") " pod="openshift-image-registry/node-ca-xkrx5"
Oct 30 21:49:09.284634 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.284040    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/c8b13e55-b3c7-4656-9845-16749704265f-hostroot\") pod \"multus-v8j9s\" (UID: \"c8b13e55-b3c7-4656-9845-16749704265f\") " pod="openshift-multus/multus-v8j9s"
Oct 30 21:49:09.284634 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.284073    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d5e5ac6e-d37d-457d-9ca7-93065f5925ff-host-run-netns\") pod \"ovnkube-node-wrxww\" (UID: \"d5e5ac6e-d37d-457d-9ca7-93065f5925ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrxww"
Oct 30 21:49:09.284634 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.284109    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/d5e5ac6e-d37d-457d-9ca7-93065f5925ff-run-systemd\") pod \"ovnkube-node-wrxww\" (UID: \"d5e5ac6e-d37d-457d-9ca7-93065f5925ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrxww"
Oct 30 21:49:09.284634 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.284170    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d5e5ac6e-d37d-457d-9ca7-93065f5925ff-run-openvswitch\") pod \"ovnkube-node-wrxww\" (UID: \"d5e5ac6e-d37d-457d-9ca7-93065f5925ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrxww"
Oct 30 21:49:09.284634 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.284200    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/d5e5ac6e-d37d-457d-9ca7-93065f5925ff-ovnkube-script-lib\") pod \"ovnkube-node-wrxww\" (UID: \"d5e5ac6e-d37d-457d-9ca7-93065f5925ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrxww"
Oct 30 21:49:09.284634 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.284226    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8f2f95c4-21b2-477b-a2cb-7c8590ec340d-host\") pod \"tuned-ldvhf\" (UID: \"8f2f95c4-21b2-477b-a2cb-7c8590ec340d\") " pod="openshift-cluster-node-tuning-operator/tuned-ldvhf"
Oct 30 21:49:09.284634 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.284255    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c8b13e55-b3c7-4656-9845-16749704265f-system-cni-dir\") pod \"multus-v8j9s\" (UID: \"c8b13e55-b3c7-4656-9845-16749704265f\") " pod="openshift-multus/multus-v8j9s"
Oct 30 21:49:09.284634 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.284279    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d5e5ac6e-d37d-457d-9ca7-93065f5925ff-host-slash\") pod \"ovnkube-node-wrxww\" (UID: \"d5e5ac6e-d37d-457d-9ca7-93065f5925ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrxww"
Oct 30 21:49:09.284634 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.284294    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/d5e5ac6e-d37d-457d-9ca7-93065f5925ff-node-log\") pod \"ovnkube-node-wrxww\" (UID: \"d5e5ac6e-d37d-457d-9ca7-93065f5925ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrxww"
Oct 30 21:49:09.284634 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.284340    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/dc220f0e-8a21-4d7f-96b3-e7055c242f42-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-kggmc\" (UID: \"dc220f0e-8a21-4d7f-96b3-e7055c242f42\") " pod="openshift-multus/multus-additional-cni-plugins-kggmc"
Oct 30 21:49:09.284634 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.284381    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/db4b06a3-45e5-4d45-9d7a-407d117d9454-hosts-file\") pod \"node-resolver-krtnr\" (UID: \"db4b06a3-45e5-4d45-9d7a-407d117d9454\") " pod="openshift-dns/node-resolver-krtnr"
Oct 30 21:49:09.284634 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.284426    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hrxwg\" (UniqueName: \"kubernetes.io/projected/04465eda-e6db-42d3-8d5f-c9be668038a9-kube-api-access-hrxwg\") pod \"node-ca-xkrx5\" (UID: \"04465eda-e6db-42d3-8d5f-c9be668038a9\") " pod="openshift-image-registry/node-ca-xkrx5"
Oct 30 21:49:09.284634 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.284452    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d5e5ac6e-d37d-457d-9ca7-93065f5925ff-host-run-ovn-kubernetes\") pod \"ovnkube-node-wrxww\" (UID: \"d5e5ac6e-d37d-457d-9ca7-93065f5925ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrxww"
Oct 30 21:49:09.284634 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.284477    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c8b13e55-b3c7-4656-9845-16749704265f-host-var-lib-kubelet\") pod \"multus-v8j9s\" (UID: \"c8b13e55-b3c7-4656-9845-16749704265f\") " pod="openshift-multus/multus-v8j9s"
Oct 30 21:49:09.284634 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.284503    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c8b13e55-b3c7-4656-9845-16749704265f-multus-conf-dir\") pod \"multus-v8j9s\" (UID: \"c8b13e55-b3c7-4656-9845-16749704265f\") " pod="openshift-multus/multus-v8j9s"
Oct 30 21:49:09.284634 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.284530    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/d5e5ac6e-d37d-457d-9ca7-93065f5925ff-host-kubelet\") pod \"ovnkube-node-wrxww\" (UID: \"d5e5ac6e-d37d-457d-9ca7-93065f5925ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrxww"
Oct 30 21:49:09.284634 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.284562    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/d5e5ac6e-d37d-457d-9ca7-93065f5925ff-systemd-units\") pod \"ovnkube-node-wrxww\" (UID: \"d5e5ac6e-d37d-457d-9ca7-93065f5925ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrxww"
Oct 30 21:49:09.285268 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.284580    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/8f2f95c4-21b2-477b-a2cb-7c8590ec340d-etc-sysconfig\") pod \"tuned-ldvhf\" (UID: \"8f2f95c4-21b2-477b-a2cb-7c8590ec340d\") " pod="openshift-cluster-node-tuning-operator/tuned-ldvhf"
Oct 30 21:49:09.285268 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.284604    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/db4b06a3-45e5-4d45-9d7a-407d117d9454-tmp-dir\") pod \"node-resolver-krtnr\" (UID: \"db4b06a3-45e5-4d45-9d7a-407d117d9454\") " pod="openshift-dns/node-resolver-krtnr"
Oct 30 21:49:09.285268 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.284628    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/c8b13e55-b3c7-4656-9845-16749704265f-multus-socket-dir-parent\") pod \"multus-v8j9s\" (UID: \"c8b13e55-b3c7-4656-9845-16749704265f\") " pod="openshift-multus/multus-v8j9s"
Oct 30 21:49:09.285268 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.284646    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d5e5ac6e-d37d-457d-9ca7-93065f5925ff-host-cni-bin\") pod \"ovnkube-node-wrxww\" (UID: \"d5e5ac6e-d37d-457d-9ca7-93065f5925ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrxww"
Oct 30 21:49:09.285268 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.284676    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/d5e5ac6e-d37d-457d-9ca7-93065f5925ff-ovn-node-metrics-cert\") pod \"ovnkube-node-wrxww\" (UID: \"d5e5ac6e-d37d-457d-9ca7-93065f5925ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrxww"
Oct 30 21:49:09.285268 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.284696    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8f2f95c4-21b2-477b-a2cb-7c8590ec340d-etc-kubernetes\") pod \"tuned-ldvhf\" (UID: \"8f2f95c4-21b2-477b-a2cb-7c8590ec340d\") " pod="openshift-cluster-node-tuning-operator/tuned-ldvhf"
Oct 30 21:49:09.285268 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.284744    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c8b13e55-b3c7-4656-9845-16749704265f-host-var-lib-cni-bin\") pod \"multus-v8j9s\" (UID: \"c8b13e55-b3c7-4656-9845-16749704265f\") " pod="openshift-multus/multus-v8j9s"
Oct 30 21:49:09.285268 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.284763    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c8b13e55-b3c7-4656-9845-16749704265f-etc-kubernetes\") pod \"multus-v8j9s\" (UID: \"c8b13e55-b3c7-4656-9845-16749704265f\") " pod="openshift-multus/multus-v8j9s"
Oct 30 21:49:09.285268 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.284793    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0ab70415-9f1d-4988-a228-df6ed621f0d8-host-slash\") pod \"iptables-alerter-l9d98\" (UID: \"0ab70415-9f1d-4988-a228-df6ed621f0d8\") " pod="openshift-network-operator/iptables-alerter-l9d98"
Oct 30 21:49:09.285268 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.284827    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/d5e5ac6e-d37d-457d-9ca7-93065f5925ff-host-cni-netd\") pod \"ovnkube-node-wrxww\" (UID: \"d5e5ac6e-d37d-457d-9ca7-93065f5925ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrxww"
Oct 30 21:49:09.285268 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.284858    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tq25h\" (UniqueName: \"kubernetes.io/projected/dc220f0e-8a21-4d7f-96b3-e7055c242f42-kube-api-access-tq25h\") pod \"multus-additional-cni-plugins-kggmc\" (UID: \"dc220f0e-8a21-4d7f-96b3-e7055c242f42\") " pod="openshift-multus/multus-additional-cni-plugins-kggmc"
Oct 30 21:49:09.285268 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.284890    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zxwjt\" (UniqueName: \"kubernetes.io/projected/52eb01f8-e86a-4458-8875-639394a87b30-kube-api-access-zxwjt\") pod \"network-metrics-daemon-vmrkz\" (UID: \"52eb01f8-e86a-4458-8875-639394a87b30\") " pod="openshift-multus/network-metrics-daemon-vmrkz"
Oct 30 21:49:09.285268 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.284907    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c8b13e55-b3c7-4656-9845-16749704265f-multus-cni-dir\") pod \"multus-v8j9s\" (UID: \"c8b13e55-b3c7-4656-9845-16749704265f\") " pod="openshift-multus/multus-v8j9s"
Oct 30 21:49:09.295007 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.294979    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Oct 30 21:49:09.295007 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.294998    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-92jf8\""
Oct 30 21:49:09.295130 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.295001    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Oct 30 21:49:09.295941 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.295924    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Oct 30 21:49:09.296036 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.296020    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Oct 30 21:49:09.296551 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.296528    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Oct 30 21:49:09.296623 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.296576    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Oct 30 21:49:09.296837 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.296822    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-gbc4r\""
Oct 30 21:49:09.296915 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.296894    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Oct 30 21:49:09.301758 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.301739    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-7d2ft\""
Oct 30 21:49:09.301878 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.301861    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Oct 30 21:49:09.301972 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.301957    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Oct 30 21:49:09.303327 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.303309    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Oct 30 21:49:09.303398 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.303348    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-6gwn9\""
Oct 30 21:49:09.303398 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.303380    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Oct 30 21:49:09.305896 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.305882    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Oct 30 21:49:09.306743 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.306713    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Oct 30 21:49:09.306876 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.306864    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Oct 30 21:49:09.306968 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.306954    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-nwhdt\""
Oct 30 21:49:09.345627 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.345608    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-hm4gw\""
Oct 30 21:49:09.346067 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.346052    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Oct 30 21:49:09.385195 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.385171    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/8f2f95c4-21b2-477b-a2cb-7c8590ec340d-etc-sysconfig\") pod \"tuned-ldvhf\" (UID: \"8f2f95c4-21b2-477b-a2cb-7c8590ec340d\") " pod="openshift-cluster-node-tuning-operator/tuned-ldvhf"
Oct 30 21:49:09.385311 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.385216    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/db4b06a3-45e5-4d45-9d7a-407d117d9454-tmp-dir\") pod \"node-resolver-krtnr\" (UID: \"db4b06a3-45e5-4d45-9d7a-407d117d9454\") " pod="openshift-dns/node-resolver-krtnr"
Oct 30 21:49:09.385311 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.385248    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/c8b13e55-b3c7-4656-9845-16749704265f-multus-socket-dir-parent\") pod \"multus-v8j9s\" (UID: \"c8b13e55-b3c7-4656-9845-16749704265f\") " pod="openshift-multus/multus-v8j9s"
Oct 30 21:49:09.385311 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.385260    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/8f2f95c4-21b2-477b-a2cb-7c8590ec340d-etc-sysconfig\") pod \"tuned-ldvhf\" (UID: \"8f2f95c4-21b2-477b-a2cb-7c8590ec340d\") " pod="openshift-cluster-node-tuning-operator/tuned-ldvhf"
Oct 30 21:49:09.385574 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.385274    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d5e5ac6e-d37d-457d-9ca7-93065f5925ff-host-cni-bin\") pod \"ovnkube-node-wrxww\" (UID: \"d5e5ac6e-d37d-457d-9ca7-93065f5925ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrxww"
Oct 30 21:49:09.385784 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.385743    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/d5e5ac6e-d37d-457d-9ca7-93065f5925ff-ovn-node-metrics-cert\") pod \"ovnkube-node-wrxww\" (UID: \"d5e5ac6e-d37d-457d-9ca7-93065f5925ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrxww"
Oct 30 21:49:09.385889 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.385792    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8f2f95c4-21b2-477b-a2cb-7c8590ec340d-etc-kubernetes\") pod \"tuned-ldvhf\" (UID: \"8f2f95c4-21b2-477b-a2cb-7c8590ec340d\") " pod="openshift-cluster-node-tuning-operator/tuned-ldvhf"
Oct 30 21:49:09.385889 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.385833    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c8b13e55-b3c7-4656-9845-16749704265f-host-var-lib-cni-bin\") pod \"multus-v8j9s\" (UID: \"c8b13e55-b3c7-4656-9845-16749704265f\") " pod="openshift-multus/multus-v8j9s"
Oct 30 21:49:09.385889 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.385857    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d5e5ac6e-d37d-457d-9ca7-93065f5925ff-host-cni-bin\") pod \"ovnkube-node-wrxww\" (UID: \"d5e5ac6e-d37d-457d-9ca7-93065f5925ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrxww"
Oct 30 21:49:09.385889 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.385873    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c8b13e55-b3c7-4656-9845-16749704265f-etc-kubernetes\") pod \"multus-v8j9s\" (UID: \"c8b13e55-b3c7-4656-9845-16749704265f\") " pod="openshift-multus/multus-v8j9s"
Oct 30 21:49:09.386091 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.385896    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/c8b13e55-b3c7-4656-9845-16749704265f-multus-socket-dir-parent\") pod \"multus-v8j9s\" (UID: \"c8b13e55-b3c7-4656-9845-16749704265f\") " pod="openshift-multus/multus-v8j9s"
Oct 30 21:49:09.386091 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.385957    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c8b13e55-b3c7-4656-9845-16749704265f-etc-kubernetes\") pod \"multus-v8j9s\" (UID: \"c8b13e55-b3c7-4656-9845-16749704265f\") " pod="openshift-multus/multus-v8j9s"
Oct 30 21:49:09.386091 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.385961    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8f2f95c4-21b2-477b-a2cb-7c8590ec340d-etc-kubernetes\") pod \"tuned-ldvhf\" (UID: \"8f2f95c4-21b2-477b-a2cb-7c8590ec340d\") " pod="openshift-cluster-node-tuning-operator/tuned-ldvhf"
Oct 30 21:49:09.386091 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.385966    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c8b13e55-b3c7-4656-9845-16749704265f-host-var-lib-cni-bin\") pod \"multus-v8j9s\" (UID: \"c8b13e55-b3c7-4656-9845-16749704265f\") " pod="openshift-multus/multus-v8j9s"
Oct 30 21:49:09.386091 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.385998    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0ab70415-9f1d-4988-a228-df6ed621f0d8-host-slash\") pod \"iptables-alerter-l9d98\" (UID: \"0ab70415-9f1d-4988-a228-df6ed621f0d8\") " pod="openshift-network-operator/iptables-alerter-l9d98"
Oct 30 21:49:09.386091 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.386035    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0ab70415-9f1d-4988-a228-df6ed621f0d8-host-slash\") pod \"iptables-alerter-l9d98\" (UID: \"0ab70415-9f1d-4988-a228-df6ed621f0d8\") " pod="openshift-network-operator/iptables-alerter-l9d98"
Oct 30 21:49:09.386091 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.386041    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/db4b06a3-45e5-4d45-9d7a-407d117d9454-tmp-dir\") pod \"node-resolver-krtnr\" (UID: \"db4b06a3-45e5-4d45-9d7a-407d117d9454\") " pod="openshift-dns/node-resolver-krtnr"
Oct 30 21:49:09.386091 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.386061    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/d5e5ac6e-d37d-457d-9ca7-93065f5925ff-host-cni-netd\") pod \"ovnkube-node-wrxww\" (UID: \"d5e5ac6e-d37d-457d-9ca7-93065f5925ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrxww"
Oct 30 21:49:09.386472 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.386118    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tq25h\" (UniqueName: \"kubernetes.io/projected/dc220f0e-8a21-4d7f-96b3-e7055c242f42-kube-api-access-tq25h\") pod \"multus-additional-cni-plugins-kggmc\" (UID: \"dc220f0e-8a21-4d7f-96b3-e7055c242f42\") " pod="openshift-multus/multus-additional-cni-plugins-kggmc"
Oct 30 21:49:09.386472 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.386152    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zxwjt\" (UniqueName: \"kubernetes.io/projected/52eb01f8-e86a-4458-8875-639394a87b30-kube-api-access-zxwjt\") pod \"network-metrics-daemon-vmrkz\" (UID: \"52eb01f8-e86a-4458-8875-639394a87b30\") " pod="openshift-multus/network-metrics-daemon-vmrkz"
Oct 30 21:49:09.386472 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.386156    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/d5e5ac6e-d37d-457d-9ca7-93065f5925ff-host-cni-netd\") pod \"ovnkube-node-wrxww\" (UID: \"d5e5ac6e-d37d-457d-9ca7-93065f5925ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrxww"
Oct 30 21:49:09.386472 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.386199    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c8b13e55-b3c7-4656-9845-16749704265f-multus-cni-dir\") pod \"multus-v8j9s\" (UID: \"c8b13e55-b3c7-4656-9845-16749704265f\") " pod="openshift-multus/multus-v8j9s"
Oct 30 21:49:09.386472 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.386221    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/c8b13e55-b3c7-4656-9845-16749704265f-host-run-k8s-cni-cncf-io\") pod \"multus-v8j9s\" (UID: \"c8b13e55-b3c7-4656-9845-16749704265f\") " pod="openshift-multus/multus-v8j9s"
Oct 30 21:49:09.386472 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.386270    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c8b13e55-b3c7-4656-9845-16749704265f-host-run-netns\") pod \"multus-v8j9s\" (UID: \"c8b13e55-b3c7-4656-9845-16749704265f\") " pod="openshift-multus/multus-v8j9s"
Oct 30 21:49:09.386472 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.386312    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d5e5ac6e-d37d-457d-9ca7-93065f5925ff-var-lib-openvswitch\") pod \"ovnkube-node-wrxww\" (UID: \"d5e5ac6e-d37d-457d-9ca7-93065f5925ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrxww"
Oct 30 21:49:09.386472 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.386319    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c8b13e55-b3c7-4656-9845-16749704265f-multus-cni-dir\") pod \"multus-v8j9s\" (UID: \"c8b13e55-b3c7-4656-9845-16749704265f\") " pod="openshift-multus/multus-v8j9s"
Oct 30 21:49:09.386472 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.386353    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d7c355fa-2141-463e-8cbc-bae3c44a16f1-device-dir\") pod \"aws-ebs-csi-driver-node-cqxms\" (UID: \"d7c355fa-2141-463e-8cbc-bae3c44a16f1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cqxms"
Oct 30 21:49:09.386472 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.386375    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/c8b13e55-b3c7-4656-9845-16749704265f-host-run-k8s-cni-cncf-io\") pod \"multus-v8j9s\" (UID: \"c8b13e55-b3c7-4656-9845-16749704265f\") " pod="openshift-multus/multus-v8j9s"
Oct 30 21:49:09.386472 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.386379    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c8b13e55-b3c7-4656-9845-16749704265f-host-run-netns\") pod \"multus-v8j9s\" (UID: \"c8b13e55-b3c7-4656-9845-16749704265f\") " pod="openshift-multus/multus-v8j9s"
Oct 30 21:49:09.386472 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.386446    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d7c355fa-2141-463e-8cbc-bae3c44a16f1-device-dir\") pod \"aws-ebs-csi-driver-node-cqxms\" (UID: \"d7c355fa-2141-463e-8cbc-bae3c44a16f1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cqxms"
Oct 30 21:49:09.387065 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.386527    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8f2f95c4-21b2-477b-a2cb-7c8590ec340d-sys\") pod \"tuned-ldvhf\" (UID: \"8f2f95c4-21b2-477b-a2cb-7c8590ec340d\") " pod="openshift-cluster-node-tuning-operator/tuned-ldvhf"
Oct 30 21:49:09.387065 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.386581    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8f2f95c4-21b2-477b-a2cb-7c8590ec340d-sys\") pod \"tuned-ldvhf\" (UID: \"8f2f95c4-21b2-477b-a2cb-7c8590ec340d\") " pod="openshift-cluster-node-tuning-operator/tuned-ldvhf"
Oct 30 21:49:09.387065 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.386627    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d5e5ac6e-d37d-457d-9ca7-93065f5925ff-var-lib-openvswitch\") pod \"ovnkube-node-wrxww\" (UID: \"d5e5ac6e-d37d-457d-9ca7-93065f5925ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrxww"
Oct 30 21:49:09.387065 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.386634    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d7c355fa-2141-463e-8cbc-bae3c44a16f1-socket-dir\") pod \"aws-ebs-csi-driver-node-cqxms\" (UID: \"d7c355fa-2141-463e-8cbc-bae3c44a16f1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cqxms"
Oct 30 21:49:09.387065 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.386670    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8f2f95c4-21b2-477b-a2cb-7c8590ec340d-lib-modules\") pod \"tuned-ldvhf\" (UID: \"8f2f95c4-21b2-477b-a2cb-7c8590ec340d\") " pod="openshift-cluster-node-tuning-operator/tuned-ldvhf"
Oct 30 21:49:09.387065 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.386718    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/52eb01f8-e86a-4458-8875-639394a87b30-metrics-certs\") pod \"network-metrics-daemon-vmrkz\" (UID: \"52eb01f8-e86a-4458-8875-639394a87b30\") " pod="openshift-multus/network-metrics-daemon-vmrkz"
Oct 30 21:49:09.387065 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.386751    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/8f2f95c4-21b2-477b-a2cb-7c8590ec340d-run\") pod \"tuned-ldvhf\" (UID: \"8f2f95c4-21b2-477b-a2cb-7c8590ec340d\") " pod="openshift-cluster-node-tuning-operator/tuned-ldvhf"
Oct 30 21:49:09.387065 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.386783    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d7c355fa-2141-463e-8cbc-bae3c44a16f1-socket-dir\") pod \"aws-ebs-csi-driver-node-cqxms\" (UID: \"d7c355fa-2141-463e-8cbc-bae3c44a16f1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cqxms"
Oct 30 21:49:09.387065 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.386810    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8f2f95c4-21b2-477b-a2cb-7c8590ec340d-lib-modules\") pod \"tuned-ldvhf\" (UID: \"8f2f95c4-21b2-477b-a2cb-7c8590ec340d\") " pod="openshift-cluster-node-tuning-operator/tuned-ldvhf"
Oct 30 21:49:09.387065 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.386788    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-td6rr\" (UniqueName: \"kubernetes.io/projected/0ab70415-9f1d-4988-a228-df6ed621f0d8-kube-api-access-td6rr\") pod \"iptables-alerter-l9d98\" (UID: \"0ab70415-9f1d-4988-a228-df6ed621f0d8\") " pod="openshift-network-operator/iptables-alerter-l9d98"
Oct 30 21:49:09.387065 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.386902    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/d5e5ac6e-d37d-457d-9ca7-93065f5925ff-ovnkube-config\") pod \"ovnkube-node-wrxww\" (UID: \"d5e5ac6e-d37d-457d-9ca7-93065f5925ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrxww"
Oct 30 21:49:09.387065 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.386912    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/8f2f95c4-21b2-477b-a2cb-7c8590ec340d-run\") pod \"tuned-ldvhf\" (UID: \"8f2f95c4-21b2-477b-a2cb-7c8590ec340d\") " pod="openshift-cluster-node-tuning-operator/tuned-ldvhf"
Oct 30 21:49:09.387065 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.386940    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/d5e5ac6e-d37d-457d-9ca7-93065f5925ff-env-overrides\") pod \"ovnkube-node-wrxww\" (UID: \"d5e5ac6e-d37d-457d-9ca7-93065f5925ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrxww"
Oct 30 21:49:09.387065 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.386969    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/dc220f0e-8a21-4d7f-96b3-e7055c242f42-cni-binary-copy\") pod \"multus-additional-cni-plugins-kggmc\" (UID: \"dc220f0e-8a21-4d7f-96b3-e7055c242f42\") " pod="openshift-multus/multus-additional-cni-plugins-kggmc"
Oct 30 21:49:09.387065 ip-10-0-136-210 kubenswrapper[2576]: E1030 21:49:09.386998    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 21:49:09.387065 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.387004    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/dc220f0e-8a21-4d7f-96b3-e7055c242f42-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-kggmc\" (UID: \"dc220f0e-8a21-4d7f-96b3-e7055c242f42\") " pod="openshift-multus/multus-additional-cni-plugins-kggmc"
Oct 30 21:49:09.387065 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.387037    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/c8b13e55-b3c7-4656-9845-16749704265f-host-run-multus-certs\") pod \"multus-v8j9s\" (UID: \"c8b13e55-b3c7-4656-9845-16749704265f\") " pod="openshift-multus/multus-v8j9s"
Oct 30 21:49:09.387871 ip-10-0-136-210 kubenswrapper[2576]: E1030 21:49:09.387222    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/52eb01f8-e86a-4458-8875-639394a87b30-metrics-certs podName:52eb01f8-e86a-4458-8875-639394a87b30 nodeName:}" failed. No retries permitted until 2026-10-30 21:49:09.887051882 +0000 UTC m=+2.319000153 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/52eb01f8-e86a-4458-8875-639394a87b30-metrics-certs") pod "network-metrics-daemon-vmrkz" (UID: "52eb01f8-e86a-4458-8875-639394a87b30") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 21:49:09.387871 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.387251    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d7c355fa-2141-463e-8cbc-bae3c44a16f1-etc-selinux\") pod \"aws-ebs-csi-driver-node-cqxms\" (UID: \"d7c355fa-2141-463e-8cbc-bae3c44a16f1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cqxms"
Oct 30 21:49:09.387871 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.387284    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d7c355fa-2141-463e-8cbc-bae3c44a16f1-sys-fs\") pod \"aws-ebs-csi-driver-node-cqxms\" (UID: \"d7c355fa-2141-463e-8cbc-bae3c44a16f1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cqxms"
Oct 30 21:49:09.387871 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.387292    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/c8b13e55-b3c7-4656-9845-16749704265f-host-run-multus-certs\") pod \"multus-v8j9s\" (UID: \"c8b13e55-b3c7-4656-9845-16749704265f\") " pod="openshift-multus/multus-v8j9s"
Oct 30 21:49:09.387871 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.387319    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vg89l\" (UniqueName: \"kubernetes.io/projected/d7c355fa-2141-463e-8cbc-bae3c44a16f1-kube-api-access-vg89l\") pod \"aws-ebs-csi-driver-node-cqxms\" (UID: \"d7c355fa-2141-463e-8cbc-bae3c44a16f1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cqxms"
Oct 30 21:49:09.387871 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.387434    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/c8b13e55-b3c7-4656-9845-16749704265f-multus-daemon-config\") pod \"multus-v8j9s\" (UID: \"c8b13e55-b3c7-4656-9845-16749704265f\") " pod="openshift-multus/multus-v8j9s"
Oct 30 21:49:09.387871 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.387475    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f69b6a2e-d33f-4cc8-8e0a-598b20948f38-agent-certs\") pod \"konnectivity-agent-5mw6n\" (UID: \"f69b6a2e-d33f-4cc8-8e0a-598b20948f38\") " pod="kube-system/konnectivity-agent-5mw6n"
Oct 30 21:49:09.387871 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.387478    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d7c355fa-2141-463e-8cbc-bae3c44a16f1-sys-fs\") pod \"aws-ebs-csi-driver-node-cqxms\" (UID: \"d7c355fa-2141-463e-8cbc-bae3c44a16f1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cqxms"
Oct 30 21:49:09.387871 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.387489    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d7c355fa-2141-463e-8cbc-bae3c44a16f1-etc-selinux\") pod \"aws-ebs-csi-driver-node-cqxms\" (UID: \"d7c355fa-2141-463e-8cbc-bae3c44a16f1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cqxms"
Oct 30 21:49:09.387871 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.387518    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/8f2f95c4-21b2-477b-a2cb-7c8590ec340d-etc-sysctl-conf\") pod \"tuned-ldvhf\" (UID: \"8f2f95c4-21b2-477b-a2cb-7c8590ec340d\") " pod="openshift-cluster-node-tuning-operator/tuned-ldvhf"
Oct 30 21:49:09.387871 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.387559    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t7ldj\" (UniqueName: \"kubernetes.io/projected/8f2f95c4-21b2-477b-a2cb-7c8590ec340d-kube-api-access-t7ldj\") pod \"tuned-ldvhf\" (UID: \"8f2f95c4-21b2-477b-a2cb-7c8590ec340d\") " pod="openshift-cluster-node-tuning-operator/tuned-ldvhf"
Oct 30 21:49:09.387871 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.387593    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c8b13e55-b3c7-4656-9845-16749704265f-cni-binary-copy\") pod \"multus-v8j9s\" (UID: \"c8b13e55-b3c7-4656-9845-16749704265f\") " pod="openshift-multus/multus-v8j9s"
Oct 30 21:49:09.387871 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.387661    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/d5e5ac6e-d37d-457d-9ca7-93065f5925ff-run-ovn\") pod \"ovnkube-node-wrxww\" (UID: \"d5e5ac6e-d37d-457d-9ca7-93065f5925ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrxww"
Oct 30 21:49:09.387871 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.387698    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z2p79\" (UniqueName: \"kubernetes.io/projected/d5e5ac6e-d37d-457d-9ca7-93065f5925ff-kube-api-access-z2p79\") pod \"ovnkube-node-wrxww\" (UID: \"d5e5ac6e-d37d-457d-9ca7-93065f5925ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrxww"
Oct 30 21:49:09.387871 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.387777    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8f2f95c4-21b2-477b-a2cb-7c8590ec340d-var-lib-kubelet\") pod \"tuned-ldvhf\" (UID: \"8f2f95c4-21b2-477b-a2cb-7c8590ec340d\") " pod="openshift-cluster-node-tuning-operator/tuned-ldvhf"
Oct 30 21:49:09.387871 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.387810    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e810bcd1-78da-409f-b86e-08c47031939e-kubelet-config\") pod \"global-pull-secret-syncer-68jrf\" (UID: \"e810bcd1-78da-409f-b86e-08c47031939e\") " pod="kube-system/global-pull-secret-syncer-68jrf"
Oct 30 21:49:09.388682 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.387843    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/dc220f0e-8a21-4d7f-96b3-e7055c242f42-cnibin\") pod \"multus-additional-cni-plugins-kggmc\" (UID: \"dc220f0e-8a21-4d7f-96b3-e7055c242f42\") " pod="openshift-multus/multus-additional-cni-plugins-kggmc"
Oct 30 21:49:09.388682 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.387883    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/dc220f0e-8a21-4d7f-96b3-e7055c242f42-tuning-conf-dir\") pod \"multus-additional-cni-plugins-kggmc\" (UID: \"dc220f0e-8a21-4d7f-96b3-e7055c242f42\") " pod="openshift-multus/multus-additional-cni-plugins-kggmc"
Oct 30 21:49:09.388682 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.387921    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fcjg4\" (UniqueName: \"kubernetes.io/projected/73c208eb-ffc0-46f2-8bd4-05ade5adb529-kube-api-access-fcjg4\") pod \"network-check-target-jmbng\" (UID: \"73c208eb-ffc0-46f2-8bd4-05ade5adb529\") " pod="openshift-network-diagnostics/network-check-target-jmbng"
Oct 30 21:49:09.388682 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.387960    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8f2f95c4-21b2-477b-a2cb-7c8590ec340d-tmp\") pod \"tuned-ldvhf\" (UID: \"8f2f95c4-21b2-477b-a2cb-7c8590ec340d\") " pod="openshift-cluster-node-tuning-operator/tuned-ldvhf"
Oct 30 21:49:09.388682 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.388026    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d7c355fa-2141-463e-8cbc-bae3c44a16f1-registration-dir\") pod \"aws-ebs-csi-driver-node-cqxms\" (UID: \"d7c355fa-2141-463e-8cbc-bae3c44a16f1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cqxms"
Oct 30 21:49:09.388682 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.388060    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/8f2f95c4-21b2-477b-a2cb-7c8590ec340d-etc-sysctl-d\") pod \"tuned-ldvhf\" (UID: \"8f2f95c4-21b2-477b-a2cb-7c8590ec340d\") " pod="openshift-cluster-node-tuning-operator/tuned-ldvhf"
Oct 30 21:49:09.388682 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.388096    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/dc220f0e-8a21-4d7f-96b3-e7055c242f42-cni-binary-copy\") pod \"multus-additional-cni-plugins-kggmc\" (UID: \"dc220f0e-8a21-4d7f-96b3-e7055c242f42\") " pod="openshift-multus/multus-additional-cni-plugins-kggmc"
Oct 30 21:49:09.388682 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.388101    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9krk9\" (UniqueName: \"kubernetes.io/projected/db4b06a3-45e5-4d45-9d7a-407d117d9454-kube-api-access-9krk9\") pod \"node-resolver-krtnr\" (UID: \"db4b06a3-45e5-4d45-9d7a-407d117d9454\") " pod="openshift-dns/node-resolver-krtnr"
Oct 30 21:49:09.388682 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.388132    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/04465eda-e6db-42d3-8d5f-c9be668038a9-host\") pod \"node-ca-xkrx5\" (UID: \"04465eda-e6db-42d3-8d5f-c9be668038a9\") " pod="openshift-image-registry/node-ca-xkrx5"
Oct 30 21:49:09.388682 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.388164    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v4d5j\" (UniqueName: \"kubernetes.io/projected/c8b13e55-b3c7-4656-9845-16749704265f-kube-api-access-v4d5j\") pod \"multus-v8j9s\" (UID: \"c8b13e55-b3c7-4656-9845-16749704265f\") " pod="openshift-multus/multus-v8j9s"
Oct 30 21:49:09.388682 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.388235    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/0ab70415-9f1d-4988-a228-df6ed621f0d8-iptables-alerter-script\") pod \"iptables-alerter-l9d98\" (UID: \"0ab70415-9f1d-4988-a228-df6ed621f0d8\") " pod="openshift-network-operator/iptables-alerter-l9d98"
Oct 30 21:49:09.388682 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.388270    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dc220f0e-8a21-4d7f-96b3-e7055c242f42-system-cni-dir\") pod \"multus-additional-cni-plugins-kggmc\" (UID: \"dc220f0e-8a21-4d7f-96b3-e7055c242f42\") " pod="openshift-multus/multus-additional-cni-plugins-kggmc"
Oct 30 21:49:09.388682 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.388303    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f69b6a2e-d33f-4cc8-8e0a-598b20948f38-konnectivity-ca\") pod \"konnectivity-agent-5mw6n\" (UID: \"f69b6a2e-d33f-4cc8-8e0a-598b20948f38\") " pod="kube-system/konnectivity-agent-5mw6n"
Oct 30 21:49:09.388682 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.388338    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/8f2f95c4-21b2-477b-a2cb-7c8590ec340d-etc-tuned\") pod \"tuned-ldvhf\" (UID: \"8f2f95c4-21b2-477b-a2cb-7c8590ec340d\") " pod="openshift-cluster-node-tuning-operator/tuned-ldvhf"
Oct 30 21:49:09.388682 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.388366    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/c8b13e55-b3c7-4656-9845-16749704265f-host-var-lib-cni-multus\") pod \"multus-v8j9s\" (UID: \"c8b13e55-b3c7-4656-9845-16749704265f\") " pod="openshift-multus/multus-v8j9s"
Oct 30 21:49:09.388682 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.388511    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c8b13e55-b3c7-4656-9845-16749704265f-cni-binary-copy\") pod \"multus-v8j9s\" (UID: \"c8b13e55-b3c7-4656-9845-16749704265f\") " pod="openshift-multus/multus-v8j9s"
Oct 30 21:49:09.388682 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.388492    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d5e5ac6e-d37d-457d-9ca7-93065f5925ff-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-wrxww\" (UID: \"d5e5ac6e-d37d-457d-9ca7-93065f5925ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrxww"
Oct 30 21:49:09.389498 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.388560    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d7c355fa-2141-463e-8cbc-bae3c44a16f1-kubelet-dir\") pod \"aws-ebs-csi-driver-node-cqxms\" (UID: \"d7c355fa-2141-463e-8cbc-bae3c44a16f1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cqxms"
Oct 30 21:49:09.389498 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.388594    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/8f2f95c4-21b2-477b-a2cb-7c8590ec340d-etc-modprobe-d\") pod \"tuned-ldvhf\" (UID: \"8f2f95c4-21b2-477b-a2cb-7c8590ec340d\") " pod="openshift-cluster-node-tuning-operator/tuned-ldvhf"
Oct 30 21:49:09.389498 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.388639    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/c8b13e55-b3c7-4656-9845-16749704265f-multus-daemon-config\") pod \"multus-v8j9s\" (UID: \"c8b13e55-b3c7-4656-9845-16749704265f\") " pod="openshift-multus/multus-v8j9s"
Oct 30 21:49:09.389498 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.388631    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/8f2f95c4-21b2-477b-a2cb-7c8590ec340d-etc-systemd\") pod \"tuned-ldvhf\" (UID: \"8f2f95c4-21b2-477b-a2cb-7c8590ec340d\") " pod="openshift-cluster-node-tuning-operator/tuned-ldvhf"
Oct 30 21:49:09.389498 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.388761    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c8b13e55-b3c7-4656-9845-16749704265f-cnibin\") pod \"multus-v8j9s\" (UID: \"c8b13e55-b3c7-4656-9845-16749704265f\") " pod="openshift-multus/multus-v8j9s"
Oct 30 21:49:09.389498 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.388795    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d5e5ac6e-d37d-457d-9ca7-93065f5925ff-etc-openvswitch\") pod \"ovnkube-node-wrxww\" (UID: \"d5e5ac6e-d37d-457d-9ca7-93065f5925ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrxww"
Oct 30 21:49:09.389498 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.388823    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/d5e5ac6e-d37d-457d-9ca7-93065f5925ff-log-socket\") pod \"ovnkube-node-wrxww\" (UID: \"d5e5ac6e-d37d-457d-9ca7-93065f5925ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrxww"
Oct 30 21:49:09.389498 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.388857    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e810bcd1-78da-409f-b86e-08c47031939e-dbus\") pod \"global-pull-secret-syncer-68jrf\" (UID: \"e810bcd1-78da-409f-b86e-08c47031939e\") " pod="kube-system/global-pull-secret-syncer-68jrf"
Oct 30 21:49:09.389498 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.388831    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/8f2f95c4-21b2-477b-a2cb-7c8590ec340d-etc-sysctl-d\") pod \"tuned-ldvhf\" (UID: \"8f2f95c4-21b2-477b-a2cb-7c8590ec340d\") " pod="openshift-cluster-node-tuning-operator/tuned-ldvhf"
Oct 30 21:49:09.389498 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.388966    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/d5e5ac6e-d37d-457d-9ca7-93065f5925ff-run-ovn\") pod \"ovnkube-node-wrxww\" (UID: \"d5e5ac6e-d37d-457d-9ca7-93065f5925ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrxww"
Oct 30 21:49:09.389498 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.389050    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e810bcd1-78da-409f-b86e-08c47031939e-dbus\") pod \"global-pull-secret-syncer-68jrf\" (UID: \"e810bcd1-78da-409f-b86e-08c47031939e\") " pod="kube-system/global-pull-secret-syncer-68jrf"
Oct 30 21:49:09.389498 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.389066    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/8f2f95c4-21b2-477b-a2cb-7c8590ec340d-etc-sysctl-conf\") pod \"tuned-ldvhf\" (UID: \"8f2f95c4-21b2-477b-a2cb-7c8590ec340d\") " pod="openshift-cluster-node-tuning-operator/tuned-ldvhf"
Oct 30 21:49:09.389498 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.389128    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/04465eda-e6db-42d3-8d5f-c9be668038a9-host\") pod \"node-ca-xkrx5\" (UID: \"04465eda-e6db-42d3-8d5f-c9be668038a9\") " pod="openshift-image-registry/node-ca-xkrx5"
Oct 30 21:49:09.389498 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.389176    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/c8b13e55-b3c7-4656-9845-16749704265f-host-var-lib-cni-multus\") pod \"multus-v8j9s\" (UID: \"c8b13e55-b3c7-4656-9845-16749704265f\") " pod="openshift-multus/multus-v8j9s"
Oct 30 21:49:09.389498 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.389231    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/dc220f0e-8a21-4d7f-96b3-e7055c242f42-tuning-conf-dir\") pod \"multus-additional-cni-plugins-kggmc\" (UID: \"dc220f0e-8a21-4d7f-96b3-e7055c242f42\") " pod="openshift-multus/multus-additional-cni-plugins-kggmc"
Oct 30 21:49:09.389498 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.389312    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8f2f95c4-21b2-477b-a2cb-7c8590ec340d-var-lib-kubelet\") pod \"tuned-ldvhf\" (UID: \"8f2f95c4-21b2-477b-a2cb-7c8590ec340d\") " pod="openshift-cluster-node-tuning-operator/tuned-ldvhf"
Oct 30 21:49:09.389498 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.389335    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d5e5ac6e-d37d-457d-9ca7-93065f5925ff-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-wrxww\" (UID: \"d5e5ac6e-d37d-457d-9ca7-93065f5925ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrxww"
Oct 30 21:49:09.389498 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.389373    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e810bcd1-78da-409f-b86e-08c47031939e-kubelet-config\") pod \"global-pull-secret-syncer-68jrf\" (UID: \"e810bcd1-78da-409f-b86e-08c47031939e\") " pod="kube-system/global-pull-secret-syncer-68jrf"
Oct 30 21:49:09.390382 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.389389    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d7c355fa-2141-463e-8cbc-bae3c44a16f1-kubelet-dir\") pod \"aws-ebs-csi-driver-node-cqxms\" (UID: \"d7c355fa-2141-463e-8cbc-bae3c44a16f1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cqxms"
Oct 30 21:49:09.390382 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.389429    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/dc220f0e-8a21-4d7f-96b3-e7055c242f42-cnibin\") pod \"multus-additional-cni-plugins-kggmc\" (UID: \"dc220f0e-8a21-4d7f-96b3-e7055c242f42\") " pod="openshift-multus/multus-additional-cni-plugins-kggmc"
Oct 30 21:49:09.390382 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.389486    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/8f2f95c4-21b2-477b-a2cb-7c8590ec340d-etc-systemd\") pod \"tuned-ldvhf\" (UID: \"8f2f95c4-21b2-477b-a2cb-7c8590ec340d\") " pod="openshift-cluster-node-tuning-operator/tuned-ldvhf"
Oct 30 21:49:09.390382 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.389525    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c8b13e55-b3c7-4656-9845-16749704265f-cnibin\") pod \"multus-v8j9s\" (UID: \"c8b13e55-b3c7-4656-9845-16749704265f\") " pod="openshift-multus/multus-v8j9s"
Oct 30 21:49:09.390382 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.389563    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d5e5ac6e-d37d-457d-9ca7-93065f5925ff-etc-openvswitch\") pod \"ovnkube-node-wrxww\" (UID: \"d5e5ac6e-d37d-457d-9ca7-93065f5925ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrxww"
Oct 30 21:49:09.390382 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.389733    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d7c355fa-2141-463e-8cbc-bae3c44a16f1-registration-dir\") pod \"aws-ebs-csi-driver-node-cqxms\" (UID: \"d7c355fa-2141-463e-8cbc-bae3c44a16f1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cqxms"
Oct 30 21:49:09.390382 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.389798    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/0ab70415-9f1d-4988-a228-df6ed621f0d8-iptables-alerter-script\") pod \"iptables-alerter-l9d98\" (UID: \"0ab70415-9f1d-4988-a228-df6ed621f0d8\") " pod="openshift-network-operator/iptables-alerter-l9d98"
Oct 30 21:49:09.390382 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.389809    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dc220f0e-8a21-4d7f-96b3-e7055c242f42-system-cni-dir\") pod \"multus-additional-cni-plugins-kggmc\" (UID: \"dc220f0e-8a21-4d7f-96b3-e7055c242f42\") " pod="openshift-multus/multus-additional-cni-plugins-kggmc"
Oct 30 21:49:09.390382 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.389819    2576 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Oct 30 21:49:09.390382 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.389862    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/8f2f95c4-21b2-477b-a2cb-7c8590ec340d-etc-modprobe-d\") pod \"tuned-ldvhf\" (UID: \"8f2f95c4-21b2-477b-a2cb-7c8590ec340d\") " pod="openshift-cluster-node-tuning-operator/tuned-ldvhf"
Oct 30 21:49:09.390382 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.389890    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/d5e5ac6e-d37d-457d-9ca7-93065f5925ff-log-socket\") pod \"ovnkube-node-wrxww\" (UID: \"d5e5ac6e-d37d-457d-9ca7-93065f5925ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrxww"
Oct 30 21:49:09.390382 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.389914    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e810bcd1-78da-409f-b86e-08c47031939e-original-pull-secret\") pod \"global-pull-secret-syncer-68jrf\" (UID: \"e810bcd1-78da-409f-b86e-08c47031939e\") " pod="kube-system/global-pull-secret-syncer-68jrf"
Oct 30 21:49:09.390382 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.389971    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/04465eda-e6db-42d3-8d5f-c9be668038a9-serviceca\") pod \"node-ca-xkrx5\" (UID: \"04465eda-e6db-42d3-8d5f-c9be668038a9\") " pod="openshift-image-registry/node-ca-xkrx5"
Oct 30 21:49:09.390382 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.390007    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/c8b13e55-b3c7-4656-9845-16749704265f-hostroot\") pod \"multus-v8j9s\" (UID: \"c8b13e55-b3c7-4656-9845-16749704265f\") " pod="openshift-multus/multus-v8j9s"
Oct 30 21:49:09.390382 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.390036    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d5e5ac6e-d37d-457d-9ca7-93065f5925ff-host-run-netns\") pod \"ovnkube-node-wrxww\" (UID: \"d5e5ac6e-d37d-457d-9ca7-93065f5925ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrxww"
Oct 30 21:49:09.390382 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.390072    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/d5e5ac6e-d37d-457d-9ca7-93065f5925ff-run-systemd\") pod \"ovnkube-node-wrxww\" (UID: \"d5e5ac6e-d37d-457d-9ca7-93065f5925ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrxww"
Oct 30 21:49:09.390382 ip-10-0-136-210 kubenswrapper[2576]: E1030 21:49:09.390078    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 21:49:09.390382 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.390107    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d5e5ac6e-d37d-457d-9ca7-93065f5925ff-run-openvswitch\") pod \"ovnkube-node-wrxww\" (UID: \"d5e5ac6e-d37d-457d-9ca7-93065f5925ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrxww"
Oct 30 21:49:09.391219 ip-10-0-136-210 kubenswrapper[2576]: E1030 21:49:09.390142    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e810bcd1-78da-409f-b86e-08c47031939e-original-pull-secret podName:e810bcd1-78da-409f-b86e-08c47031939e nodeName:}" failed. No retries permitted until 2026-10-30 21:49:09.890118412 +0000 UTC m=+2.322066678 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e810bcd1-78da-409f-b86e-08c47031939e-original-pull-secret") pod "global-pull-secret-syncer-68jrf" (UID: "e810bcd1-78da-409f-b86e-08c47031939e") : object "kube-system"/"original-pull-secret" not registered
Oct 30 21:49:09.391219 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.390166    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d5e5ac6e-d37d-457d-9ca7-93065f5925ff-run-openvswitch\") pod \"ovnkube-node-wrxww\" (UID: \"d5e5ac6e-d37d-457d-9ca7-93065f5925ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrxww"
Oct 30 21:49:09.391219 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.390175    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/d5e5ac6e-d37d-457d-9ca7-93065f5925ff-ovnkube-script-lib\") pod \"ovnkube-node-wrxww\" (UID: \"d5e5ac6e-d37d-457d-9ca7-93065f5925ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrxww"
Oct 30 21:49:09.391219 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.390211    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8f2f95c4-21b2-477b-a2cb-7c8590ec340d-host\") pod \"tuned-ldvhf\" (UID: \"8f2f95c4-21b2-477b-a2cb-7c8590ec340d\") " pod="openshift-cluster-node-tuning-operator/tuned-ldvhf"
Oct 30 21:49:09.391219 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.390267    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/c8b13e55-b3c7-4656-9845-16749704265f-hostroot\") pod \"multus-v8j9s\" (UID: \"c8b13e55-b3c7-4656-9845-16749704265f\") " pod="openshift-multus/multus-v8j9s"
Oct 30 21:49:09.391219 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.390285    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8f2f95c4-21b2-477b-a2cb-7c8590ec340d-host\") pod \"tuned-ldvhf\" (UID: \"8f2f95c4-21b2-477b-a2cb-7c8590ec340d\") " pod="openshift-cluster-node-tuning-operator/tuned-ldvhf"
Oct 30 21:49:09.391219 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.390313    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d5e5ac6e-d37d-457d-9ca7-93065f5925ff-host-run-netns\") pod \"ovnkube-node-wrxww\" (UID: \"d5e5ac6e-d37d-457d-9ca7-93065f5925ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrxww"
Oct 30 21:49:09.391219 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.390359    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c8b13e55-b3c7-4656-9845-16749704265f-system-cni-dir\") pod \"multus-v8j9s\" (UID: \"c8b13e55-b3c7-4656-9845-16749704265f\") " pod="openshift-multus/multus-v8j9s"
Oct 30 21:49:09.391219 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.390518    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d5e5ac6e-d37d-457d-9ca7-93065f5925ff-host-slash\") pod \"ovnkube-node-wrxww\" (UID: \"d5e5ac6e-d37d-457d-9ca7-93065f5925ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrxww"
Oct 30 21:49:09.391219 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.390551    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/d5e5ac6e-d37d-457d-9ca7-93065f5925ff-node-log\") pod \"ovnkube-node-wrxww\" (UID: \"d5e5ac6e-d37d-457d-9ca7-93065f5925ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrxww"
Oct 30 21:49:09.391219 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.390658    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/dc220f0e-8a21-4d7f-96b3-e7055c242f42-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-kggmc\" (UID: \"dc220f0e-8a21-4d7f-96b3-e7055c242f42\") " pod="openshift-multus/multus-additional-cni-plugins-kggmc"
Oct 30 21:49:09.391219 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.390748    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c8b13e55-b3c7-4656-9845-16749704265f-system-cni-dir\") pod \"multus-v8j9s\" (UID: \"c8b13e55-b3c7-4656-9845-16749704265f\") " pod="openshift-multus/multus-v8j9s"
Oct 30 21:49:09.391219 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.390978    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/d5e5ac6e-d37d-457d-9ca7-93065f5925ff-ovnkube-script-lib\") pod \"ovnkube-node-wrxww\" (UID: \"d5e5ac6e-d37d-457d-9ca7-93065f5925ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrxww"
Oct 30 21:49:09.391219 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.391066    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d5e5ac6e-d37d-457d-9ca7-93065f5925ff-host-slash\") pod \"ovnkube-node-wrxww\" (UID: \"d5e5ac6e-d37d-457d-9ca7-93065f5925ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrxww"
Oct 30 21:49:09.391219 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.391117    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/db4b06a3-45e5-4d45-9d7a-407d117d9454-hosts-file\") pod \"node-resolver-krtnr\" (UID: \"db4b06a3-45e5-4d45-9d7a-407d117d9454\") " pod="openshift-dns/node-resolver-krtnr"
Oct 30 21:49:09.391219 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.391149    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hrxwg\" (UniqueName: \"kubernetes.io/projected/04465eda-e6db-42d3-8d5f-c9be668038a9-kube-api-access-hrxwg\") pod \"node-ca-xkrx5\" (UID: \"04465eda-e6db-42d3-8d5f-c9be668038a9\") " pod="openshift-image-registry/node-ca-xkrx5"
Oct 30 21:49:09.391219 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.391164    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/db4b06a3-45e5-4d45-9d7a-407d117d9454-hosts-file\") pod \"node-resolver-krtnr\" (UID: \"db4b06a3-45e5-4d45-9d7a-407d117d9454\") " pod="openshift-dns/node-resolver-krtnr"
Oct 30 21:49:09.392069 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.391210    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d5e5ac6e-d37d-457d-9ca7-93065f5925ff-host-run-ovn-kubernetes\") pod \"ovnkube-node-wrxww\" (UID: \"d5e5ac6e-d37d-457d-9ca7-93065f5925ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrxww"
Oct 30 21:49:09.392069 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.391243    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c8b13e55-b3c7-4656-9845-16749704265f-host-var-lib-kubelet\") pod \"multus-v8j9s\" (UID: \"c8b13e55-b3c7-4656-9845-16749704265f\") " pod="openshift-multus/multus-v8j9s"
Oct 30 21:49:09.392069 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.391276    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c8b13e55-b3c7-4656-9845-16749704265f-multus-conf-dir\") pod \"multus-v8j9s\" (UID: \"c8b13e55-b3c7-4656-9845-16749704265f\") " pod="openshift-multus/multus-v8j9s"
Oct 30 21:49:09.392069 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.391344    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/d5e5ac6e-d37d-457d-9ca7-93065f5925ff-host-kubelet\") pod \"ovnkube-node-wrxww\" (UID: \"d5e5ac6e-d37d-457d-9ca7-93065f5925ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrxww"
Oct 30 21:49:09.392069 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.391416    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/d5e5ac6e-d37d-457d-9ca7-93065f5925ff-systemd-units\") pod \"ovnkube-node-wrxww\" (UID: \"d5e5ac6e-d37d-457d-9ca7-93065f5925ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrxww"
Oct 30 21:49:09.392069 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.391468    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/d5e5ac6e-d37d-457d-9ca7-93065f5925ff-node-log\") pod \"ovnkube-node-wrxww\" (UID: \"d5e5ac6e-d37d-457d-9ca7-93065f5925ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrxww"
Oct 30 21:49:09.392069 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.391521    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/d5e5ac6e-d37d-457d-9ca7-93065f5925ff-run-systemd\") pod \"ovnkube-node-wrxww\" (UID: \"d5e5ac6e-d37d-457d-9ca7-93065f5925ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrxww"
Oct 30 21:49:09.392069 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.391549    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c8b13e55-b3c7-4656-9845-16749704265f-multus-conf-dir\") pod \"multus-v8j9s\" (UID: \"c8b13e55-b3c7-4656-9845-16749704265f\") " pod="openshift-multus/multus-v8j9s"
Oct 30 21:49:09.392069 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.391556    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d5e5ac6e-d37d-457d-9ca7-93065f5925ff-host-run-ovn-kubernetes\") pod \"ovnkube-node-wrxww\" (UID: \"d5e5ac6e-d37d-457d-9ca7-93065f5925ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrxww"
Oct 30 21:49:09.392069 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.391569    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/d5e5ac6e-d37d-457d-9ca7-93065f5925ff-host-kubelet\") pod \"ovnkube-node-wrxww\" (UID: \"d5e5ac6e-d37d-457d-9ca7-93065f5925ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrxww"
Oct 30 21:49:09.392069 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.391632    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c8b13e55-b3c7-4656-9845-16749704265f-host-var-lib-kubelet\") pod \"multus-v8j9s\" (UID: \"c8b13e55-b3c7-4656-9845-16749704265f\") " pod="openshift-multus/multus-v8j9s"
Oct 30 21:49:09.392069 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.391677    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/d5e5ac6e-d37d-457d-9ca7-93065f5925ff-systemd-units\") pod \"ovnkube-node-wrxww\" (UID: \"d5e5ac6e-d37d-457d-9ca7-93065f5925ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrxww"
Oct 30 21:49:09.393540 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.393508    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/8f2f95c4-21b2-477b-a2cb-7c8590ec340d-etc-tuned\") pod \"tuned-ldvhf\" (UID: \"8f2f95c4-21b2-477b-a2cb-7c8590ec340d\") " pod="openshift-cluster-node-tuning-operator/tuned-ldvhf"
Oct 30 21:49:09.393648 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.393580    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8f2f95c4-21b2-477b-a2cb-7c8590ec340d-tmp\") pod \"tuned-ldvhf\" (UID: \"8f2f95c4-21b2-477b-a2cb-7c8590ec340d\") " pod="openshift-cluster-node-tuning-operator/tuned-ldvhf"
Oct 30 21:49:09.398760 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.398742    2576 reflector.go:446] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Oct 30 21:49:09.398841 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.398809    2576 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-f7lss"
Oct 30 21:49:09.404111 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.404092    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Oct 30 21:49:09.404182 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.404155    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Oct 30 21:49:09.404182 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.404156    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Oct 30 21:49:09.404300 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.404282    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-dvh7z\""
Oct 30 21:49:09.404347 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.404312    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Oct 30 21:49:09.404347 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.404330    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Oct 30 21:49:09.404449 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.404432    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Oct 30 21:49:09.404506 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.404472    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Oct 30 21:49:09.404506 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.404486    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Oct 30 21:49:09.404506 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.404501    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Oct 30 21:49:09.404641 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.404503    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-9mttd\""
Oct 30 21:49:09.404641 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.404555    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Oct 30 21:49:09.404641 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.404580    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Oct 30 21:49:09.404641 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.404590    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Oct 30 21:49:09.404641 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.404593    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-cmhxq\""
Oct 30 21:49:09.407616 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.407592    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/d5e5ac6e-d37d-457d-9ca7-93065f5925ff-env-overrides\") pod \"ovnkube-node-wrxww\" (UID: \"d5e5ac6e-d37d-457d-9ca7-93065f5925ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrxww"
Oct 30 21:49:09.407973 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.407956    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/d5e5ac6e-d37d-457d-9ca7-93065f5925ff-ovnkube-config\") pod \"ovnkube-node-wrxww\" (UID: \"d5e5ac6e-d37d-457d-9ca7-93065f5925ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrxww"
Oct 30 21:49:09.408174 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.408157    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/dc220f0e-8a21-4d7f-96b3-e7055c242f42-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-kggmc\" (UID: \"dc220f0e-8a21-4d7f-96b3-e7055c242f42\") " pod="openshift-multus/multus-additional-cni-plugins-kggmc"
Oct 30 21:49:09.409153 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.409134    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/d5e5ac6e-d37d-457d-9ca7-93065f5925ff-ovn-node-metrics-cert\") pod \"ovnkube-node-wrxww\" (UID: \"d5e5ac6e-d37d-457d-9ca7-93065f5925ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrxww"
Oct 30 21:49:09.409467 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.409446    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f69b6a2e-d33f-4cc8-8e0a-598b20948f38-konnectivity-ca\") pod \"konnectivity-agent-5mw6n\" (UID: \"f69b6a2e-d33f-4cc8-8e0a-598b20948f38\") " pod="kube-system/konnectivity-agent-5mw6n"
Oct 30 21:49:09.409958 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.409944    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f69b6a2e-d33f-4cc8-8e0a-598b20948f38-agent-certs\") pod \"konnectivity-agent-5mw6n\" (UID: \"f69b6a2e-d33f-4cc8-8e0a-598b20948f38\") " pod="kube-system/konnectivity-agent-5mw6n"
Oct 30 21:49:09.411688 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.411661    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/04465eda-e6db-42d3-8d5f-c9be668038a9-serviceca\") pod \"node-ca-xkrx5\" (UID: \"04465eda-e6db-42d3-8d5f-c9be668038a9\") " pod="openshift-image-registry/node-ca-xkrx5"
Oct 30 21:49:09.412056 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.412035    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/dc220f0e-8a21-4d7f-96b3-e7055c242f42-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-kggmc\" (UID: \"dc220f0e-8a21-4d7f-96b3-e7055c242f42\") " pod="openshift-multus/multus-additional-cni-plugins-kggmc"
Oct 30 21:49:09.449725 ip-10-0-136-210 kubenswrapper[2576]: E1030 21:49:09.449638    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 21:49:09.449725 ip-10-0-136-210 kubenswrapper[2576]: E1030 21:49:09.449678    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 21:49:09.449725 ip-10-0-136-210 kubenswrapper[2576]: E1030 21:49:09.449693    2576 projected.go:196] Error preparing data for projected volume kube-api-access-fcjg4 for pod openshift-network-diagnostics/network-check-target-jmbng: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 21:49:09.449935 ip-10-0-136-210 kubenswrapper[2576]: E1030 21:49:09.449777    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/73c208eb-ffc0-46f2-8bd4-05ade5adb529-kube-api-access-fcjg4 podName:73c208eb-ffc0-46f2-8bd4-05ade5adb529 nodeName:}" failed. No retries permitted until 2026-10-30 21:49:09.949756388 +0000 UTC m=+2.381704662 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-fcjg4" (UniqueName: "kubernetes.io/projected/73c208eb-ffc0-46f2-8bd4-05ade5adb529-kube-api-access-fcjg4") pod "network-check-target-jmbng" (UID: "73c208eb-ffc0-46f2-8bd4-05ade5adb529") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 21:49:09.449935 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.449864    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tq25h\" (UniqueName: \"kubernetes.io/projected/dc220f0e-8a21-4d7f-96b3-e7055c242f42-kube-api-access-tq25h\") pod \"multus-additional-cni-plugins-kggmc\" (UID: \"dc220f0e-8a21-4d7f-96b3-e7055c242f42\") " pod="openshift-multus/multus-additional-cni-plugins-kggmc"
Oct 30 21:49:09.451141 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.451116    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vg89l\" (UniqueName: \"kubernetes.io/projected/d7c355fa-2141-463e-8cbc-bae3c44a16f1-kube-api-access-vg89l\") pod \"aws-ebs-csi-driver-node-cqxms\" (UID: \"d7c355fa-2141-463e-8cbc-bae3c44a16f1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cqxms"
Oct 30 21:49:09.451237 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.451120    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-td6rr\" (UniqueName: \"kubernetes.io/projected/0ab70415-9f1d-4988-a228-df6ed621f0d8-kube-api-access-td6rr\") pod \"iptables-alerter-l9d98\" (UID: \"0ab70415-9f1d-4988-a228-df6ed621f0d8\") " pod="openshift-network-operator/iptables-alerter-l9d98"
Oct 30 21:49:09.451348 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.451327    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zxwjt\" (UniqueName: \"kubernetes.io/projected/52eb01f8-e86a-4458-8875-639394a87b30-kube-api-access-zxwjt\") pod \"network-metrics-daemon-vmrkz\" (UID: \"52eb01f8-e86a-4458-8875-639394a87b30\") " pod="openshift-multus/network-metrics-daemon-vmrkz"
Oct 30 21:49:09.451708 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.451690    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t7ldj\" (UniqueName: \"kubernetes.io/projected/8f2f95c4-21b2-477b-a2cb-7c8590ec340d-kube-api-access-t7ldj\") pod \"tuned-ldvhf\" (UID: \"8f2f95c4-21b2-477b-a2cb-7c8590ec340d\") " pod="openshift-cluster-node-tuning-operator/tuned-ldvhf"
Oct 30 21:49:09.452789 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.452772    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z2p79\" (UniqueName: \"kubernetes.io/projected/d5e5ac6e-d37d-457d-9ca7-93065f5925ff-kube-api-access-z2p79\") pod \"ovnkube-node-wrxww\" (UID: \"d5e5ac6e-d37d-457d-9ca7-93065f5925ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrxww"
Oct 30 21:49:09.452842 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.452829    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v4d5j\" (UniqueName: \"kubernetes.io/projected/c8b13e55-b3c7-4656-9845-16749704265f-kube-api-access-v4d5j\") pod \"multus-v8j9s\" (UID: \"c8b13e55-b3c7-4656-9845-16749704265f\") " pod="openshift-multus/multus-v8j9s"
Oct 30 21:49:09.454843 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.454822    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9krk9\" (UniqueName: \"kubernetes.io/projected/db4b06a3-45e5-4d45-9d7a-407d117d9454-kube-api-access-9krk9\") pod \"node-resolver-krtnr\" (UID: \"db4b06a3-45e5-4d45-9d7a-407d117d9454\") " pod="openshift-dns/node-resolver-krtnr"
Oct 30 21:49:09.455444 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.455427    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hrxwg\" (UniqueName: \"kubernetes.io/projected/04465eda-e6db-42d3-8d5f-c9be668038a9-kube-api-access-hrxwg\") pod \"node-ca-xkrx5\" (UID: \"04465eda-e6db-42d3-8d5f-c9be668038a9\") " pod="openshift-image-registry/node-ca-xkrx5"
Oct 30 21:49:09.577108 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.577079    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-v8j9s"
Oct 30 21:49:09.589228 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.589211    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cqxms"
Oct 30 21:49:09.603493 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.603456    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-ldvhf"
Oct 30 21:49:09.625451 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.625425    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-krtnr"
Oct 30 21:49:09.640726 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.640702    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-l9d98"
Oct 30 21:49:09.668706 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.668679    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-wrxww"
Oct 30 21:49:09.671335 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.671299    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-5mw6n"
Oct 30 21:49:09.694377 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.694355    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-xkrx5"
Oct 30 21:49:09.716301 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.716247    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-kggmc"
Oct 30 21:49:09.895903 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.895875    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e810bcd1-78da-409f-b86e-08c47031939e-original-pull-secret\") pod \"global-pull-secret-syncer-68jrf\" (UID: \"e810bcd1-78da-409f-b86e-08c47031939e\") " pod="kube-system/global-pull-secret-syncer-68jrf"
Oct 30 21:49:09.896128 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.895921    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/52eb01f8-e86a-4458-8875-639394a87b30-metrics-certs\") pod \"network-metrics-daemon-vmrkz\" (UID: \"52eb01f8-e86a-4458-8875-639394a87b30\") " pod="openshift-multus/network-metrics-daemon-vmrkz"
Oct 30 21:49:09.896128 ip-10-0-136-210 kubenswrapper[2576]: E1030 21:49:09.896006    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 21:49:09.896128 ip-10-0-136-210 kubenswrapper[2576]: E1030 21:49:09.896059    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e810bcd1-78da-409f-b86e-08c47031939e-original-pull-secret podName:e810bcd1-78da-409f-b86e-08c47031939e nodeName:}" failed. No retries permitted until 2026-10-30 21:49:10.896045847 +0000 UTC m=+3.327994094 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e810bcd1-78da-409f-b86e-08c47031939e-original-pull-secret") pod "global-pull-secret-syncer-68jrf" (UID: "e810bcd1-78da-409f-b86e-08c47031939e") : object "kube-system"/"original-pull-secret" not registered
Oct 30 21:49:09.896128 ip-10-0-136-210 kubenswrapper[2576]: E1030 21:49:09.896014    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 21:49:09.896128 ip-10-0-136-210 kubenswrapper[2576]: E1030 21:49:09.896093    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/52eb01f8-e86a-4458-8875-639394a87b30-metrics-certs podName:52eb01f8-e86a-4458-8875-639394a87b30 nodeName:}" failed. No retries permitted until 2026-10-30 21:49:10.896087293 +0000 UTC m=+3.328035539 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/52eb01f8-e86a-4458-8875-639394a87b30-metrics-certs") pod "network-metrics-daemon-vmrkz" (UID: "52eb01f8-e86a-4458-8875-639394a87b30") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 21:49:09.946597 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:09.946557    2576 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd91a9843014f769d901fa6aba545e397.slice/crio-327a89302c3fd96103b398b276678511aec903499e1c71d72134532152be1296 WatchSource:0}: Error finding container 327a89302c3fd96103b398b276678511aec903499e1c71d72134532152be1296: Status 404 returned error can't find the container with id 327a89302c3fd96103b398b276678511aec903499e1c71d72134532152be1296
Oct 30 21:49:09.947108 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:09.946926    2576 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4492449458f4d650448405f5044e8d24.slice/crio-c1581eb1a638205af61c045effda2bf3708d1f9f54df9b7a6c6c4d69436f7817 WatchSource:0}: Error finding container c1581eb1a638205af61c045effda2bf3708d1f9f54df9b7a6c6c4d69436f7817: Status 404 returned error can't find the container with id c1581eb1a638205af61c045effda2bf3708d1f9f54df9b7a6c6c4d69436f7817
Oct 30 21:49:09.953497 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.953479    2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 21:49:09.996862 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:09.996798    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fcjg4\" (UniqueName: \"kubernetes.io/projected/73c208eb-ffc0-46f2-8bd4-05ade5adb529-kube-api-access-fcjg4\") pod \"network-check-target-jmbng\" (UID: \"73c208eb-ffc0-46f2-8bd4-05ade5adb529\") " pod="openshift-network-diagnostics/network-check-target-jmbng"
Oct 30 21:49:09.996965 ip-10-0-136-210 kubenswrapper[2576]: E1030 21:49:09.996905    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 21:49:09.996965 ip-10-0-136-210 kubenswrapper[2576]: E1030 21:49:09.996921    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 21:49:09.996965 ip-10-0-136-210 kubenswrapper[2576]: E1030 21:49:09.996930    2576 projected.go:196] Error preparing data for projected volume kube-api-access-fcjg4 for pod openshift-network-diagnostics/network-check-target-jmbng: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 21:49:09.997051 ip-10-0-136-210 kubenswrapper[2576]: E1030 21:49:09.996976    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/73c208eb-ffc0-46f2-8bd4-05ade5adb529-kube-api-access-fcjg4 podName:73c208eb-ffc0-46f2-8bd4-05ade5adb529 nodeName:}" failed. No retries permitted until 2026-10-30 21:49:10.996960376 +0000 UTC m=+3.428908636 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-fcjg4" (UniqueName: "kubernetes.io/projected/73c208eb-ffc0-46f2-8bd4-05ade5adb529-kube-api-access-fcjg4") pod "network-check-target-jmbng" (UID: "73c208eb-ffc0-46f2-8bd4-05ade5adb529") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 21:49:10.123196 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:10.123174    2576 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 21:49:10.212962 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:10.212911    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-210.us-west-2.compute.internal" event={"ID":"d91a9843014f769d901fa6aba545e397","Type":"ContainerStarted","Data":"327a89302c3fd96103b398b276678511aec903499e1c71d72134532152be1296"}
Oct 30 21:49:10.213921 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:10.213901    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-210.us-west-2.compute.internal" event={"ID":"4492449458f4d650448405f5044e8d24","Type":"ContainerStarted","Data":"c1581eb1a638205af61c045effda2bf3708d1f9f54df9b7a6c6c4d69436f7817"}
Oct 30 21:49:10.362320 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:10.362288    2576 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 21:49:10.402079 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:10.402022    2576 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-29 21:44:09 +0000 UTC" deadline="2028-06-08 21:57:05.044817509 +0000 UTC"
Oct 30 21:49:10.402079 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:10.402069    2576 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14088h7m54.642752738s"
Oct 30 21:49:10.457010 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:10.456978    2576 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f2f95c4_21b2_477b_a2cb_7c8590ec340d.slice/crio-000ee86193020b63cfe66a6d2a6988f405c4eec29ed7aa585c98b8afe0f778ef WatchSource:0}: Error finding container 000ee86193020b63cfe66a6d2a6988f405c4eec29ed7aa585c98b8afe0f778ef: Status 404 returned error can't find the container with id 000ee86193020b63cfe66a6d2a6988f405c4eec29ed7aa585c98b8afe0f778ef
Oct 30 21:49:10.462054 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:10.462030    2576 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8b13e55_b3c7_4656_9845_16749704265f.slice/crio-29097bc9c91c0bbb3e5034ccc9cf5e145105cca40a1dfb3dee33c7d2ad65ea31 WatchSource:0}: Error finding container 29097bc9c91c0bbb3e5034ccc9cf5e145105cca40a1dfb3dee33c7d2ad65ea31: Status 404 returned error can't find the container with id 29097bc9c91c0bbb3e5034ccc9cf5e145105cca40a1dfb3dee33c7d2ad65ea31
Oct 30 21:49:10.468297 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:10.468271    2576 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7c355fa_2141_463e_8cbc_bae3c44a16f1.slice/crio-cc6e2905d1c15d26a3a375d45ff83a2a3e6b21a41741fe6c57f315c1ea464d58 WatchSource:0}: Error finding container cc6e2905d1c15d26a3a375d45ff83a2a3e6b21a41741fe6c57f315c1ea464d58: Status 404 returned error can't find the container with id cc6e2905d1c15d26a3a375d45ff83a2a3e6b21a41741fe6c57f315c1ea464d58
Oct 30 21:49:10.529588 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:10.529567    2576 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 21:49:10.551975 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:10.551941    2576 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf69b6a2e_d33f_4cc8_8e0a_598b20948f38.slice/crio-26d84296d1d451363382e3d3f1fd68416a311f8b1ba5919ac8230687bef7460c WatchSource:0}: Error finding container 26d84296d1d451363382e3d3f1fd68416a311f8b1ba5919ac8230687bef7460c: Status 404 returned error can't find the container with id 26d84296d1d451363382e3d3f1fd68416a311f8b1ba5919ac8230687bef7460c
Oct 30 21:49:10.558909 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:10.558883    2576 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb4b06a3_45e5_4d45_9d7a_407d117d9454.slice/crio-2bdc41beaf5186f2d39137449d5d2a9606e952cfcb48cef9006ad1fab03049da WatchSource:0}: Error finding container 2bdc41beaf5186f2d39137449d5d2a9606e952cfcb48cef9006ad1fab03049da: Status 404 returned error can't find the container with id 2bdc41beaf5186f2d39137449d5d2a9606e952cfcb48cef9006ad1fab03049da
Oct 30 21:49:10.573478 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:10.573456    2576 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5e5ac6e_d37d_457d_9ca7_93065f5925ff.slice/crio-847fb3c0d4eb226ac6a8ad5fdd1e6fb20a842ef516173dfee5213674f9f0d651 WatchSource:0}: Error finding container 847fb3c0d4eb226ac6a8ad5fdd1e6fb20a842ef516173dfee5213674f9f0d651: Status 404 returned error can't find the container with id 847fb3c0d4eb226ac6a8ad5fdd1e6fb20a842ef516173dfee5213674f9f0d651
Oct 30 21:49:10.580580 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:10.580554    2576 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04465eda_e6db_42d3_8d5f_c9be668038a9.slice/crio-078dbad40157a479574f1da113133d4990f89749660a3686f1fd0e1ff2882507 WatchSource:0}: Error finding container 078dbad40157a479574f1da113133d4990f89749660a3686f1fd0e1ff2882507: Status 404 returned error can't find the container with id 078dbad40157a479574f1da113133d4990f89749660a3686f1fd0e1ff2882507
Oct 30 21:49:10.581398 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:10.581377    2576 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc220f0e_8a21_4d7f_96b3_e7055c242f42.slice/crio-5791c43c54a24e8372a1e21b3010621359dc2caa4fbdac86975465bbfa40f7b3 WatchSource:0}: Error finding container 5791c43c54a24e8372a1e21b3010621359dc2caa4fbdac86975465bbfa40f7b3: Status 404 returned error can't find the container with id 5791c43c54a24e8372a1e21b3010621359dc2caa4fbdac86975465bbfa40f7b3
Oct 30 21:49:10.767803 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:10.767775    2576 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ab70415_9f1d_4988_a228_df6ed621f0d8.slice/crio-ce973673f940234e7bde5a0d7a85118c6da9bd4ed38d54bac859dc43fe25920b WatchSource:0}: Error finding container ce973673f940234e7bde5a0d7a85118c6da9bd4ed38d54bac859dc43fe25920b: Status 404 returned error can't find the container with id ce973673f940234e7bde5a0d7a85118c6da9bd4ed38d54bac859dc43fe25920b
Oct 30 21:49:10.903090 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:10.903059    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e810bcd1-78da-409f-b86e-08c47031939e-original-pull-secret\") pod \"global-pull-secret-syncer-68jrf\" (UID: \"e810bcd1-78da-409f-b86e-08c47031939e\") " pod="kube-system/global-pull-secret-syncer-68jrf"
Oct 30 21:49:10.903243 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:10.903102    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/52eb01f8-e86a-4458-8875-639394a87b30-metrics-certs\") pod \"network-metrics-daemon-vmrkz\" (UID: \"52eb01f8-e86a-4458-8875-639394a87b30\") " pod="openshift-multus/network-metrics-daemon-vmrkz"
Oct 30 21:49:10.903243 ip-10-0-136-210 kubenswrapper[2576]: E1030 21:49:10.903211    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 21:49:10.903243 ip-10-0-136-210 kubenswrapper[2576]: E1030 21:49:10.903231    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 21:49:10.903338 ip-10-0-136-210 kubenswrapper[2576]: E1030 21:49:10.903272    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e810bcd1-78da-409f-b86e-08c47031939e-original-pull-secret podName:e810bcd1-78da-409f-b86e-08c47031939e nodeName:}" failed. No retries permitted until 2026-10-30 21:49:12.903254948 +0000 UTC m=+5.335203194 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e810bcd1-78da-409f-b86e-08c47031939e-original-pull-secret") pod "global-pull-secret-syncer-68jrf" (UID: "e810bcd1-78da-409f-b86e-08c47031939e") : object "kube-system"/"original-pull-secret" not registered
Oct 30 21:49:10.903338 ip-10-0-136-210 kubenswrapper[2576]: E1030 21:49:10.903290    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/52eb01f8-e86a-4458-8875-639394a87b30-metrics-certs podName:52eb01f8-e86a-4458-8875-639394a87b30 nodeName:}" failed. No retries permitted until 2026-10-30 21:49:12.90327997 +0000 UTC m=+5.335228224 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/52eb01f8-e86a-4458-8875-639394a87b30-metrics-certs") pod "network-metrics-daemon-vmrkz" (UID: "52eb01f8-e86a-4458-8875-639394a87b30") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 21:49:11.003471 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:11.003439    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fcjg4\" (UniqueName: \"kubernetes.io/projected/73c208eb-ffc0-46f2-8bd4-05ade5adb529-kube-api-access-fcjg4\") pod \"network-check-target-jmbng\" (UID: \"73c208eb-ffc0-46f2-8bd4-05ade5adb529\") " pod="openshift-network-diagnostics/network-check-target-jmbng"
Oct 30 21:49:11.003940 ip-10-0-136-210 kubenswrapper[2576]: E1030 21:49:11.003592    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 21:49:11.003940 ip-10-0-136-210 kubenswrapper[2576]: E1030 21:49:11.003609    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 21:49:11.003940 ip-10-0-136-210 kubenswrapper[2576]: E1030 21:49:11.003618    2576 projected.go:196] Error preparing data for projected volume kube-api-access-fcjg4 for pod openshift-network-diagnostics/network-check-target-jmbng: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 21:49:11.003940 ip-10-0-136-210 kubenswrapper[2576]: E1030 21:49:11.003682    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/73c208eb-ffc0-46f2-8bd4-05ade5adb529-kube-api-access-fcjg4 podName:73c208eb-ffc0-46f2-8bd4-05ade5adb529 nodeName:}" failed. No retries permitted until 2026-10-30 21:49:13.003650462 +0000 UTC m=+5.435598727 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-fcjg4" (UniqueName: "kubernetes.io/projected/73c208eb-ffc0-46f2-8bd4-05ade5adb529-kube-api-access-fcjg4") pod "network-check-target-jmbng" (UID: "73c208eb-ffc0-46f2-8bd4-05ade5adb529") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 21:49:11.209376 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:11.209342    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jmbng"
Oct 30 21:49:11.209528 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:11.209381    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vmrkz"
Oct 30 21:49:11.209528 ip-10-0-136-210 kubenswrapper[2576]: E1030 21:49:11.209455    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jmbng" podUID="73c208eb-ffc0-46f2-8bd4-05ade5adb529"
Oct 30 21:49:11.209647 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:11.209537    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-68jrf"
Oct 30 21:49:11.209722 ip-10-0-136-210 kubenswrapper[2576]: E1030 21:49:11.209649    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vmrkz" podUID="52eb01f8-e86a-4458-8875-639394a87b30"
Oct 30 21:49:11.209798 ip-10-0-136-210 kubenswrapper[2576]: E1030 21:49:11.209777    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-68jrf" podUID="e810bcd1-78da-409f-b86e-08c47031939e"
Oct 30 21:49:11.216893 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:11.216866    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-l9d98" event={"ID":"0ab70415-9f1d-4988-a228-df6ed621f0d8","Type":"ContainerStarted","Data":"ce973673f940234e7bde5a0d7a85118c6da9bd4ed38d54bac859dc43fe25920b"}
Oct 30 21:49:11.218285 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:11.218253    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kggmc" event={"ID":"dc220f0e-8a21-4d7f-96b3-e7055c242f42","Type":"ContainerStarted","Data":"5791c43c54a24e8372a1e21b3010621359dc2caa4fbdac86975465bbfa40f7b3"}
Oct 30 21:49:11.219481 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:11.219429    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-xkrx5" event={"ID":"04465eda-e6db-42d3-8d5f-c9be668038a9","Type":"ContainerStarted","Data":"078dbad40157a479574f1da113133d4990f89749660a3686f1fd0e1ff2882507"}
Oct 30 21:49:11.220779 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:11.220754    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wrxww" event={"ID":"d5e5ac6e-d37d-457d-9ca7-93065f5925ff","Type":"ContainerStarted","Data":"847fb3c0d4eb226ac6a8ad5fdd1e6fb20a842ef516173dfee5213674f9f0d651"}
Oct 30 21:49:11.222037 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:11.222010    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-krtnr" event={"ID":"db4b06a3-45e5-4d45-9d7a-407d117d9454","Type":"ContainerStarted","Data":"2bdc41beaf5186f2d39137449d5d2a9606e952cfcb48cef9006ad1fab03049da"}
Oct 30 21:49:11.223308 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:11.223283    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-5mw6n" event={"ID":"f69b6a2e-d33f-4cc8-8e0a-598b20948f38","Type":"ContainerStarted","Data":"26d84296d1d451363382e3d3f1fd68416a311f8b1ba5919ac8230687bef7460c"}
Oct 30 21:49:11.224451 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:11.224424    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cqxms" event={"ID":"d7c355fa-2141-463e-8cbc-bae3c44a16f1","Type":"ContainerStarted","Data":"cc6e2905d1c15d26a3a375d45ff83a2a3e6b21a41741fe6c57f315c1ea464d58"}
Oct 30 21:49:11.225682 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:11.225644    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-v8j9s" event={"ID":"c8b13e55-b3c7-4656-9845-16749704265f","Type":"ContainerStarted","Data":"29097bc9c91c0bbb3e5034ccc9cf5e145105cca40a1dfb3dee33c7d2ad65ea31"}
Oct 30 21:49:11.227017 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:11.226994    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-ldvhf" event={"ID":"8f2f95c4-21b2-477b-a2cb-7c8590ec340d","Type":"ContainerStarted","Data":"000ee86193020b63cfe66a6d2a6988f405c4eec29ed7aa585c98b8afe0f778ef"}
Oct 30 21:49:11.403146 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:11.403106    2576 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-29 21:44:09 +0000 UTC" deadline="2028-04-18 14:36:13.56684935 +0000 UTC"
Oct 30 21:49:11.403146 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:11.403138    2576 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12856h47m2.163714848s"
Oct 30 21:49:12.921259 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:12.921154    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e810bcd1-78da-409f-b86e-08c47031939e-original-pull-secret\") pod \"global-pull-secret-syncer-68jrf\" (UID: \"e810bcd1-78da-409f-b86e-08c47031939e\") " pod="kube-system/global-pull-secret-syncer-68jrf"
Oct 30 21:49:12.921740 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:12.921279    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/52eb01f8-e86a-4458-8875-639394a87b30-metrics-certs\") pod \"network-metrics-daemon-vmrkz\" (UID: \"52eb01f8-e86a-4458-8875-639394a87b30\") " pod="openshift-multus/network-metrics-daemon-vmrkz"
Oct 30 21:49:12.921740 ip-10-0-136-210 kubenswrapper[2576]: E1030 21:49:12.921414    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 21:49:12.921740 ip-10-0-136-210 kubenswrapper[2576]: E1030 21:49:12.921462    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/52eb01f8-e86a-4458-8875-639394a87b30-metrics-certs podName:52eb01f8-e86a-4458-8875-639394a87b30 nodeName:}" failed. No retries permitted until 2026-10-30 21:49:16.921448906 +0000 UTC m=+9.353397152 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/52eb01f8-e86a-4458-8875-639394a87b30-metrics-certs") pod "network-metrics-daemon-vmrkz" (UID: "52eb01f8-e86a-4458-8875-639394a87b30") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 21:49:12.921740 ip-10-0-136-210 kubenswrapper[2576]: E1030 21:49:12.921510    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 21:49:12.921740 ip-10-0-136-210 kubenswrapper[2576]: E1030 21:49:12.921533    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e810bcd1-78da-409f-b86e-08c47031939e-original-pull-secret podName:e810bcd1-78da-409f-b86e-08c47031939e nodeName:}" failed. No retries permitted until 2026-10-30 21:49:16.921525121 +0000 UTC m=+9.353473367 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e810bcd1-78da-409f-b86e-08c47031939e-original-pull-secret") pod "global-pull-secret-syncer-68jrf" (UID: "e810bcd1-78da-409f-b86e-08c47031939e") : object "kube-system"/"original-pull-secret" not registered
Oct 30 21:49:13.021806 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:13.021765    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fcjg4\" (UniqueName: \"kubernetes.io/projected/73c208eb-ffc0-46f2-8bd4-05ade5adb529-kube-api-access-fcjg4\") pod \"network-check-target-jmbng\" (UID: \"73c208eb-ffc0-46f2-8bd4-05ade5adb529\") " pod="openshift-network-diagnostics/network-check-target-jmbng"
Oct 30 21:49:13.022261 ip-10-0-136-210 kubenswrapper[2576]: E1030 21:49:13.022083    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 21:49:13.022261 ip-10-0-136-210 kubenswrapper[2576]: E1030 21:49:13.022111    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 21:49:13.022261 ip-10-0-136-210 kubenswrapper[2576]: E1030 21:49:13.022124    2576 projected.go:196] Error preparing data for projected volume kube-api-access-fcjg4 for pod openshift-network-diagnostics/network-check-target-jmbng: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 21:49:13.022261 ip-10-0-136-210 kubenswrapper[2576]: E1030 21:49:13.022190    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/73c208eb-ffc0-46f2-8bd4-05ade5adb529-kube-api-access-fcjg4 podName:73c208eb-ffc0-46f2-8bd4-05ade5adb529 nodeName:}" failed. No retries permitted until 2026-10-30 21:49:17.022169739 +0000 UTC m=+9.454118007 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-fcjg4" (UniqueName: "kubernetes.io/projected/73c208eb-ffc0-46f2-8bd4-05ade5adb529-kube-api-access-fcjg4") pod "network-check-target-jmbng" (UID: "73c208eb-ffc0-46f2-8bd4-05ade5adb529") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 21:49:13.209849 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:13.209767    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jmbng"
Oct 30 21:49:13.210009 ip-10-0-136-210 kubenswrapper[2576]: E1030 21:49:13.209895    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jmbng" podUID="73c208eb-ffc0-46f2-8bd4-05ade5adb529"
Oct 30 21:49:13.210009 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:13.209956    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vmrkz"
Oct 30 21:49:13.210113 ip-10-0-136-210 kubenswrapper[2576]: E1030 21:49:13.210035    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vmrkz" podUID="52eb01f8-e86a-4458-8875-639394a87b30"
Oct 30 21:49:13.210113 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:13.210076    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-68jrf"
Oct 30 21:49:13.210216 ip-10-0-136-210 kubenswrapper[2576]: E1030 21:49:13.210140    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-68jrf" podUID="e810bcd1-78da-409f-b86e-08c47031939e"
Oct 30 21:49:15.209628 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:15.209381    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-68jrf"
Oct 30 21:49:15.210217 ip-10-0-136-210 kubenswrapper[2576]: E1030 21:49:15.209743    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-68jrf" podUID="e810bcd1-78da-409f-b86e-08c47031939e"
Oct 30 21:49:15.210217 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:15.209843    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jmbng"
Oct 30 21:49:15.210217 ip-10-0-136-210 kubenswrapper[2576]: E1030 21:49:15.209932    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jmbng" podUID="73c208eb-ffc0-46f2-8bd4-05ade5adb529"
Oct 30 21:49:15.210217 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:15.210000    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vmrkz"
Oct 30 21:49:15.210217 ip-10-0-136-210 kubenswrapper[2576]: E1030 21:49:15.210088    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vmrkz" podUID="52eb01f8-e86a-4458-8875-639394a87b30"
Oct 30 21:49:15.262898 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:15.262804    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-210.us-west-2.compute.internal" event={"ID":"4492449458f4d650448405f5044e8d24","Type":"ContainerStarted","Data":"40c8a50d2211e2d2e13c659f3c218bbf16d947d371c5fed15f67fdc551f0f2b9"}
Oct 30 21:49:15.266376 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:15.265747    2576 generic.go:356] "Generic (PLEG): container finished" podID="dc220f0e-8a21-4d7f-96b3-e7055c242f42" containerID="9348fc680b899ad9da756ea1029e7953ef9fc6c9f44a85be282c7e73eb47d028" exitCode=0
Oct 30 21:49:15.266376 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:15.265826    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kggmc" event={"ID":"dc220f0e-8a21-4d7f-96b3-e7055c242f42","Type":"ContainerDied","Data":"9348fc680b899ad9da756ea1029e7953ef9fc6c9f44a85be282c7e73eb47d028"}
Oct 30 21:49:15.270605 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:15.270516    2576 generic.go:356] "Generic (PLEG): container finished" podID="d91a9843014f769d901fa6aba545e397" containerID="3415a45fd4be33eac12c60f14895f6c19f67a6765533f5e8df07bc8b45eeaa3a" exitCode=0
Oct 30 21:49:15.270605 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:15.270552    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-210.us-west-2.compute.internal" event={"ID":"d91a9843014f769d901fa6aba545e397","Type":"ContainerDied","Data":"3415a45fd4be33eac12c60f14895f6c19f67a6765533f5e8df07bc8b45eeaa3a"}
Oct 30 21:49:15.486646 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:15.486581    2576 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-210.us-west-2.compute.internal" podStartSLOduration=7.486561774 podStartE2EDuration="7.486561774s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 21:49:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:49:15.370542552 +0000 UTC m=+7.802490824" watchObservedRunningTime="2026-10-30 21:49:15.486561774 +0000 UTC m=+7.918510043"
Oct 30 21:49:16.284363 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:16.283670    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-5mw6n" event={"ID":"f69b6a2e-d33f-4cc8-8e0a-598b20948f38","Type":"ContainerStarted","Data":"373381119a751e15c79d00e92a8553b2b3edae685d445641a632299ef115e3c1"}
Oct 30 21:49:16.288916 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:16.288827    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-210.us-west-2.compute.internal" event={"ID":"d91a9843014f769d901fa6aba545e397","Type":"ContainerStarted","Data":"8fcdfc2e2f635ddec2d7a89cb41f62bd819304fde0f88ea2b4f298ea9ccf1270"}
Oct 30 21:49:16.514086 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:16.512985    2576 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/konnectivity-agent-5mw6n" podStartSLOduration=3.432133988 podStartE2EDuration="8.512962514s" totalImagesPullingTime="5.080828526s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:49:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:49:16.396651302 +0000 UTC m=+8.828599571" watchObservedRunningTime="2026-10-30 21:49:16.512962514 +0000 UTC m=+8.944910786"
Oct 30 21:49:16.764938 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:16.764845    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:16.957741 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:16.957636    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/52eb01f8-e86a-4458-8875-639394a87b30-metrics-certs\") pod \"network-metrics-daemon-vmrkz\" (UID: \"52eb01f8-e86a-4458-8875-639394a87b30\") " pod="openshift-multus/network-metrics-daemon-vmrkz"
Oct 30 21:49:16.957741 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:16.957729    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e810bcd1-78da-409f-b86e-08c47031939e-original-pull-secret\") pod \"global-pull-secret-syncer-68jrf\" (UID: \"e810bcd1-78da-409f-b86e-08c47031939e\") " pod="kube-system/global-pull-secret-syncer-68jrf"
Oct 30 21:49:16.957960 ip-10-0-136-210 kubenswrapper[2576]: E1030 21:49:16.957877    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 21:49:16.957960 ip-10-0-136-210 kubenswrapper[2576]: E1030 21:49:16.957940    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e810bcd1-78da-409f-b86e-08c47031939e-original-pull-secret podName:e810bcd1-78da-409f-b86e-08c47031939e nodeName:}" failed. No retries permitted until 2026-10-30 21:49:24.957921241 +0000 UTC m=+17.389869487 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e810bcd1-78da-409f-b86e-08c47031939e-original-pull-secret") pod "global-pull-secret-syncer-68jrf" (UID: "e810bcd1-78da-409f-b86e-08c47031939e") : object "kube-system"/"original-pull-secret" not registered
Oct 30 21:49:16.958065 ip-10-0-136-210 kubenswrapper[2576]: E1030 21:49:16.958010    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 21:49:16.958065 ip-10-0-136-210 kubenswrapper[2576]: E1030 21:49:16.958044    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/52eb01f8-e86a-4458-8875-639394a87b30-metrics-certs podName:52eb01f8-e86a-4458-8875-639394a87b30 nodeName:}" failed. No retries permitted until 2026-10-30 21:49:24.958032961 +0000 UTC m=+17.389981210 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/52eb01f8-e86a-4458-8875-639394a87b30-metrics-certs") pod "network-metrics-daemon-vmrkz" (UID: "52eb01f8-e86a-4458-8875-639394a87b30") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 21:49:17.059049 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:17.059007    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fcjg4\" (UniqueName: \"kubernetes.io/projected/73c208eb-ffc0-46f2-8bd4-05ade5adb529-kube-api-access-fcjg4\") pod \"network-check-target-jmbng\" (UID: \"73c208eb-ffc0-46f2-8bd4-05ade5adb529\") " pod="openshift-network-diagnostics/network-check-target-jmbng"
Oct 30 21:49:17.059245 ip-10-0-136-210 kubenswrapper[2576]: E1030 21:49:17.059227    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 21:49:17.059310 ip-10-0-136-210 kubenswrapper[2576]: E1030 21:49:17.059251    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 21:49:17.059310 ip-10-0-136-210 kubenswrapper[2576]: E1030 21:49:17.059265    2576 projected.go:196] Error preparing data for projected volume kube-api-access-fcjg4 for pod openshift-network-diagnostics/network-check-target-jmbng: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 21:49:17.059403 ip-10-0-136-210 kubenswrapper[2576]: E1030 21:49:17.059322    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/73c208eb-ffc0-46f2-8bd4-05ade5adb529-kube-api-access-fcjg4 podName:73c208eb-ffc0-46f2-8bd4-05ade5adb529 nodeName:}" failed. No retries permitted until 2026-10-30 21:49:25.059303788 +0000 UTC m=+17.491252048 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-fcjg4" (UniqueName: "kubernetes.io/projected/73c208eb-ffc0-46f2-8bd4-05ade5adb529-kube-api-access-fcjg4") pod "network-check-target-jmbng" (UID: "73c208eb-ffc0-46f2-8bd4-05ade5adb529") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 21:49:17.208765 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:17.208691    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-68jrf"
Oct 30 21:49:17.208912 ip-10-0-136-210 kubenswrapper[2576]: E1030 21:49:17.208833    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-68jrf" podUID="e810bcd1-78da-409f-b86e-08c47031939e"
Oct 30 21:49:17.208912 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:17.208899    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jmbng"
Oct 30 21:49:17.209044 ip-10-0-136-210 kubenswrapper[2576]: E1030 21:49:17.208964    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jmbng" podUID="73c208eb-ffc0-46f2-8bd4-05ade5adb529"
Oct 30 21:49:17.209044 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:17.209020    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vmrkz"
Oct 30 21:49:17.209153 ip-10-0-136-210 kubenswrapper[2576]: E1030 21:49:17.209094    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vmrkz" podUID="52eb01f8-e86a-4458-8875-639394a87b30"
Oct 30 21:49:17.294849 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:17.294820    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-xkrx5" event={"ID":"04465eda-e6db-42d3-8d5f-c9be668038a9","Type":"ContainerStarted","Data":"08146d1f27c08d88de4ff9dc25991a09c48ae732cd6d77f070d3c740c8bc9230"}
Oct 30 21:49:17.406082 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:17.406035    2576 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-210.us-west-2.compute.internal" podStartSLOduration=9.40602003 podStartE2EDuration="9.40602003s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 21:49:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:49:16.514021772 +0000 UTC m=+8.945970041" watchObservedRunningTime="2026-10-30 21:49:17.40602003 +0000 UTC m=+9.837968297"
Oct 30 21:49:18.302122 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:18.302083    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cqxms" event={"ID":"d7c355fa-2141-463e-8cbc-bae3c44a16f1","Type":"ContainerStarted","Data":"bee0095e1abb06aabe800445c1f7d297c09846c51b5808aa3aa9059c0ddee584"}
Oct 30 21:49:19.209481 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:19.209411    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vmrkz"
Oct 30 21:49:19.209641 ip-10-0-136-210 kubenswrapper[2576]: E1030 21:49:19.209524    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vmrkz" podUID="52eb01f8-e86a-4458-8875-639394a87b30"
Oct 30 21:49:19.209641 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:19.209589    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-68jrf"
Oct 30 21:49:19.209778 ip-10-0-136-210 kubenswrapper[2576]: E1030 21:49:19.209692    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-68jrf" podUID="e810bcd1-78da-409f-b86e-08c47031939e"
Oct 30 21:49:19.209778 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:19.209753    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jmbng"
Oct 30 21:49:19.209877 ip-10-0-136-210 kubenswrapper[2576]: E1030 21:49:19.209830    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jmbng" podUID="73c208eb-ffc0-46f2-8bd4-05ade5adb529"
Oct 30 21:49:19.306802 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:19.306768    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-v8j9s" event={"ID":"c8b13e55-b3c7-4656-9845-16749704265f","Type":"ContainerStarted","Data":"c884febfc0ba71e159b36d2bdf0bccfdc5a4284c8171a787836610e99a61debf"}
Oct 30 21:49:19.410068 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:19.409732    2576 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Oct 30 21:49:19.419806 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:19.419763    2576 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-image-registry/node-ca-xkrx5" podStartSLOduration=5.319531548 podStartE2EDuration="11.419750653s" totalImagesPullingTime="6.100219105s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 21:49:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:49:17.406278412 +0000 UTC m=+9.838226681" watchObservedRunningTime="2026-10-30 21:49:19.419750653 +0000 UTC m=+11.851698964"
Oct 30 21:49:20.153866 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:20.153106    2576 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-10-30T21:49:19.409753886Z","UUID":"83e01fd5-5ee9-41aa-9915-c9a2dc2a6116","Handler":null,"Name":"","Endpoint":""}
Oct 30 21:49:20.155578 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:20.155152    2576 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Oct 30 21:49:20.155578 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:20.155184    2576 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Oct 30 21:49:20.311405 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:20.311371    2576 generic.go:356] "Generic (PLEG): container finished" podID="dc220f0e-8a21-4d7f-96b3-e7055c242f42" containerID="e39db2e6b3e23bfcd306c9f3a561f3eb57cd334e51521b9cb5cfd411da5980ec" exitCode=0
Oct 30 21:49:20.311859 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:20.311462    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kggmc" event={"ID":"dc220f0e-8a21-4d7f-96b3-e7055c242f42","Type":"ContainerDied","Data":"e39db2e6b3e23bfcd306c9f3a561f3eb57cd334e51521b9cb5cfd411da5980ec"}
Oct 30 21:49:20.315351 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:20.313952    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-krtnr" event={"ID":"db4b06a3-45e5-4d45-9d7a-407d117d9454","Type":"ContainerStarted","Data":"ac8fea310a7d7156e467556e4347f32366b26a725945a3093673ec09831b7d17"}
Oct 30 21:49:20.322018 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:20.321882    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cqxms" event={"ID":"d7c355fa-2141-463e-8cbc-bae3c44a16f1","Type":"ContainerStarted","Data":"6d79896b88aaf76fd73c3814da25313d63a4bae793f8f94586f992566b398868"}
Oct 30 21:49:20.428340 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:20.428074    2576 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-v8j9s" podStartSLOduration=3.929409611 podStartE2EDuration="12.428056775s" totalImagesPullingTime="8.498647164s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 21:49:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:49:19.420752145 +0000 UTC m=+11.852700415" watchObservedRunningTime="2026-10-30 21:49:20.428056775 +0000 UTC m=+12.860005045"
Oct 30 21:49:20.537528 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:20.537477    2576 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-dns/node-resolver-krtnr" podStartSLOduration=3.959526979 podStartE2EDuration="12.537459281s" totalImagesPullingTime="8.577932302s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 21:49:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:49:20.536777295 +0000 UTC m=+12.968725563" watchObservedRunningTime="2026-10-30 21:49:20.537459281 +0000 UTC m=+12.969407551"
Oct 30 21:49:20.869350 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:20.869317    2576 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-5mw6n"
Oct 30 21:49:20.870611 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:20.870427    2576 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-5mw6n"
Oct 30 21:49:20.968697 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:20.968626    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:21.209061 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:21.208984    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-68jrf"
Oct 30 21:49:21.209191 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:21.209098    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jmbng"
Oct 30 21:49:21.209191 ip-10-0-136-210 kubenswrapper[2576]: E1030 21:49:21.209116    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-68jrf" podUID="e810bcd1-78da-409f-b86e-08c47031939e"
Oct 30 21:49:21.209191 ip-10-0-136-210 kubenswrapper[2576]: E1030 21:49:21.209173    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jmbng" podUID="73c208eb-ffc0-46f2-8bd4-05ade5adb529"
Oct 30 21:49:21.209313 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:21.209217    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vmrkz"
Oct 30 21:49:21.209313 ip-10-0-136-210 kubenswrapper[2576]: E1030 21:49:21.209273    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vmrkz" podUID="52eb01f8-e86a-4458-8875-639394a87b30"
Oct 30 21:49:21.222360 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:21.222320    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:21.325712 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:21.325680    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-l9d98" event={"ID":"0ab70415-9f1d-4988-a228-df6ed621f0d8","Type":"ContainerStarted","Data":"daeb892c45a78a724fd3e4d66920ec9eb8372902a8e60d619e7b4fb686ccb52a"}
Oct 30 21:49:21.328304 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:21.328278    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cqxms" event={"ID":"d7c355fa-2141-463e-8cbc-bae3c44a16f1","Type":"ContainerStarted","Data":"3176b116f02b747fb96dd6d6963c1641e7b29f23d951b6f6f8ba45a2499c82d6"}
Oct 30 21:49:21.329093 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:21.329073    2576 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-5mw6n"
Oct 30 21:49:21.329486 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:21.329461    2576 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-5mw6n"
Oct 30 21:49:21.435007 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:21.434952    2576 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-l9d98" podStartSLOduration=4.469016236 podStartE2EDuration="13.434931986s" totalImagesPullingTime="8.96591575s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 21:49:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:49:21.434397936 +0000 UTC m=+13.866346249" watchObservedRunningTime="2026-10-30 21:49:21.434931986 +0000 UTC m=+13.866880257"
Oct 30 21:49:21.460206 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:21.460133    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:21.548501 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:21.548434    2576 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cqxms" podStartSLOduration=3.402588756 podStartE2EDuration="13.548415269s" totalImagesPullingTime="10.145826513s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 21:49:08 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:49:21.548110301 +0000 UTC m=+13.980058568" watchObservedRunningTime="2026-10-30 21:49:21.548415269 +0000 UTC m=+13.980363538"
Oct 30 21:49:21.783898 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:21.783852    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:22.332532 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:22.332496    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-ldvhf" event={"ID":"8f2f95c4-21b2-477b-a2cb-7c8590ec340d","Type":"ContainerStarted","Data":"8ac4122210cf94a81493fd51b41117106789a972ba240569aa5d7166734f20f9"}
Oct 30 21:49:22.335158 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:22.335124    2576 generic.go:356] "Generic (PLEG): container finished" podID="dc220f0e-8a21-4d7f-96b3-e7055c242f42" containerID="523893fe178fa198884903d08e16609ad04a306f22bf92f6c1cd30f9951ca09f" exitCode=0
Oct 30 21:49:22.335290 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:22.335239    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kggmc" event={"ID":"dc220f0e-8a21-4d7f-96b3-e7055c242f42","Type":"ContainerDied","Data":"523893fe178fa198884903d08e16609ad04a306f22bf92f6c1cd30f9951ca09f"}
Oct 30 21:49:22.445237 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:22.445175    2576 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-ldvhf" podStartSLOduration=3.646318558 podStartE2EDuration="14.445155898s" totalImagesPullingTime="10.79883734s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 21:49:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:49:22.444465033 +0000 UTC m=+14.876413301" watchObservedRunningTime="2026-10-30 21:49:22.445155898 +0000 UTC m=+14.877104167"
Oct 30 21:49:23.209589 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:23.209558    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-68jrf"
Oct 30 21:49:23.209766 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:23.209604    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jmbng"
Oct 30 21:49:23.209766 ip-10-0-136-210 kubenswrapper[2576]: E1030 21:49:23.209678    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-68jrf" podUID="e810bcd1-78da-409f-b86e-08c47031939e"
Oct 30 21:49:23.209766 ip-10-0-136-210 kubenswrapper[2576]: E1030 21:49:23.209723    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jmbng" podUID="73c208eb-ffc0-46f2-8bd4-05ade5adb529"
Oct 30 21:49:23.209943 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:23.209772    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vmrkz"
Oct 30 21:49:23.209943 ip-10-0-136-210 kubenswrapper[2576]: E1030 21:49:23.209875    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vmrkz" podUID="52eb01f8-e86a-4458-8875-639394a87b30"
Oct 30 21:49:24.283100 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:24.283064    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:24.342064 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:24.342028    2576 generic.go:356] "Generic (PLEG): container finished" podID="dc220f0e-8a21-4d7f-96b3-e7055c242f42" containerID="bec9121f537bc927169b50f8de94a73a09a472c6289bf5d6f1b23300d9d9abe7" exitCode=0
Oct 30 21:49:24.342229 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:24.342091    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kggmc" event={"ID":"dc220f0e-8a21-4d7f-96b3-e7055c242f42","Type":"ContainerDied","Data":"bec9121f537bc927169b50f8de94a73a09a472c6289bf5d6f1b23300d9d9abe7"}
Oct 30 21:49:24.469205 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:24.469171    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:24.712522 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:24.712442    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:24.912315 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:24.912073    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:25.019365 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:25.019330    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/52eb01f8-e86a-4458-8875-639394a87b30-metrics-certs\") pod \"network-metrics-daemon-vmrkz\" (UID: \"52eb01f8-e86a-4458-8875-639394a87b30\") " pod="openshift-multus/network-metrics-daemon-vmrkz"
Oct 30 21:49:25.019522 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:25.019402    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e810bcd1-78da-409f-b86e-08c47031939e-original-pull-secret\") pod \"global-pull-secret-syncer-68jrf\" (UID: \"e810bcd1-78da-409f-b86e-08c47031939e\") " pod="kube-system/global-pull-secret-syncer-68jrf"
Oct 30 21:49:25.019522 ip-10-0-136-210 kubenswrapper[2576]: E1030 21:49:25.019490    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 21:49:25.019592 ip-10-0-136-210 kubenswrapper[2576]: E1030 21:49:25.019553    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/52eb01f8-e86a-4458-8875-639394a87b30-metrics-certs podName:52eb01f8-e86a-4458-8875-639394a87b30 nodeName:}" failed. No retries permitted until 2026-10-30 21:49:41.019538518 +0000 UTC m=+33.451486768 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/52eb01f8-e86a-4458-8875-639394a87b30-metrics-certs") pod "network-metrics-daemon-vmrkz" (UID: "52eb01f8-e86a-4458-8875-639394a87b30") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 21:49:25.019592 ip-10-0-136-210 kubenswrapper[2576]: E1030 21:49:25.019492    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 21:49:25.019699 ip-10-0-136-210 kubenswrapper[2576]: E1030 21:49:25.019621    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e810bcd1-78da-409f-b86e-08c47031939e-original-pull-secret podName:e810bcd1-78da-409f-b86e-08c47031939e nodeName:}" failed. No retries permitted until 2026-10-30 21:49:41.019609278 +0000 UTC m=+33.451557537 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e810bcd1-78da-409f-b86e-08c47031939e-original-pull-secret") pod "global-pull-secret-syncer-68jrf" (UID: "e810bcd1-78da-409f-b86e-08c47031939e") : object "kube-system"/"original-pull-secret" not registered
Oct 30 21:49:25.116110 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:25.116073    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:25.119968 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:25.119935    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fcjg4\" (UniqueName: \"kubernetes.io/projected/73c208eb-ffc0-46f2-8bd4-05ade5adb529-kube-api-access-fcjg4\") pod \"network-check-target-jmbng\" (UID: \"73c208eb-ffc0-46f2-8bd4-05ade5adb529\") " pod="openshift-network-diagnostics/network-check-target-jmbng"
Oct 30 21:49:25.120117 ip-10-0-136-210 kubenswrapper[2576]: E1030 21:49:25.120098    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 21:49:25.120182 ip-10-0-136-210 kubenswrapper[2576]: E1030 21:49:25.120124    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 21:49:25.120182 ip-10-0-136-210 kubenswrapper[2576]: E1030 21:49:25.120137    2576 projected.go:196] Error preparing data for projected volume kube-api-access-fcjg4 for pod openshift-network-diagnostics/network-check-target-jmbng: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 21:49:25.120308 ip-10-0-136-210 kubenswrapper[2576]: E1030 21:49:25.120200    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/73c208eb-ffc0-46f2-8bd4-05ade5adb529-kube-api-access-fcjg4 podName:73c208eb-ffc0-46f2-8bd4-05ade5adb529 nodeName:}" failed. No retries permitted until 2026-10-30 21:49:41.120180933 +0000 UTC m=+33.552129180 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-fcjg4" (UniqueName: "kubernetes.io/projected/73c208eb-ffc0-46f2-8bd4-05ade5adb529-kube-api-access-fcjg4") pod "network-check-target-jmbng" (UID: "73c208eb-ffc0-46f2-8bd4-05ade5adb529") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 21:49:25.209486 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:25.209452    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-68jrf"
Oct 30 21:49:25.209694 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:25.209454    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vmrkz"
Oct 30 21:49:25.209694 ip-10-0-136-210 kubenswrapper[2576]: E1030 21:49:25.209577    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-68jrf" podUID="e810bcd1-78da-409f-b86e-08c47031939e"
Oct 30 21:49:25.209694 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:25.209454    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jmbng"
Oct 30 21:49:25.209830 ip-10-0-136-210 kubenswrapper[2576]: E1030 21:49:25.209697    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vmrkz" podUID="52eb01f8-e86a-4458-8875-639394a87b30"
Oct 30 21:49:25.209830 ip-10-0-136-210 kubenswrapper[2576]: E1030 21:49:25.209760    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jmbng" podUID="73c208eb-ffc0-46f2-8bd4-05ade5adb529"
Oct 30 21:49:25.292918 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:25.292807    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:26.305436 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:26.305398    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:26.350010 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:26.349975    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wrxww" event={"ID":"d5e5ac6e-d37d-457d-9ca7-93065f5925ff","Type":"ContainerStarted","Data":"c0e8af6577f818587d1add5d46f60c628f433df80239cc50d6ac5c9b8a1e0496"}
Oct 30 21:49:26.350155 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:26.350050    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wrxww" event={"ID":"d5e5ac6e-d37d-457d-9ca7-93065f5925ff","Type":"ContainerStarted","Data":"11ea0b7727fd42f9559e275157c7c69358a02947b991293d9a0496eac1e4ef71"}
Oct 30 21:49:26.350155 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:26.350081    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wrxww" event={"ID":"d5e5ac6e-d37d-457d-9ca7-93065f5925ff","Type":"ContainerStarted","Data":"0d4632c5f87aaf8717cee175c356dc13cda703bbdcf74adfad84d3e30d926806"}
Oct 30 21:49:26.350155 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:26.350108    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wrxww" event={"ID":"d5e5ac6e-d37d-457d-9ca7-93065f5925ff","Type":"ContainerStarted","Data":"7d76b64d6571e03422491cb274fe60abbdfa4f7f34830a29716e8aa8cd3c162c"}
Oct 30 21:49:26.350155 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:26.350136    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wrxww" event={"ID":"d5e5ac6e-d37d-457d-9ca7-93065f5925ff","Type":"ContainerStarted","Data":"f6872deb775deed0c655f36879ceaa16ea949d75a94134592e7391dd76c9a978"}
Oct 30 21:49:26.350291 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:26.350174    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wrxww" event={"ID":"d5e5ac6e-d37d-457d-9ca7-93065f5925ff","Type":"ContainerStarted","Data":"792872b0e079d27e4b3554e6a02485b415d463355af81b6a35a799ac3dc240e2"}
Oct 30 21:49:26.494517 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:26.494485    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:27.208780 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:27.208740    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-68jrf"
Oct 30 21:49:27.208780 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:27.208772    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vmrkz"
Oct 30 21:49:27.208985 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:27.208760    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jmbng"
Oct 30 21:49:27.208985 ip-10-0-136-210 kubenswrapper[2576]: E1030 21:49:27.208841    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-68jrf" podUID="e810bcd1-78da-409f-b86e-08c47031939e"
Oct 30 21:49:27.208985 ip-10-0-136-210 kubenswrapper[2576]: E1030 21:49:27.208896    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jmbng" podUID="73c208eb-ffc0-46f2-8bd4-05ade5adb529"
Oct 30 21:49:27.208985 ip-10-0-136-210 kubenswrapper[2576]: E1030 21:49:27.208961    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vmrkz" podUID="52eb01f8-e86a-4458-8875-639394a87b30"
Oct 30 21:49:27.354379 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:27.354346    2576 generic.go:356] "Generic (PLEG): container finished" podID="dc220f0e-8a21-4d7f-96b3-e7055c242f42" containerID="78543d599edd4758c0436abb669aa311f082e55f713b57ab479ebddb035a78cc" exitCode=0
Oct 30 21:49:27.354756 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:27.354390    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kggmc" event={"ID":"dc220f0e-8a21-4d7f-96b3-e7055c242f42","Type":"ContainerDied","Data":"78543d599edd4758c0436abb669aa311f082e55f713b57ab479ebddb035a78cc"}
Oct 30 21:49:27.395339 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:27.395308    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:28.054555 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:28.054528    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:28.263766 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:28.263730    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:28.360065 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:28.359976    2576 generic.go:356] "Generic (PLEG): container finished" podID="dc220f0e-8a21-4d7f-96b3-e7055c242f42" containerID="5609661adcd1a00bf7e06cdf2ed4015b56e6d563020fdb03aded3854573f531d" exitCode=0
Oct 30 21:49:28.360649 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:28.360062    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kggmc" event={"ID":"dc220f0e-8a21-4d7f-96b3-e7055c242f42","Type":"ContainerDied","Data":"5609661adcd1a00bf7e06cdf2ed4015b56e6d563020fdb03aded3854573f531d"}
Oct 30 21:49:28.363476 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:28.363458    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wrxww" event={"ID":"d5e5ac6e-d37d-457d-9ca7-93065f5925ff","Type":"ContainerStarted","Data":"04eaacc37f7fcf73fb1899a64f416a11a6fb141aae5922c0560f5299fcac3147"}
Oct 30 21:49:28.574847 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:28.574813    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:28.767162 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:28.767128    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:29.013855 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:29.013798    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:29.208649 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:29.208578    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-68jrf"
Oct 30 21:49:29.208805 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:29.208714    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jmbng"
Oct 30 21:49:29.208805 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:29.208734    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vmrkz"
Oct 30 21:49:29.208805 ip-10-0-136-210 kubenswrapper[2576]: E1030 21:49:29.208728    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-68jrf" podUID="e810bcd1-78da-409f-b86e-08c47031939e"
Oct 30 21:49:29.208901 ip-10-0-136-210 kubenswrapper[2576]: E1030 21:49:29.208799    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jmbng" podUID="73c208eb-ffc0-46f2-8bd4-05ade5adb529"
Oct 30 21:49:29.208901 ip-10-0-136-210 kubenswrapper[2576]: E1030 21:49:29.208883    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vmrkz" podUID="52eb01f8-e86a-4458-8875-639394a87b30"
Oct 30 21:49:29.370838 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:29.370804    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kggmc" event={"ID":"dc220f0e-8a21-4d7f-96b3-e7055c242f42","Type":"ContainerStarted","Data":"e25d56a7bd11507cba3a66ef55ed3f27c466c7cef6fb4fdd3eaf1484e7679e7b"}
Oct 30 21:49:29.408721 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:29.408694    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-krtnr_db4b06a3-45e5-4d45-9d7a-407d117d9454/dns-node-resolver/0.log"
Oct 30 21:49:29.490649 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:29.490569    2576 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-kggmc" podStartSLOduration=8.785126983 podStartE2EDuration="21.490554872s" totalImagesPullingTime="12.705427889s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 21:49:08 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:49:29.4901158 +0000 UTC m=+21.922064067" watchObservedRunningTime="2026-10-30 21:49:29.490554872 +0000 UTC m=+21.922503148"
Oct 30 21:49:29.811921 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:29.811886    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:30.051136 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:30.051099    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:30.314146 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:30.314121    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-xkrx5_04465eda-e6db-42d3-8d5f-c9be668038a9/node-ca/0.log"
Oct 30 21:49:30.377576 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:30.377541    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wrxww" event={"ID":"d5e5ac6e-d37d-457d-9ca7-93065f5925ff","Type":"ContainerStarted","Data":"28619bba160713cde105f585c9ec4f79f8660c24dc7778cc6ecc40d08ca8b29a"}
Oct 30 21:49:30.425945 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:30.425913    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:30.613100 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:30.613067    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:30.641171 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:30.641130    2576 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-wrxww" podStartSLOduration=7.585924509 podStartE2EDuration="22.641118091s" totalImagesPullingTime="15.055193582s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 21:49:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:49:30.593899791 +0000 UTC m=+23.025848059" watchObservedRunningTime="2026-10-30 21:49:30.641118091 +0000 UTC m=+23.073066401"
Oct 30 21:49:30.810803 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:30.810770    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:31.032911 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:31.032886    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:31.208751 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:31.208719    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jmbng"
Oct 30 21:49:31.208751 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:31.208749    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-68jrf"
Oct 30 21:49:31.209094 ip-10-0-136-210 kubenswrapper[2576]: E1030 21:49:31.208835    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jmbng" podUID="73c208eb-ffc0-46f2-8bd4-05ade5adb529"
Oct 30 21:49:31.209094 ip-10-0-136-210 kubenswrapper[2576]: E1030 21:49:31.208870    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-68jrf" podUID="e810bcd1-78da-409f-b86e-08c47031939e"
Oct 30 21:49:31.209094 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:31.208933    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vmrkz"
Oct 30 21:49:31.209094 ip-10-0-136-210 kubenswrapper[2576]: E1030 21:49:31.209013    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vmrkz" podUID="52eb01f8-e86a-4458-8875-639394a87b30"
Oct 30 21:49:31.229025 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:31.228998    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:31.380487 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:31.380400    2576 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-wrxww"
Oct 30 21:49:31.380487 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:31.380464    2576 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-wrxww"
Oct 30 21:49:31.380921 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:31.380504    2576 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-wrxww"
Oct 30 21:49:31.395945 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:31.395921    2576 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-wrxww"
Oct 30 21:49:31.396122 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:31.396112    2576 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-wrxww"
Oct 30 21:49:31.827956 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:31.827929    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:32.013803 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:32.013766    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:32.419637 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:32.419602    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:32.614334 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:32.614295    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:33.153737 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:33.153699    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:33.209268 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:33.209240    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-68jrf"
Oct 30 21:49:33.209418 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:33.209310    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vmrkz"
Oct 30 21:49:33.209418 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:33.209398    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jmbng"
Oct 30 21:49:33.209495 ip-10-0-136-210 kubenswrapper[2576]: E1030 21:49:33.209409    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-68jrf" podUID="e810bcd1-78da-409f-b86e-08c47031939e"
Oct 30 21:49:33.209495 ip-10-0-136-210 kubenswrapper[2576]: E1030 21:49:33.209458    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jmbng" podUID="73c208eb-ffc0-46f2-8bd4-05ade5adb529"
Oct 30 21:49:33.209568 ip-10-0-136-210 kubenswrapper[2576]: E1030 21:49:33.209536    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vmrkz" podUID="52eb01f8-e86a-4458-8875-639394a87b30"
Oct 30 21:49:33.481675 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:33.481596    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:33.825242 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:33.825214    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:34.028713 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:34.028673    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:34.212444 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:34.212325    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-68jrf"
Oct 30 21:49:34.212578 ip-10-0-136-210 kubenswrapper[2576]: E1030 21:49:34.212440    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-68jrf" podUID="e810bcd1-78da-409f-b86e-08c47031939e"
Oct 30 21:49:34.418548 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:34.418511    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:34.625978 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:34.625948    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:35.037253 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:35.037228    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:35.208796 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:35.208769    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jmbng"
Oct 30 21:49:35.208969 ip-10-0-136-210 kubenswrapper[2576]: E1030 21:49:35.208861    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jmbng" podUID="73c208eb-ffc0-46f2-8bd4-05ade5adb529"
Oct 30 21:49:35.208969 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:35.208923    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vmrkz"
Oct 30 21:49:35.209083 ip-10-0-136-210 kubenswrapper[2576]: E1030 21:49:35.209039    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vmrkz" podUID="52eb01f8-e86a-4458-8875-639394a87b30"
Oct 30 21:49:35.548172 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:35.548136    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:35.834009 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:35.833930    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:36.021050 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:36.021016    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:36.208705 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:36.208611    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-68jrf"
Oct 30 21:49:36.208855 ip-10-0-136-210 kubenswrapper[2576]: E1030 21:49:36.208745    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-68jrf" podUID="e810bcd1-78da-409f-b86e-08c47031939e"
Oct 30 21:49:36.419365 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:36.419334    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:36.622076 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:36.622047    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:37.121294 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:37.121266    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:37.209625 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:37.209429    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jmbng"
Oct 30 21:49:37.209829 ip-10-0-136-210 kubenswrapper[2576]: E1030 21:49:37.209773    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jmbng" podUID="73c208eb-ffc0-46f2-8bd4-05ade5adb529"
Oct 30 21:49:37.209829 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:37.209814    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vmrkz"
Oct 30 21:49:37.209958 ip-10-0-136-210 kubenswrapper[2576]: E1030 21:49:37.209939    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vmrkz" podUID="52eb01f8-e86a-4458-8875-639394a87b30"
Oct 30 21:49:37.369539 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:37.369502    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:37.567333 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:37.567301    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:37.796366 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:37.796333    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:38.033215 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:38.033181    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:38.210030 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:38.209996    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-68jrf"
Oct 30 21:49:38.210469 ip-10-0-136-210 kubenswrapper[2576]: E1030 21:49:38.210098    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-68jrf" podUID="e810bcd1-78da-409f-b86e-08c47031939e"
Oct 30 21:49:38.238151 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:38.238127    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:38.436737 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:38.436642    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:38.630751 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:38.630715    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:38.836388 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:38.836364    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:39.036624 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:39.036597    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:39.209236 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:39.209162    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vmrkz"
Oct 30 21:49:39.209367 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:39.209166    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jmbng"
Oct 30 21:49:39.209367 ip-10-0-136-210 kubenswrapper[2576]: E1030 21:49:39.209264    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vmrkz" podUID="52eb01f8-e86a-4458-8875-639394a87b30"
Oct 30 21:49:39.209367 ip-10-0-136-210 kubenswrapper[2576]: E1030 21:49:39.209322    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jmbng" podUID="73c208eb-ffc0-46f2-8bd4-05ade5adb529"
Oct 30 21:49:39.253643 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:39.253612    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:39.459628 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:39.459542    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:40.046159 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:40.046130    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:40.208953 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:40.208925    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-68jrf"
Oct 30 21:49:40.209091 ip-10-0-136-210 kubenswrapper[2576]: E1030 21:49:40.209023    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-68jrf" podUID="e810bcd1-78da-409f-b86e-08c47031939e"
Oct 30 21:49:41.036725 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:41.036691    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/52eb01f8-e86a-4458-8875-639394a87b30-metrics-certs\") pod \"network-metrics-daemon-vmrkz\" (UID: \"52eb01f8-e86a-4458-8875-639394a87b30\") " pod="openshift-multus/network-metrics-daemon-vmrkz"
Oct 30 21:49:41.037123 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:41.036748    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e810bcd1-78da-409f-b86e-08c47031939e-original-pull-secret\") pod \"global-pull-secret-syncer-68jrf\" (UID: \"e810bcd1-78da-409f-b86e-08c47031939e\") " pod="kube-system/global-pull-secret-syncer-68jrf"
Oct 30 21:49:41.037123 ip-10-0-136-210 kubenswrapper[2576]: E1030 21:49:41.036836    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 21:49:41.037123 ip-10-0-136-210 kubenswrapper[2576]: E1030 21:49:41.036837    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 21:49:41.037123 ip-10-0-136-210 kubenswrapper[2576]: E1030 21:49:41.036883    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e810bcd1-78da-409f-b86e-08c47031939e-original-pull-secret podName:e810bcd1-78da-409f-b86e-08c47031939e nodeName:}" failed. No retries permitted until 2026-10-30 21:50:13.036871161 +0000 UTC m=+65.468819421 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e810bcd1-78da-409f-b86e-08c47031939e-original-pull-secret") pod "global-pull-secret-syncer-68jrf" (UID: "e810bcd1-78da-409f-b86e-08c47031939e") : object "kube-system"/"original-pull-secret" not registered
Oct 30 21:49:41.037123 ip-10-0-136-210 kubenswrapper[2576]: E1030 21:49:41.036898    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/52eb01f8-e86a-4458-8875-639394a87b30-metrics-certs podName:52eb01f8-e86a-4458-8875-639394a87b30 nodeName:}" failed. No retries permitted until 2026-10-30 21:50:13.036891183 +0000 UTC m=+65.468839429 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/52eb01f8-e86a-4458-8875-639394a87b30-metrics-certs") pod "network-metrics-daemon-vmrkz" (UID: "52eb01f8-e86a-4458-8875-639394a87b30") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 21:49:41.137815 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:41.137785    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fcjg4\" (UniqueName: \"kubernetes.io/projected/73c208eb-ffc0-46f2-8bd4-05ade5adb529-kube-api-access-fcjg4\") pod \"network-check-target-jmbng\" (UID: \"73c208eb-ffc0-46f2-8bd4-05ade5adb529\") " pod="openshift-network-diagnostics/network-check-target-jmbng"
Oct 30 21:49:41.137954 ip-10-0-136-210 kubenswrapper[2576]: E1030 21:49:41.137893    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 21:49:41.137954 ip-10-0-136-210 kubenswrapper[2576]: E1030 21:49:41.137906    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 21:49:41.137954 ip-10-0-136-210 kubenswrapper[2576]: E1030 21:49:41.137915    2576 projected.go:196] Error preparing data for projected volume kube-api-access-fcjg4 for pod openshift-network-diagnostics/network-check-target-jmbng: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 21:49:41.138042 ip-10-0-136-210 kubenswrapper[2576]: E1030 21:49:41.137966    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/73c208eb-ffc0-46f2-8bd4-05ade5adb529-kube-api-access-fcjg4 podName:73c208eb-ffc0-46f2-8bd4-05ade5adb529 nodeName:}" failed. No retries permitted until 2026-10-30 21:50:13.137954387 +0000 UTC m=+65.569902646 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-fcjg4" (UniqueName: "kubernetes.io/projected/73c208eb-ffc0-46f2-8bd4-05ade5adb529-kube-api-access-fcjg4") pod "network-check-target-jmbng" (UID: "73c208eb-ffc0-46f2-8bd4-05ade5adb529") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 21:49:41.208934 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:41.208903    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jmbng"
Oct 30 21:49:41.209060 ip-10-0-136-210 kubenswrapper[2576]: E1030 21:49:41.209008    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jmbng" podUID="73c208eb-ffc0-46f2-8bd4-05ade5adb529"
Oct 30 21:49:41.209060 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:41.209033    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vmrkz"
Oct 30 21:49:41.209125 ip-10-0-136-210 kubenswrapper[2576]: E1030 21:49:41.209096    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vmrkz" podUID="52eb01f8-e86a-4458-8875-639394a87b30"
Oct 30 21:49:42.208873 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:42.208836    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-68jrf"
Oct 30 21:49:42.209484 ip-10-0-136-210 kubenswrapper[2576]: E1030 21:49:42.208967    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-68jrf" podUID="e810bcd1-78da-409f-b86e-08c47031939e"
Oct 30 21:49:43.209173 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:43.209140    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vmrkz"
Oct 30 21:49:43.209582 ip-10-0-136-210 kubenswrapper[2576]: E1030 21:49:43.209252    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vmrkz" podUID="52eb01f8-e86a-4458-8875-639394a87b30"
Oct 30 21:49:43.209582 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:43.209286    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jmbng"
Oct 30 21:49:43.209582 ip-10-0-136-210 kubenswrapper[2576]: E1030 21:49:43.209333    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jmbng" podUID="73c208eb-ffc0-46f2-8bd4-05ade5adb529"
Oct 30 21:49:44.208642 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:44.208615    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-68jrf"
Oct 30 21:49:44.208817 ip-10-0-136-210 kubenswrapper[2576]: E1030 21:49:44.208725    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-68jrf" podUID="e810bcd1-78da-409f-b86e-08c47031939e"
Oct 30 21:49:45.208902 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:45.208865    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vmrkz"
Oct 30 21:49:45.209279 ip-10-0-136-210 kubenswrapper[2576]: E1030 21:49:45.208983    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vmrkz" podUID="52eb01f8-e86a-4458-8875-639394a87b30"
Oct 30 21:49:45.209279 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:45.209011    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jmbng"
Oct 30 21:49:45.209279 ip-10-0-136-210 kubenswrapper[2576]: E1030 21:49:45.209055    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jmbng" podUID="73c208eb-ffc0-46f2-8bd4-05ade5adb529"
Oct 30 21:49:46.209391 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:46.209365    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-68jrf"
Oct 30 21:49:46.209775 ip-10-0-136-210 kubenswrapper[2576]: E1030 21:49:46.209460    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-68jrf" podUID="e810bcd1-78da-409f-b86e-08c47031939e"
Oct 30 21:49:47.209340 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:47.209297    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vmrkz"
Oct 30 21:49:47.209340 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:47.209332    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jmbng"
Oct 30 21:49:47.209828 ip-10-0-136-210 kubenswrapper[2576]: E1030 21:49:47.209430    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vmrkz" podUID="52eb01f8-e86a-4458-8875-639394a87b30"
Oct 30 21:49:47.209828 ip-10-0-136-210 kubenswrapper[2576]: E1030 21:49:47.209506    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jmbng" podUID="73c208eb-ffc0-46f2-8bd4-05ade5adb529"
Oct 30 21:49:48.209972 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:48.209942    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-68jrf"
Oct 30 21:49:48.210442 ip-10-0-136-210 kubenswrapper[2576]: E1030 21:49:48.210062    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-68jrf" podUID="e810bcd1-78da-409f-b86e-08c47031939e"
Oct 30 21:49:48.382220 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:48.382184    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:48.673368 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:48.673332    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:48.863092 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:48.863059    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:49.208881 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:49.208838    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vmrkz"
Oct 30 21:49:49.209066 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:49.208856    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jmbng"
Oct 30 21:49:49.209066 ip-10-0-136-210 kubenswrapper[2576]: E1030 21:49:49.208969    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vmrkz" podUID="52eb01f8-e86a-4458-8875-639394a87b30"
Oct 30 21:49:49.209066 ip-10-0-136-210 kubenswrapper[2576]: E1030 21:49:49.209010    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jmbng" podUID="73c208eb-ffc0-46f2-8bd4-05ade5adb529"
Oct 30 21:49:49.982749 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:49.982718    2576 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-68jrf"]
Oct 30 21:49:49.983215 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:49.982839    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-68jrf"
Oct 30 21:49:49.983215 ip-10-0-136-210 kubenswrapper[2576]: E1030 21:49:49.982930    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-68jrf" podUID="e810bcd1-78da-409f-b86e-08c47031939e"
Oct 30 21:49:49.985697 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:49.985649    2576 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-jmbng"]
Oct 30 21:49:49.985824 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:49.985780    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jmbng"
Oct 30 21:49:49.985889 ip-10-0-136-210 kubenswrapper[2576]: E1030 21:49:49.985869    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jmbng" podUID="73c208eb-ffc0-46f2-8bd4-05ade5adb529"
Oct 30 21:49:49.986165 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:49.986144    2576 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-vmrkz"]
Oct 30 21:49:49.986238 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:49.986225    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vmrkz"
Oct 30 21:49:49.986327 ip-10-0-136-210 kubenswrapper[2576]: E1030 21:49:49.986310    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vmrkz" podUID="52eb01f8-e86a-4458-8875-639394a87b30"
Oct 30 21:49:50.072275 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:50.072241    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:50.267098 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:50.267068    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:51.209108 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:51.208924    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jmbng"
Oct 30 21:49:51.209509 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:51.208924    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-68jrf"
Oct 30 21:49:51.209509 ip-10-0-136-210 kubenswrapper[2576]: E1030 21:49:51.209194    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jmbng" podUID="73c208eb-ffc0-46f2-8bd4-05ade5adb529"
Oct 30 21:49:51.209509 ip-10-0-136-210 kubenswrapper[2576]: E1030 21:49:51.209242    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-68jrf" podUID="e810bcd1-78da-409f-b86e-08c47031939e"
Oct 30 21:49:52.211245 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:52.211218    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vmrkz"
Oct 30 21:49:52.211615 ip-10-0-136-210 kubenswrapper[2576]: E1030 21:49:52.211322    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vmrkz" podUID="52eb01f8-e86a-4458-8875-639394a87b30"
Oct 30 21:49:52.669498 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:52.669453    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:52.900946 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:52.900913    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:53.109538 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:53.109496    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:53.208674 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:53.208622    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-68jrf"
Oct 30 21:49:53.208859 ip-10-0-136-210 kubenswrapper[2576]: E1030 21:49:53.208739    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-68jrf" podUID="e810bcd1-78da-409f-b86e-08c47031939e"
Oct 30 21:49:53.208859 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:53.208800    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jmbng"
Oct 30 21:49:53.208983 ip-10-0-136-210 kubenswrapper[2576]: E1030 21:49:53.208894    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jmbng" podUID="73c208eb-ffc0-46f2-8bd4-05ade5adb529"
Oct 30 21:49:53.778788 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:53.778758    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:54.138088 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:54.138008    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:54.209205 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:54.209178    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vmrkz"
Oct 30 21:49:54.209338 ip-10-0-136-210 kubenswrapper[2576]: E1030 21:49:54.209310    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vmrkz" podUID="52eb01f8-e86a-4458-8875-639394a87b30"
Oct 30 21:49:54.377225 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:54.377196    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:54.588440 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:54.588412    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:54.783923 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:54.783890    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:54.827519 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:54.827453    2576 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-210.us-west-2.compute.internal" event="NodeReady"
Oct 30 21:49:54.827638 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:54.827588    2576 kubelet_node_status.go:477] "Fast updating node status as it just became ready"
Oct 30 21:49:54.973527 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:54.973495    2576 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-t8tbp"]
Oct 30 21:49:54.975228 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:54.975212    2576 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-6gjbf"]
Oct 30 21:49:54.975348 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:54.975331    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-t8tbp"
Oct 30 21:49:54.977224 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:54.977206    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-6gjbf"
Oct 30 21:49:55.026631 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:55.026600    2576 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-w5ck5"]
Oct 30 21:49:55.028334 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:55.028318    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-w5ck5"
Oct 30 21:49:55.028694 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:55.028670    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Oct 30 21:49:55.028694 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:55.028680    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Oct 30 21:49:55.028928 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:55.028912    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-8lg5r\""
Oct 30 21:49:55.029012 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:55.028956    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Oct 30 21:49:55.029052 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:55.029009    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Oct 30 21:49:55.055314 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:55.055296    2576 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-t8tbp"]
Oct 30 21:49:55.058934 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:55.058901    2576 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-6gjbf"]
Oct 30 21:49:55.124989 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:55.124898    2576 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-w5ck5"]
Oct 30 21:49:55.129424 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:55.129393    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Oct 30 21:49:55.129424 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:55.129392    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Oct 30 21:49:55.129628 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:55.129427    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Oct 30 21:49:55.129628 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:55.129487    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Oct 30 21:49:55.129628 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:55.129498    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Oct 30 21:49:55.129628 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:55.129585    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-bbh9g\""
Oct 30 21:49:55.129836 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:55.129723    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-52nbt\""
Oct 30 21:49:55.136923 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:55.136904    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vh5d6\" (UniqueName: \"kubernetes.io/projected/d8dc5e17-81b4-413a-a48f-6175d0ed3ec0-kube-api-access-vh5d6\") pod \"ingress-canary-t8tbp\" (UID: \"d8dc5e17-81b4-413a-a48f-6175d0ed3ec0\") " pod="openshift-ingress-canary/ingress-canary-t8tbp"
Oct 30 21:49:55.136999 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:55.136930    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/7d6190d5-6574-4704-97cb-2241398548e3-crio-socket\") pod \"insights-runtime-extractor-6gjbf\" (UID: \"7d6190d5-6574-4704-97cb-2241398548e3\") " pod="openshift-insights/insights-runtime-extractor-6gjbf"
Oct 30 21:49:55.136999 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:55.136949    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7d6190d5-6574-4704-97cb-2241398548e3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6gjbf\" (UID: \"7d6190d5-6574-4704-97cb-2241398548e3\") " pod="openshift-insights/insights-runtime-extractor-6gjbf"
Oct 30 21:49:55.136999 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:55.136972    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2fe50918-ee11-4d17-b0fe-27c7898912c6-metrics-tls\") pod \"dns-default-w5ck5\" (UID: \"2fe50918-ee11-4d17-b0fe-27c7898912c6\") " pod="openshift-dns/dns-default-w5ck5"
Oct 30 21:49:55.137110 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:55.137093    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d8dc5e17-81b4-413a-a48f-6175d0ed3ec0-cert\") pod \"ingress-canary-t8tbp\" (UID: \"d8dc5e17-81b4-413a-a48f-6175d0ed3ec0\") " pod="openshift-ingress-canary/ingress-canary-t8tbp"
Oct 30 21:49:55.137150 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:55.137136    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7d6190d5-6574-4704-97cb-2241398548e3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6gjbf\" (UID: \"7d6190d5-6574-4704-97cb-2241398548e3\") " pod="openshift-insights/insights-runtime-extractor-6gjbf"
Oct 30 21:49:55.137180 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:55.137157    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-25ggc\" (UniqueName: \"kubernetes.io/projected/2fe50918-ee11-4d17-b0fe-27c7898912c6-kube-api-access-25ggc\") pod \"dns-default-w5ck5\" (UID: \"2fe50918-ee11-4d17-b0fe-27c7898912c6\") " pod="openshift-dns/dns-default-w5ck5"
Oct 30 21:49:55.137180 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:55.137175    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/7d6190d5-6574-4704-97cb-2241398548e3-data-volume\") pod \"insights-runtime-extractor-6gjbf\" (UID: \"7d6190d5-6574-4704-97cb-2241398548e3\") " pod="openshift-insights/insights-runtime-extractor-6gjbf"
Oct 30 21:49:55.137249 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:55.137233    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2fe50918-ee11-4d17-b0fe-27c7898912c6-tmp-dir\") pod \"dns-default-w5ck5\" (UID: \"2fe50918-ee11-4d17-b0fe-27c7898912c6\") " pod="openshift-dns/dns-default-w5ck5"
Oct 30 21:49:55.137297 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:55.137285    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ggf4d\" (UniqueName: \"kubernetes.io/projected/7d6190d5-6574-4704-97cb-2241398548e3-kube-api-access-ggf4d\") pod \"insights-runtime-extractor-6gjbf\" (UID: \"7d6190d5-6574-4704-97cb-2241398548e3\") " pod="openshift-insights/insights-runtime-extractor-6gjbf"
Oct 30 21:49:55.137329 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:55.137306    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/2fe50918-ee11-4d17-b0fe-27c7898912c6-config-volume\") pod \"dns-default-w5ck5\" (UID: \"2fe50918-ee11-4d17-b0fe-27c7898912c6\") " pod="openshift-dns/dns-default-w5ck5"
Oct 30 21:49:55.209032 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:55.209002    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-68jrf"
Oct 30 21:49:55.209187 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:55.209162    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jmbng"
Oct 30 21:49:55.237691 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:55.237651    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vh5d6\" (UniqueName: \"kubernetes.io/projected/d8dc5e17-81b4-413a-a48f-6175d0ed3ec0-kube-api-access-vh5d6\") pod \"ingress-canary-t8tbp\" (UID: \"d8dc5e17-81b4-413a-a48f-6175d0ed3ec0\") " pod="openshift-ingress-canary/ingress-canary-t8tbp"
Oct 30 21:49:55.237792 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:55.237705    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/7d6190d5-6574-4704-97cb-2241398548e3-crio-socket\") pod \"insights-runtime-extractor-6gjbf\" (UID: \"7d6190d5-6574-4704-97cb-2241398548e3\") " pod="openshift-insights/insights-runtime-extractor-6gjbf"
Oct 30 21:49:55.237792 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:55.237729    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7d6190d5-6574-4704-97cb-2241398548e3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6gjbf\" (UID: \"7d6190d5-6574-4704-97cb-2241398548e3\") " pod="openshift-insights/insights-runtime-extractor-6gjbf"
Oct 30 21:49:55.237792 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:55.237753    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2fe50918-ee11-4d17-b0fe-27c7898912c6-metrics-tls\") pod \"dns-default-w5ck5\" (UID: \"2fe50918-ee11-4d17-b0fe-27c7898912c6\") " pod="openshift-dns/dns-default-w5ck5"
Oct 30 21:49:55.237943 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:55.237925    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d8dc5e17-81b4-413a-a48f-6175d0ed3ec0-cert\") pod \"ingress-canary-t8tbp\" (UID: \"d8dc5e17-81b4-413a-a48f-6175d0ed3ec0\") " pod="openshift-ingress-canary/ingress-canary-t8tbp"
Oct 30 21:49:55.237998 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:55.237944    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/7d6190d5-6574-4704-97cb-2241398548e3-crio-socket\") pod \"insights-runtime-extractor-6gjbf\" (UID: \"7d6190d5-6574-4704-97cb-2241398548e3\") " pod="openshift-insights/insights-runtime-extractor-6gjbf"
Oct 30 21:49:55.238050 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:55.238000    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7d6190d5-6574-4704-97cb-2241398548e3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6gjbf\" (UID: \"7d6190d5-6574-4704-97cb-2241398548e3\") " pod="openshift-insights/insights-runtime-extractor-6gjbf"
Oct 30 21:49:55.238050 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:55.238036    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-25ggc\" (UniqueName: \"kubernetes.io/projected/2fe50918-ee11-4d17-b0fe-27c7898912c6-kube-api-access-25ggc\") pod \"dns-default-w5ck5\" (UID: \"2fe50918-ee11-4d17-b0fe-27c7898912c6\") " pod="openshift-dns/dns-default-w5ck5"
Oct 30 21:49:55.238150 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:55.238064    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/7d6190d5-6574-4704-97cb-2241398548e3-data-volume\") pod \"insights-runtime-extractor-6gjbf\" (UID: \"7d6190d5-6574-4704-97cb-2241398548e3\") " pod="openshift-insights/insights-runtime-extractor-6gjbf"
Oct 30 21:49:55.238150 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:55.238091    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2fe50918-ee11-4d17-b0fe-27c7898912c6-tmp-dir\") pod \"dns-default-w5ck5\" (UID: \"2fe50918-ee11-4d17-b0fe-27c7898912c6\") " pod="openshift-dns/dns-default-w5ck5"
Oct 30 21:49:55.238243 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:55.238151    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ggf4d\" (UniqueName: \"kubernetes.io/projected/7d6190d5-6574-4704-97cb-2241398548e3-kube-api-access-ggf4d\") pod \"insights-runtime-extractor-6gjbf\" (UID: \"7d6190d5-6574-4704-97cb-2241398548e3\") " pod="openshift-insights/insights-runtime-extractor-6gjbf"
Oct 30 21:49:55.238243 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:55.238180    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/2fe50918-ee11-4d17-b0fe-27c7898912c6-config-volume\") pod \"dns-default-w5ck5\" (UID: \"2fe50918-ee11-4d17-b0fe-27c7898912c6\") " pod="openshift-dns/dns-default-w5ck5"
Oct 30 21:49:55.238347 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:55.238259    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7d6190d5-6574-4704-97cb-2241398548e3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6gjbf\" (UID: \"7d6190d5-6574-4704-97cb-2241398548e3\") " pod="openshift-insights/insights-runtime-extractor-6gjbf"
Oct 30 21:49:55.238430 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:55.238408    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/7d6190d5-6574-4704-97cb-2241398548e3-data-volume\") pod \"insights-runtime-extractor-6gjbf\" (UID: \"7d6190d5-6574-4704-97cb-2241398548e3\") " pod="openshift-insights/insights-runtime-extractor-6gjbf"
Oct 30 21:49:55.238487 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:55.238433    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2fe50918-ee11-4d17-b0fe-27c7898912c6-tmp-dir\") pod \"dns-default-w5ck5\" (UID: \"2fe50918-ee11-4d17-b0fe-27c7898912c6\") " pod="openshift-dns/dns-default-w5ck5"
Oct 30 21:49:55.239048 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:55.239020    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/2fe50918-ee11-4d17-b0fe-27c7898912c6-config-volume\") pod \"dns-default-w5ck5\" (UID: \"2fe50918-ee11-4d17-b0fe-27c7898912c6\") " pod="openshift-dns/dns-default-w5ck5"
Oct 30 21:49:55.242323 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:55.242295    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7d6190d5-6574-4704-97cb-2241398548e3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6gjbf\" (UID: \"7d6190d5-6574-4704-97cb-2241398548e3\") " pod="openshift-insights/insights-runtime-extractor-6gjbf"
Oct 30 21:49:55.242408 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:55.242296    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2fe50918-ee11-4d17-b0fe-27c7898912c6-metrics-tls\") pod \"dns-default-w5ck5\" (UID: \"2fe50918-ee11-4d17-b0fe-27c7898912c6\") " pod="openshift-dns/dns-default-w5ck5"
Oct 30 21:49:55.243046 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:55.243026    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d8dc5e17-81b4-413a-a48f-6175d0ed3ec0-cert\") pod \"ingress-canary-t8tbp\" (UID: \"d8dc5e17-81b4-413a-a48f-6175d0ed3ec0\") " pod="openshift-ingress-canary/ingress-canary-t8tbp"
Oct 30 21:49:55.263240 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:55.263206    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:55.295862 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:55.295826    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vh5d6\" (UniqueName: \"kubernetes.io/projected/d8dc5e17-81b4-413a-a48f-6175d0ed3ec0-kube-api-access-vh5d6\") pod \"ingress-canary-t8tbp\" (UID: \"d8dc5e17-81b4-413a-a48f-6175d0ed3ec0\") " pod="openshift-ingress-canary/ingress-canary-t8tbp"
Oct 30 21:49:55.295985 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:55.295828    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ggf4d\" (UniqueName: \"kubernetes.io/projected/7d6190d5-6574-4704-97cb-2241398548e3-kube-api-access-ggf4d\") pod \"insights-runtime-extractor-6gjbf\" (UID: \"7d6190d5-6574-4704-97cb-2241398548e3\") " pod="openshift-insights/insights-runtime-extractor-6gjbf"
Oct 30 21:49:55.296277 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:55.296260    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-25ggc\" (UniqueName: \"kubernetes.io/projected/2fe50918-ee11-4d17-b0fe-27c7898912c6-kube-api-access-25ggc\") pod \"dns-default-w5ck5\" (UID: \"2fe50918-ee11-4d17-b0fe-27c7898912c6\") " pod="openshift-dns/dns-default-w5ck5"
Oct 30 21:49:55.334070 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:55.334043    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Oct 30 21:49:55.334384 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:55.334356    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Oct 30 21:49:55.337509 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:55.337493    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-jbflf\""
Oct 30 21:49:55.337619 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:55.337514    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Oct 30 21:49:55.337619 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:55.337550    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-9mgc8\""
Oct 30 21:49:55.338608 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:55.338593    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-w5ck5"
Oct 30 21:49:55.506582 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:55.506548    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:55.588031 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:55.587980    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-t8tbp"
Oct 30 21:49:55.594139 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:55.594031    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-6gjbf"
Oct 30 21:49:55.714184 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:55.714151    2576 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-w5ck5"]
Oct 30 21:49:55.718410 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:55.718384    2576 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2fe50918_ee11_4d17_b0fe_27c7898912c6.slice/crio-4d6cb4d22f1a3341a2a3e4f90510ae7fbec361fb13b62997a07cb2f4f9fe864b WatchSource:0}: Error finding container 4d6cb4d22f1a3341a2a3e4f90510ae7fbec361fb13b62997a07cb2f4f9fe864b: Status 404 returned error can't find the container with id 4d6cb4d22f1a3341a2a3e4f90510ae7fbec361fb13b62997a07cb2f4f9fe864b
Oct 30 21:49:55.952980 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:55.952900    2576 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-t8tbp"]
Oct 30 21:49:55.956034 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:55.956003    2576 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd8dc5e17_81b4_413a_a48f_6175d0ed3ec0.slice/crio-ff49e81b45715f352095c16b17b12a8c2cc3ccc92322711b2615d651155d2dbb WatchSource:0}: Error finding container ff49e81b45715f352095c16b17b12a8c2cc3ccc92322711b2615d651155d2dbb: Status 404 returned error can't find the container with id ff49e81b45715f352095c16b17b12a8c2cc3ccc92322711b2615d651155d2dbb
Oct 30 21:49:55.998363 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:55.998338    2576 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-6gjbf"]
Oct 30 21:49:56.001769 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:49:56.001744    2576 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d6190d5_6574_4704_97cb_2241398548e3.slice/crio-fa5a20193e2a7edb9a384afb717ec20cb79c193302573d94ad22f8eecfbb3a16 WatchSource:0}: Error finding container fa5a20193e2a7edb9a384afb717ec20cb79c193302573d94ad22f8eecfbb3a16: Status 404 returned error can't find the container with id fa5a20193e2a7edb9a384afb717ec20cb79c193302573d94ad22f8eecfbb3a16
Oct 30 21:49:56.209281 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:56.209185    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vmrkz"
Oct 30 21:49:56.260341 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:56.260309    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Oct 30 21:49:56.260468 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:56.260350    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-9bjlv\""
Oct 30 21:49:56.434455 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:56.434423    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-6gjbf" event={"ID":"7d6190d5-6574-4704-97cb-2241398548e3","Type":"ContainerStarted","Data":"10109aa369063778d8e784858f9bda79a605cd9193b8847d12efc30453a821c7"}
Oct 30 21:49:56.434626 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:56.434481    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-6gjbf" event={"ID":"7d6190d5-6574-4704-97cb-2241398548e3","Type":"ContainerStarted","Data":"fa5a20193e2a7edb9a384afb717ec20cb79c193302573d94ad22f8eecfbb3a16"}
Oct 30 21:49:56.435498 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:56.435477    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-t8tbp" event={"ID":"d8dc5e17-81b4-413a-a48f-6175d0ed3ec0","Type":"ContainerStarted","Data":"ff49e81b45715f352095c16b17b12a8c2cc3ccc92322711b2615d651155d2dbb"}
Oct 30 21:49:56.436539 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:56.436522    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-w5ck5" event={"ID":"2fe50918-ee11-4d17-b0fe-27c7898912c6","Type":"ContainerStarted","Data":"4d6cb4d22f1a3341a2a3e4f90510ae7fbec361fb13b62997a07cb2f4f9fe864b"}
Oct 30 21:49:57.223471 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:57.223434    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:57.441219 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:57.441173    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-6gjbf" event={"ID":"7d6190d5-6574-4704-97cb-2241398548e3","Type":"ContainerStarted","Data":"f62f8e062eac65b1fc0e0db22d6c785038836139da98aa97f9edce0f7d7fd91d"}
Oct 30 21:49:58.059566 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:58.059537    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:58.317579 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:58.317542    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:58.446563 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:58.446530    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-w5ck5" event={"ID":"2fe50918-ee11-4d17-b0fe-27c7898912c6","Type":"ContainerStarted","Data":"1a2180577c88a0bf51f0bab1f9008dc209ea328b5a08046a0edb181b59565ca9"}
Oct 30 21:49:58.446767 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:58.446621    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-w5ck5" event={"ID":"2fe50918-ee11-4d17-b0fe-27c7898912c6","Type":"ContainerStarted","Data":"c0bce7c588220ad09d5716f5d90f130e1b9a1086e9817c33fb5bb8338556eae1"}
Oct 30 21:49:58.446767 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:58.446677    2576 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-w5ck5"
Oct 30 21:49:58.535145 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:58.535109    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:58.559472 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:58.559412    2576 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-dns/dns-default-w5ck5" podStartSLOduration=2.235861284 podStartE2EDuration="4.559393661s" totalImagesPullingTime="2.323532377s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:49:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:49:58.558746249 +0000 UTC m=+50.990694516" watchObservedRunningTime="2026-10-30 21:49:58.559393661 +0000 UTC m=+50.991341930"
Oct 30 21:49:59.669092 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:59.669008    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:59.902439 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:49:59.902412    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:00.103512 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:00.103478    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:00.452730 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:00.452617    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-6gjbf" event={"ID":"7d6190d5-6574-4704-97cb-2241398548e3","Type":"ContainerStarted","Data":"2817e382645d4b622268ec7026688a4d795eef2d28a93575c5a184137bd60a45"}
Oct 30 21:50:00.454006 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:00.453979    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-t8tbp" event={"ID":"d8dc5e17-81b4-413a-a48f-6175d0ed3ec0","Type":"ContainerStarted","Data":"ac88dda3373f8b2d6f6681e9c027232fc868842a202a375a97eb6b240b4dcaf8"}
Oct 30 21:50:00.676452 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:00.676399    2576 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-6gjbf" podStartSLOduration=3.427478437 podStartE2EDuration="6.676385455s" totalImagesPullingTime="3.248907018s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 21:49:54 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:50:00.629596748 +0000 UTC m=+53.061545017" watchObservedRunningTime="2026-10-30 21:50:00.676385455 +0000 UTC m=+53.108333701"
Oct 30 21:50:00.739464 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:00.739356    2576 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-t8tbp" podStartSLOduration=3.293543234 podStartE2EDuration="6.739341307s" totalImagesPullingTime="3.445798073s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:49:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:50:00.739109301 +0000 UTC m=+53.171057572" watchObservedRunningTime="2026-10-30 21:50:00.739341307 +0000 UTC m=+53.171289574"
Oct 30 21:50:00.774775 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:00.774738    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:01.012158 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:01.012077    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:01.638492 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:01.638459    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:01.825430 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:01.825394    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:02.045966 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:02.045933    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:03.395061 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:03.395036    2576 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-wrxww"
Oct 30 21:50:03.744145 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:03.744072    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:04.312583 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:04.312550    2576 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-6d5866d6cd-b6dpl"]
Oct 30 21:50:04.315546 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:04.315531    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6d5866d6cd-b6dpl"
Oct 30 21:50:04.366131 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:04.366099    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:04.371978 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:04.371957    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Oct 30 21:50:04.372088 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:04.372001    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Oct 30 21:50:04.372088 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:04.372039    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-jlg2v\""
Oct 30 21:50:04.403644 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:04.403606    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-tmp\") pod \"console-6d5866d6cd-b6dpl\" (UID: \"42d54b8a-4f48-46a7-8a34-1f0f29cd1602\") " pod="openshift-console/console-6d5866d6cd-b6dpl"
Oct 30 21:50:04.404053 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:04.403688    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-oauth-serving-cert\") pod \"console-6d5866d6cd-b6dpl\" (UID: \"42d54b8a-4f48-46a7-8a34-1f0f29cd1602\") " pod="openshift-console/console-6d5866d6cd-b6dpl"
Oct 30 21:50:04.404053 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:04.403736    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5c4fj\" (UniqueName: \"kubernetes.io/projected/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-kube-api-access-5c4fj\") pod \"console-6d5866d6cd-b6dpl\" (UID: \"42d54b8a-4f48-46a7-8a34-1f0f29cd1602\") " pod="openshift-console/console-6d5866d6cd-b6dpl"
Oct 30 21:50:04.404053 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:04.403758    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-console-oauth-config\") pod \"console-6d5866d6cd-b6dpl\" (UID: \"42d54b8a-4f48-46a7-8a34-1f0f29cd1602\") " pod="openshift-console/console-6d5866d6cd-b6dpl"
Oct 30 21:50:04.404053 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:04.403848    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-console-config\") pod \"console-6d5866d6cd-b6dpl\" (UID: \"42d54b8a-4f48-46a7-8a34-1f0f29cd1602\") " pod="openshift-console/console-6d5866d6cd-b6dpl"
Oct 30 21:50:04.404053 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:04.403893    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-trusted-ca-bundle\") pod \"console-6d5866d6cd-b6dpl\" (UID: \"42d54b8a-4f48-46a7-8a34-1f0f29cd1602\") " pod="openshift-console/console-6d5866d6cd-b6dpl"
Oct 30 21:50:04.404053 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:04.403930    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-console-serving-cert\") pod \"console-6d5866d6cd-b6dpl\" (UID: \"42d54b8a-4f48-46a7-8a34-1f0f29cd1602\") " pod="openshift-console/console-6d5866d6cd-b6dpl"
Oct 30 21:50:04.404053 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:04.403990    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-service-ca\") pod \"console-6d5866d6cd-b6dpl\" (UID: \"42d54b8a-4f48-46a7-8a34-1f0f29cd1602\") " pod="openshift-console/console-6d5866d6cd-b6dpl"
Oct 30 21:50:04.412322 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:04.412297    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Oct 30 21:50:04.412322 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:04.412300    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Oct 30 21:50:04.412452 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:04.412340    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Oct 30 21:50:04.412452 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:04.412348    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Oct 30 21:50:04.412452 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:04.412378    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Oct 30 21:50:04.504322 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:04.504289    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-console-oauth-config\") pod \"console-6d5866d6cd-b6dpl\" (UID: \"42d54b8a-4f48-46a7-8a34-1f0f29cd1602\") " pod="openshift-console/console-6d5866d6cd-b6dpl"
Oct 30 21:50:04.504322 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:04.504320    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-console-config\") pod \"console-6d5866d6cd-b6dpl\" (UID: \"42d54b8a-4f48-46a7-8a34-1f0f29cd1602\") " pod="openshift-console/console-6d5866d6cd-b6dpl"
Oct 30 21:50:04.504535 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:04.504341    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-trusted-ca-bundle\") pod \"console-6d5866d6cd-b6dpl\" (UID: \"42d54b8a-4f48-46a7-8a34-1f0f29cd1602\") " pod="openshift-console/console-6d5866d6cd-b6dpl"
Oct 30 21:50:04.504535 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:04.504366    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-console-serving-cert\") pod \"console-6d5866d6cd-b6dpl\" (UID: \"42d54b8a-4f48-46a7-8a34-1f0f29cd1602\") " pod="openshift-console/console-6d5866d6cd-b6dpl"
Oct 30 21:50:04.504535 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:04.504430    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-service-ca\") pod \"console-6d5866d6cd-b6dpl\" (UID: \"42d54b8a-4f48-46a7-8a34-1f0f29cd1602\") " pod="openshift-console/console-6d5866d6cd-b6dpl"
Oct 30 21:50:04.504535 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:04.504461    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-tmp\") pod \"console-6d5866d6cd-b6dpl\" (UID: \"42d54b8a-4f48-46a7-8a34-1f0f29cd1602\") " pod="openshift-console/console-6d5866d6cd-b6dpl"
Oct 30 21:50:04.504535 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:04.504504    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-oauth-serving-cert\") pod \"console-6d5866d6cd-b6dpl\" (UID: \"42d54b8a-4f48-46a7-8a34-1f0f29cd1602\") " pod="openshift-console/console-6d5866d6cd-b6dpl"
Oct 30 21:50:04.504535 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:04.504530    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5c4fj\" (UniqueName: \"kubernetes.io/projected/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-kube-api-access-5c4fj\") pod \"console-6d5866d6cd-b6dpl\" (UID: \"42d54b8a-4f48-46a7-8a34-1f0f29cd1602\") " pod="openshift-console/console-6d5866d6cd-b6dpl"
Oct 30 21:50:04.505055 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:04.505025    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-tmp\") pod \"console-6d5866d6cd-b6dpl\" (UID: \"42d54b8a-4f48-46a7-8a34-1f0f29cd1602\") " pod="openshift-console/console-6d5866d6cd-b6dpl"
Oct 30 21:50:04.505261 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:04.505239    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-service-ca\") pod \"console-6d5866d6cd-b6dpl\" (UID: \"42d54b8a-4f48-46a7-8a34-1f0f29cd1602\") " pod="openshift-console/console-6d5866d6cd-b6dpl"
Oct 30 21:50:04.505341 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:04.505256    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-console-config\") pod \"console-6d5866d6cd-b6dpl\" (UID: \"42d54b8a-4f48-46a7-8a34-1f0f29cd1602\") " pod="openshift-console/console-6d5866d6cd-b6dpl"
Oct 30 21:50:04.505341 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:04.505241    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-oauth-serving-cert\") pod \"console-6d5866d6cd-b6dpl\" (UID: \"42d54b8a-4f48-46a7-8a34-1f0f29cd1602\") " pod="openshift-console/console-6d5866d6cd-b6dpl"
Oct 30 21:50:04.506501 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:04.506473    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Oct 30 21:50:04.507050 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:04.507030    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-console-oauth-config\") pod \"console-6d5866d6cd-b6dpl\" (UID: \"42d54b8a-4f48-46a7-8a34-1f0f29cd1602\") " pod="openshift-console/console-6d5866d6cd-b6dpl"
Oct 30 21:50:04.507139 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:04.507034    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-console-serving-cert\") pod \"console-6d5866d6cd-b6dpl\" (UID: \"42d54b8a-4f48-46a7-8a34-1f0f29cd1602\") " pod="openshift-console/console-6d5866d6cd-b6dpl"
Oct 30 21:50:04.507448 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:04.507432    2576 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6d5866d6cd-b6dpl"]
Oct 30 21:50:04.515638 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:04.515619    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-trusted-ca-bundle\") pod \"console-6d5866d6cd-b6dpl\" (UID: \"42d54b8a-4f48-46a7-8a34-1f0f29cd1602\") " pod="openshift-console/console-6d5866d6cd-b6dpl"
Oct 30 21:50:04.563376 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:04.563317    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5c4fj\" (UniqueName: \"kubernetes.io/projected/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-kube-api-access-5c4fj\") pod \"console-6d5866d6cd-b6dpl\" (UID: \"42d54b8a-4f48-46a7-8a34-1f0f29cd1602\") " pod="openshift-console/console-6d5866d6cd-b6dpl"
Oct 30 21:50:04.588383 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:04.588354    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:04.626481 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:04.626452    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6d5866d6cd-b6dpl"
Oct 30 21:50:04.865911 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:04.865836    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:04.992839 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:04.992806    2576 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6d5866d6cd-b6dpl"]
Oct 30 21:50:05.019817 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:50:05.019789    2576 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42d54b8a_4f48_46a7_8a34_1f0f29cd1602.slice/crio-fbffb78323609cf8dd21b10efbe35374ed8184fc5b3e616fe9f4b87fd69b0936 WatchSource:0}: Error finding container fbffb78323609cf8dd21b10efbe35374ed8184fc5b3e616fe9f4b87fd69b0936: Status 404 returned error can't find the container with id fbffb78323609cf8dd21b10efbe35374ed8184fc5b3e616fe9f4b87fd69b0936
Oct 30 21:50:05.023021 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:05.023002    2576 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-7bd7489957-h95jn"]
Oct 30 21:50:05.027237 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:05.027218    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-h95jn"
Oct 30 21:50:05.056988 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:05.056945    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:05.076805 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:05.076786    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Oct 30 21:50:05.076911 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:05.076819    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-brb56\""
Oct 30 21:50:05.076911 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:05.076834    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Oct 30 21:50:05.076911 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:05.076843    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Oct 30 21:50:05.076911 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:05.076836    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Oct 30 21:50:05.076911 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:05.076819    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Oct 30 21:50:05.087956 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:05.087938    2576 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-7bd7489957-h95jn"]
Oct 30 21:50:05.107614 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:05.107589    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3abd55eb-d1d8-4fdf-b82c-d35894b652ac-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-h95jn\" (UID: \"3abd55eb-d1d8-4fdf-b82c-d35894b652ac\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-h95jn"
Oct 30 21:50:05.107716 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:05.107626    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/3abd55eb-d1d8-4fdf-b82c-d35894b652ac-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-h95jn\" (UID: \"3abd55eb-d1d8-4fdf-b82c-d35894b652ac\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-h95jn"
Oct 30 21:50:05.107716 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:05.107678    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3abd55eb-d1d8-4fdf-b82c-d35894b652ac-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-h95jn\" (UID: \"3abd55eb-d1d8-4fdf-b82c-d35894b652ac\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-h95jn"
Oct 30 21:50:05.107788 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:05.107743    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f2qbm\" (UniqueName: \"kubernetes.io/projected/3abd55eb-d1d8-4fdf-b82c-d35894b652ac-kube-api-access-f2qbm\") pod \"openshift-state-metrics-7bd7489957-h95jn\" (UID: \"3abd55eb-d1d8-4fdf-b82c-d35894b652ac\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-h95jn"
Oct 30 21:50:05.160375 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:05.160292    2576 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-mpsnt"]
Oct 30 21:50:05.163446 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:05.163424    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-mpsnt"
Oct 30 21:50:05.209749 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:05.208591    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/d414339d-1a64-4a2a-ad15-805b71ebfd1b-node-exporter-textfile\") pod \"node-exporter-mpsnt\" (UID: \"d414339d-1a64-4a2a-ad15-805b71ebfd1b\") " pod="openshift-monitoring/node-exporter-mpsnt"
Oct 30 21:50:05.209749 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:05.208648    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/d414339d-1a64-4a2a-ad15-805b71ebfd1b-root\") pod \"node-exporter-mpsnt\" (UID: \"d414339d-1a64-4a2a-ad15-805b71ebfd1b\") " pod="openshift-monitoring/node-exporter-mpsnt"
Oct 30 21:50:05.209749 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:05.208720    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d414339d-1a64-4a2a-ad15-805b71ebfd1b-sys\") pod \"node-exporter-mpsnt\" (UID: \"d414339d-1a64-4a2a-ad15-805b71ebfd1b\") " pod="openshift-monitoring/node-exporter-mpsnt"
Oct 30 21:50:05.209749 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:05.208749    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rq9p9\" (UniqueName: \"kubernetes.io/projected/d414339d-1a64-4a2a-ad15-805b71ebfd1b-kube-api-access-rq9p9\") pod \"node-exporter-mpsnt\" (UID: \"d414339d-1a64-4a2a-ad15-805b71ebfd1b\") " pod="openshift-monitoring/node-exporter-mpsnt"
Oct 30 21:50:05.209749 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:05.208794    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/d414339d-1a64-4a2a-ad15-805b71ebfd1b-node-exporter-wtmp\") pod \"node-exporter-mpsnt\" (UID: \"d414339d-1a64-4a2a-ad15-805b71ebfd1b\") " pod="openshift-monitoring/node-exporter-mpsnt"
Oct 30 21:50:05.209749 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:05.208839    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d414339d-1a64-4a2a-ad15-805b71ebfd1b-node-exporter-tls\") pod \"node-exporter-mpsnt\" (UID: \"d414339d-1a64-4a2a-ad15-805b71ebfd1b\") " pod="openshift-monitoring/node-exporter-mpsnt"
Oct 30 21:50:05.209749 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:05.208865    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/d414339d-1a64-4a2a-ad15-805b71ebfd1b-node-exporter-accelerators-collector-config\") pod \"node-exporter-mpsnt\" (UID: \"d414339d-1a64-4a2a-ad15-805b71ebfd1b\") " pod="openshift-monitoring/node-exporter-mpsnt"
Oct 30 21:50:05.209749 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:05.208904    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/3abd55eb-d1d8-4fdf-b82c-d35894b652ac-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-h95jn\" (UID: \"3abd55eb-d1d8-4fdf-b82c-d35894b652ac\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-h95jn"
Oct 30 21:50:05.209749 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:05.208951    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3abd55eb-d1d8-4fdf-b82c-d35894b652ac-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-h95jn\" (UID: \"3abd55eb-d1d8-4fdf-b82c-d35894b652ac\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-h95jn"
Oct 30 21:50:05.209749 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:05.208986    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f2qbm\" (UniqueName: \"kubernetes.io/projected/3abd55eb-d1d8-4fdf-b82c-d35894b652ac-kube-api-access-f2qbm\") pod \"openshift-state-metrics-7bd7489957-h95jn\" (UID: \"3abd55eb-d1d8-4fdf-b82c-d35894b652ac\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-h95jn"
Oct 30 21:50:05.209749 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:05.209027    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d414339d-1a64-4a2a-ad15-805b71ebfd1b-metrics-client-ca\") pod \"node-exporter-mpsnt\" (UID: \"d414339d-1a64-4a2a-ad15-805b71ebfd1b\") " pod="openshift-monitoring/node-exporter-mpsnt"
Oct 30 21:50:05.209749 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:05.209085    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3abd55eb-d1d8-4fdf-b82c-d35894b652ac-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-h95jn\" (UID: \"3abd55eb-d1d8-4fdf-b82c-d35894b652ac\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-h95jn"
Oct 30 21:50:05.209749 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:05.209118    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d414339d-1a64-4a2a-ad15-805b71ebfd1b-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-mpsnt\" (UID: \"d414339d-1a64-4a2a-ad15-805b71ebfd1b\") " pod="openshift-monitoring/node-exporter-mpsnt"
Oct 30 21:50:05.211667 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:05.211625    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3abd55eb-d1d8-4fdf-b82c-d35894b652ac-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-h95jn\" (UID: \"3abd55eb-d1d8-4fdf-b82c-d35894b652ac\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-h95jn"
Oct 30 21:50:05.212825 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:05.212804    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/3abd55eb-d1d8-4fdf-b82c-d35894b652ac-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-h95jn\" (UID: \"3abd55eb-d1d8-4fdf-b82c-d35894b652ac\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-h95jn"
Oct 30 21:50:05.213765 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:05.213749    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3abd55eb-d1d8-4fdf-b82c-d35894b652ac-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-h95jn\" (UID: \"3abd55eb-d1d8-4fdf-b82c-d35894b652ac\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-h95jn"
Oct 30 21:50:05.222106 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:05.222080    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Oct 30 21:50:05.222930 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:05.222917    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Oct 30 21:50:05.223071 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:05.223056    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Oct 30 21:50:05.223106 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:05.223073    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-dhl5t\""
Oct 30 21:50:05.253485 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:05.253462    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:05.270721 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:05.270691    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f2qbm\" (UniqueName: \"kubernetes.io/projected/3abd55eb-d1d8-4fdf-b82c-d35894b652ac-kube-api-access-f2qbm\") pod \"openshift-state-metrics-7bd7489957-h95jn\" (UID: \"3abd55eb-d1d8-4fdf-b82c-d35894b652ac\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-h95jn"
Oct 30 21:50:05.310133 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:05.310105    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/d414339d-1a64-4a2a-ad15-805b71ebfd1b-root\") pod \"node-exporter-mpsnt\" (UID: \"d414339d-1a64-4a2a-ad15-805b71ebfd1b\") " pod="openshift-monitoring/node-exporter-mpsnt"
Oct 30 21:50:05.310227 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:05.310148    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d414339d-1a64-4a2a-ad15-805b71ebfd1b-sys\") pod \"node-exporter-mpsnt\" (UID: \"d414339d-1a64-4a2a-ad15-805b71ebfd1b\") " pod="openshift-monitoring/node-exporter-mpsnt"
Oct 30 21:50:05.310227 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:05.310164    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rq9p9\" (UniqueName: \"kubernetes.io/projected/d414339d-1a64-4a2a-ad15-805b71ebfd1b-kube-api-access-rq9p9\") pod \"node-exporter-mpsnt\" (UID: \"d414339d-1a64-4a2a-ad15-805b71ebfd1b\") " pod="openshift-monitoring/node-exporter-mpsnt"
Oct 30 21:50:05.310227 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:05.310189    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/d414339d-1a64-4a2a-ad15-805b71ebfd1b-node-exporter-wtmp\") pod \"node-exporter-mpsnt\" (UID: \"d414339d-1a64-4a2a-ad15-805b71ebfd1b\") " pod="openshift-monitoring/node-exporter-mpsnt"
Oct 30 21:50:05.310227 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:05.310220    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d414339d-1a64-4a2a-ad15-805b71ebfd1b-node-exporter-tls\") pod \"node-exporter-mpsnt\" (UID: \"d414339d-1a64-4a2a-ad15-805b71ebfd1b\") " pod="openshift-monitoring/node-exporter-mpsnt"
Oct 30 21:50:05.310361 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:05.310220    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/d414339d-1a64-4a2a-ad15-805b71ebfd1b-root\") pod \"node-exporter-mpsnt\" (UID: \"d414339d-1a64-4a2a-ad15-805b71ebfd1b\") " pod="openshift-monitoring/node-exporter-mpsnt"
Oct 30 21:50:05.310361 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:05.310234    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d414339d-1a64-4a2a-ad15-805b71ebfd1b-sys\") pod \"node-exporter-mpsnt\" (UID: \"d414339d-1a64-4a2a-ad15-805b71ebfd1b\") " pod="openshift-monitoring/node-exporter-mpsnt"
Oct 30 21:50:05.310361 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:05.310238    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/d414339d-1a64-4a2a-ad15-805b71ebfd1b-node-exporter-accelerators-collector-config\") pod \"node-exporter-mpsnt\" (UID: \"d414339d-1a64-4a2a-ad15-805b71ebfd1b\") " pod="openshift-monitoring/node-exporter-mpsnt"
Oct 30 21:50:05.310361 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:05.310313    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d414339d-1a64-4a2a-ad15-805b71ebfd1b-metrics-client-ca\") pod \"node-exporter-mpsnt\" (UID: \"d414339d-1a64-4a2a-ad15-805b71ebfd1b\") " pod="openshift-monitoring/node-exporter-mpsnt"
Oct 30 21:50:05.310498 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:05.310359    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d414339d-1a64-4a2a-ad15-805b71ebfd1b-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-mpsnt\" (UID: \"d414339d-1a64-4a2a-ad15-805b71ebfd1b\") " pod="openshift-monitoring/node-exporter-mpsnt"
Oct 30 21:50:05.310498 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:05.310380    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/d414339d-1a64-4a2a-ad15-805b71ebfd1b-node-exporter-wtmp\") pod \"node-exporter-mpsnt\" (UID: \"d414339d-1a64-4a2a-ad15-805b71ebfd1b\") " pod="openshift-monitoring/node-exporter-mpsnt"
Oct 30 21:50:05.310498 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:05.310395    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/d414339d-1a64-4a2a-ad15-805b71ebfd1b-node-exporter-textfile\") pod \"node-exporter-mpsnt\" (UID: \"d414339d-1a64-4a2a-ad15-805b71ebfd1b\") " pod="openshift-monitoring/node-exporter-mpsnt"
Oct 30 21:50:05.311343 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:05.311308    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/d414339d-1a64-4a2a-ad15-805b71ebfd1b-node-exporter-textfile\") pod \"node-exporter-mpsnt\" (UID: \"d414339d-1a64-4a2a-ad15-805b71ebfd1b\") " pod="openshift-monitoring/node-exporter-mpsnt"
Oct 30 21:50:05.311453 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:05.311433    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d414339d-1a64-4a2a-ad15-805b71ebfd1b-metrics-client-ca\") pod \"node-exporter-mpsnt\" (UID: \"d414339d-1a64-4a2a-ad15-805b71ebfd1b\") " pod="openshift-monitoring/node-exporter-mpsnt"
Oct 30 21:50:05.311520 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:05.311448    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/d414339d-1a64-4a2a-ad15-805b71ebfd1b-node-exporter-accelerators-collector-config\") pod \"node-exporter-mpsnt\" (UID: \"d414339d-1a64-4a2a-ad15-805b71ebfd1b\") " pod="openshift-monitoring/node-exporter-mpsnt"
Oct 30 21:50:05.312671 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:05.312632    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d414339d-1a64-4a2a-ad15-805b71ebfd1b-node-exporter-tls\") pod \"node-exporter-mpsnt\" (UID: \"d414339d-1a64-4a2a-ad15-805b71ebfd1b\") " pod="openshift-monitoring/node-exporter-mpsnt"
Oct 30 21:50:05.313085 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:05.313066    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d414339d-1a64-4a2a-ad15-805b71ebfd1b-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-mpsnt\" (UID: \"d414339d-1a64-4a2a-ad15-805b71ebfd1b\") " pod="openshift-monitoring/node-exporter-mpsnt"
Oct 30 21:50:05.342226 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:05.342191    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-h95jn"
Oct 30 21:50:05.368180 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:05.368153    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rq9p9\" (UniqueName: \"kubernetes.io/projected/d414339d-1a64-4a2a-ad15-805b71ebfd1b-kube-api-access-rq9p9\") pod \"node-exporter-mpsnt\" (UID: \"d414339d-1a64-4a2a-ad15-805b71ebfd1b\") " pod="openshift-monitoring/node-exporter-mpsnt"
Oct 30 21:50:05.467418 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:05.467336    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6d5866d6cd-b6dpl" event={"ID":"42d54b8a-4f48-46a7-8a34-1f0f29cd1602","Type":"ContainerStarted","Data":"fbffb78323609cf8dd21b10efbe35374ed8184fc5b3e616fe9f4b87fd69b0936"}
Oct 30 21:50:05.478535 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:05.478511    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-mpsnt"
Oct 30 21:50:05.489089 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:50:05.489067    2576 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd414339d_1a64_4a2a_ad15_805b71ebfd1b.slice/crio-f45d4e00313976fd643259c26716f35ec4f66325b137a79e47d646a0ddaf4cd0 WatchSource:0}: Error finding container f45d4e00313976fd643259c26716f35ec4f66325b137a79e47d646a0ddaf4cd0: Status 404 returned error can't find the container with id f45d4e00313976fd643259c26716f35ec4f66325b137a79e47d646a0ddaf4cd0
Oct 30 21:50:05.511705 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:05.511670    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:05.765320 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:05.765238    2576 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-7bd7489957-h95jn"]
Oct 30 21:50:05.768635 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:50:05.768595    2576 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3abd55eb_d1d8_4fdf_b82c_d35894b652ac.slice/crio-7c658481457a80544871f4902bf0c0cb431e33d2e3c54401546755d56b0716b4 WatchSource:0}: Error finding container 7c658481457a80544871f4902bf0c0cb431e33d2e3c54401546755d56b0716b4: Status 404 returned error can't find the container with id 7c658481457a80544871f4902bf0c0cb431e33d2e3c54401546755d56b0716b4
Oct 30 21:50:05.775356 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:05.775331    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:05.963605 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:05.963571    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:06.439693 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:06.439639    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:06.475615 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:06.475577    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mpsnt" event={"ID":"d414339d-1a64-4a2a-ad15-805b71ebfd1b","Type":"ContainerStarted","Data":"f45d4e00313976fd643259c26716f35ec4f66325b137a79e47d646a0ddaf4cd0"}
Oct 30 21:50:06.477755 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:06.477726    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-h95jn" event={"ID":"3abd55eb-d1d8-4fdf-b82c-d35894b652ac","Type":"ContainerStarted","Data":"b049de23c740760bddd8415ef07eaa88a5a21382d3e7753898103b41829e942a"}
Oct 30 21:50:06.477878 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:06.477782    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-h95jn" event={"ID":"3abd55eb-d1d8-4fdf-b82c-d35894b652ac","Type":"ContainerStarted","Data":"8de3fde6629ffeb65cb2a3a9440767e3f68e128320ef41b43e3e080fb189a499"}
Oct 30 21:50:06.477878 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:06.477802    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-h95jn" event={"ID":"3abd55eb-d1d8-4fdf-b82c-d35894b652ac","Type":"ContainerStarted","Data":"7c658481457a80544871f4902bf0c0cb431e33d2e3c54401546755d56b0716b4"}
Oct 30 21:50:06.776224 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:06.776189    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:06.903173 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:06.903142    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-xkrx5_04465eda-e6db-42d3-8d5f-c9be668038a9/node-ca/0.log"
Oct 30 21:50:07.376154 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:07.376080    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:07.470296 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:07.470262    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-t8tbp_d8dc5e17-81b4-413a-a48f-6175d0ed3ec0/serve-healthcheck-canary/0.log"
Oct 30 21:50:07.483115 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:07.483087    2576 generic.go:356] "Generic (PLEG): container finished" podID="d414339d-1a64-4a2a-ad15-805b71ebfd1b" containerID="c9e285756bcb8985bd79e7662e7dbea3151694964916f6717ec31d02a33a4b43" exitCode=0
Oct 30 21:50:07.483539 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:07.483121    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mpsnt" event={"ID":"d414339d-1a64-4a2a-ad15-805b71ebfd1b","Type":"ContainerDied","Data":"c9e285756bcb8985bd79e7662e7dbea3151694964916f6717ec31d02a33a4b43"}
Oct 30 21:50:08.452290 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:08.452266    2576 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-w5ck5"
Oct 30 21:50:08.488283 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:08.488251    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mpsnt" event={"ID":"d414339d-1a64-4a2a-ad15-805b71ebfd1b","Type":"ContainerStarted","Data":"4b2dec14b1f8f77fe577017254171ff7e1e41a86d9b32fa6758c7aba0a1148cc"}
Oct 30 21:50:08.488699 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:08.488316    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mpsnt" event={"ID":"d414339d-1a64-4a2a-ad15-805b71ebfd1b","Type":"ContainerStarted","Data":"73c7c05aae09d7da00fa60ec6dab2bf947d7849aa9835a7fac64d283660bb036"}
Oct 30 21:50:08.490612 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:08.490590    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-h95jn" event={"ID":"3abd55eb-d1d8-4fdf-b82c-d35894b652ac","Type":"ContainerStarted","Data":"70024b6a10441392b3fcef714818e67f1d5ac88c278772c3bc5917313db6e076"}
Oct 30 21:50:08.712477 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:08.712383    2576 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-h95jn" podStartSLOduration=3.087604111 podStartE2EDuration="4.712367761s" totalImagesPullingTime="1.62476365s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:50:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:50:08.707898541 +0000 UTC m=+61.139846810" watchObservedRunningTime="2026-10-30 21:50:08.712367761 +0000 UTC m=+61.144316029"
Oct 30 21:50:08.822252 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:08.822198    2576 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-mpsnt" podStartSLOduration=2.140756816 podStartE2EDuration="3.822183154s" totalImagesPullingTime="1.602377785s" totalInitContainerRuntime="79.048553ms" isStatefulPod=true podCreationTimestamp="2026-10-30 21:50:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:50:08.821153489 +0000 UTC m=+61.253101757" watchObservedRunningTime="2026-10-30 21:50:08.822183154 +0000 UTC m=+61.254131418"
Oct 30 21:50:08.924611 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:08.924575    2576 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-7c848fbc95-8jm99"]
Oct 30 21:50:08.928357 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:08.928336    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-7c848fbc95-8jm99"
Oct 30 21:50:08.941090 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:08.941068    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/8c88b730-5cf1-4434-950f-4a3aa8f739cd-secret-thanos-querier-tls\") pod \"thanos-querier-7c848fbc95-8jm99\" (UID: \"8c88b730-5cf1-4434-950f-4a3aa8f739cd\") " pod="openshift-monitoring/thanos-querier-7c848fbc95-8jm99"
Oct 30 21:50:08.941186 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:08.941110    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8c88b730-5cf1-4434-950f-4a3aa8f739cd-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7c848fbc95-8jm99\" (UID: \"8c88b730-5cf1-4434-950f-4a3aa8f739cd\") " pod="openshift-monitoring/thanos-querier-7c848fbc95-8jm99"
Oct 30 21:50:08.941186 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:08.941133    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8c88b730-5cf1-4434-950f-4a3aa8f739cd-metrics-client-ca\") pod \"thanos-querier-7c848fbc95-8jm99\" (UID: \"8c88b730-5cf1-4434-950f-4a3aa8f739cd\") " pod="openshift-monitoring/thanos-querier-7c848fbc95-8jm99"
Oct 30 21:50:08.941287 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:08.941225    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/8c88b730-5cf1-4434-950f-4a3aa8f739cd-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7c848fbc95-8jm99\" (UID: \"8c88b730-5cf1-4434-950f-4a3aa8f739cd\") " pod="openshift-monitoring/thanos-querier-7c848fbc95-8jm99"
Oct 30 21:50:08.941287 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:08.941264    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ttgkm\" (UniqueName: \"kubernetes.io/projected/8c88b730-5cf1-4434-950f-4a3aa8f739cd-kube-api-access-ttgkm\") pod \"thanos-querier-7c848fbc95-8jm99\" (UID: \"8c88b730-5cf1-4434-950f-4a3aa8f739cd\") " pod="openshift-monitoring/thanos-querier-7c848fbc95-8jm99"
Oct 30 21:50:08.941389 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:08.941285    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8c88b730-5cf1-4434-950f-4a3aa8f739cd-tmp\") pod \"thanos-querier-7c848fbc95-8jm99\" (UID: \"8c88b730-5cf1-4434-950f-4a3aa8f739cd\") " pod="openshift-monitoring/thanos-querier-7c848fbc95-8jm99"
Oct 30 21:50:08.941389 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:08.941357    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8c88b730-5cf1-4434-950f-4a3aa8f739cd-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7c848fbc95-8jm99\" (UID: \"8c88b730-5cf1-4434-950f-4a3aa8f739cd\") " pod="openshift-monitoring/thanos-querier-7c848fbc95-8jm99"
Oct 30 21:50:08.941478 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:08.941407    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8c88b730-5cf1-4434-950f-4a3aa8f739cd-secret-grpc-tls\") pod \"thanos-querier-7c848fbc95-8jm99\" (UID: \"8c88b730-5cf1-4434-950f-4a3aa8f739cd\") " pod="openshift-monitoring/thanos-querier-7c848fbc95-8jm99"
Oct 30 21:50:08.941478 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:08.941427    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/8c88b730-5cf1-4434-950f-4a3aa8f739cd-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7c848fbc95-8jm99\" (UID: \"8c88b730-5cf1-4434-950f-4a3aa8f739cd\") " pod="openshift-monitoring/thanos-querier-7c848fbc95-8jm99"
Oct 30 21:50:08.981472 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:08.981403    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Oct 30 21:50:08.981472 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:08.981404    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Oct 30 21:50:08.981631 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:08.981474    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-5ik3e6gooc4tm\""
Oct 30 21:50:08.981631 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:08.981499    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Oct 30 21:50:08.981631 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:08.981520    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-h2qxd\""
Oct 30 21:50:08.981631 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:08.981541    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Oct 30 21:50:08.981631 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:08.981543    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Oct 30 21:50:08.994709 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:08.994688    2576 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-7c848fbc95-8jm99"]
Oct 30 21:50:09.042172 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:09.042141    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/8c88b730-5cf1-4434-950f-4a3aa8f739cd-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7c848fbc95-8jm99\" (UID: \"8c88b730-5cf1-4434-950f-4a3aa8f739cd\") " pod="openshift-monitoring/thanos-querier-7c848fbc95-8jm99"
Oct 30 21:50:09.042306 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:09.042178    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/8c88b730-5cf1-4434-950f-4a3aa8f739cd-secret-thanos-querier-tls\") pod \"thanos-querier-7c848fbc95-8jm99\" (UID: \"8c88b730-5cf1-4434-950f-4a3aa8f739cd\") " pod="openshift-monitoring/thanos-querier-7c848fbc95-8jm99"
Oct 30 21:50:09.042306 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:09.042208    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8c88b730-5cf1-4434-950f-4a3aa8f739cd-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7c848fbc95-8jm99\" (UID: \"8c88b730-5cf1-4434-950f-4a3aa8f739cd\") " pod="openshift-monitoring/thanos-querier-7c848fbc95-8jm99"
Oct 30 21:50:09.042306 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:09.042242    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8c88b730-5cf1-4434-950f-4a3aa8f739cd-metrics-client-ca\") pod \"thanos-querier-7c848fbc95-8jm99\" (UID: \"8c88b730-5cf1-4434-950f-4a3aa8f739cd\") " pod="openshift-monitoring/thanos-querier-7c848fbc95-8jm99"
Oct 30 21:50:09.042306 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:09.042289    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/8c88b730-5cf1-4434-950f-4a3aa8f739cd-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7c848fbc95-8jm99\" (UID: \"8c88b730-5cf1-4434-950f-4a3aa8f739cd\") " pod="openshift-monitoring/thanos-querier-7c848fbc95-8jm99"
Oct 30 21:50:09.042480 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:09.042336    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ttgkm\" (UniqueName: \"kubernetes.io/projected/8c88b730-5cf1-4434-950f-4a3aa8f739cd-kube-api-access-ttgkm\") pod \"thanos-querier-7c848fbc95-8jm99\" (UID: \"8c88b730-5cf1-4434-950f-4a3aa8f739cd\") " pod="openshift-monitoring/thanos-querier-7c848fbc95-8jm99"
Oct 30 21:50:09.042546 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:09.042519    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8c88b730-5cf1-4434-950f-4a3aa8f739cd-tmp\") pod \"thanos-querier-7c848fbc95-8jm99\" (UID: \"8c88b730-5cf1-4434-950f-4a3aa8f739cd\") " pod="openshift-monitoring/thanos-querier-7c848fbc95-8jm99"
Oct 30 21:50:09.042601 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:09.042572    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8c88b730-5cf1-4434-950f-4a3aa8f739cd-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7c848fbc95-8jm99\" (UID: \"8c88b730-5cf1-4434-950f-4a3aa8f739cd\") " pod="openshift-monitoring/thanos-querier-7c848fbc95-8jm99"
Oct 30 21:50:09.042680 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:09.042627    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8c88b730-5cf1-4434-950f-4a3aa8f739cd-secret-grpc-tls\") pod \"thanos-querier-7c848fbc95-8jm99\" (UID: \"8c88b730-5cf1-4434-950f-4a3aa8f739cd\") " pod="openshift-monitoring/thanos-querier-7c848fbc95-8jm99"
Oct 30 21:50:09.042904 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:09.042875    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8c88b730-5cf1-4434-950f-4a3aa8f739cd-tmp\") pod \"thanos-querier-7c848fbc95-8jm99\" (UID: \"8c88b730-5cf1-4434-950f-4a3aa8f739cd\") " pod="openshift-monitoring/thanos-querier-7c848fbc95-8jm99"
Oct 30 21:50:09.043257 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:09.043234    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8c88b730-5cf1-4434-950f-4a3aa8f739cd-metrics-client-ca\") pod \"thanos-querier-7c848fbc95-8jm99\" (UID: \"8c88b730-5cf1-4434-950f-4a3aa8f739cd\") " pod="openshift-monitoring/thanos-querier-7c848fbc95-8jm99"
Oct 30 21:50:09.045157 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:09.045127    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/8c88b730-5cf1-4434-950f-4a3aa8f739cd-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7c848fbc95-8jm99\" (UID: \"8c88b730-5cf1-4434-950f-4a3aa8f739cd\") " pod="openshift-monitoring/thanos-querier-7c848fbc95-8jm99"
Oct 30 21:50:09.045271 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:09.045190    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/8c88b730-5cf1-4434-950f-4a3aa8f739cd-secret-thanos-querier-tls\") pod \"thanos-querier-7c848fbc95-8jm99\" (UID: \"8c88b730-5cf1-4434-950f-4a3aa8f739cd\") " pod="openshift-monitoring/thanos-querier-7c848fbc95-8jm99"
Oct 30 21:50:09.045271 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:09.045244    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8c88b730-5cf1-4434-950f-4a3aa8f739cd-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7c848fbc95-8jm99\" (UID: \"8c88b730-5cf1-4434-950f-4a3aa8f739cd\") " pod="openshift-monitoring/thanos-querier-7c848fbc95-8jm99"
Oct 30 21:50:09.045381 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:09.045359    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8c88b730-5cf1-4434-950f-4a3aa8f739cd-secret-grpc-tls\") pod \"thanos-querier-7c848fbc95-8jm99\" (UID: \"8c88b730-5cf1-4434-950f-4a3aa8f739cd\") " pod="openshift-monitoring/thanos-querier-7c848fbc95-8jm99"
Oct 30 21:50:09.045430 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:09.045389    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/8c88b730-5cf1-4434-950f-4a3aa8f739cd-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7c848fbc95-8jm99\" (UID: \"8c88b730-5cf1-4434-950f-4a3aa8f739cd\") " pod="openshift-monitoring/thanos-querier-7c848fbc95-8jm99"
Oct 30 21:50:09.045482 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:09.045432    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8c88b730-5cf1-4434-950f-4a3aa8f739cd-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7c848fbc95-8jm99\" (UID: \"8c88b730-5cf1-4434-950f-4a3aa8f739cd\") " pod="openshift-monitoring/thanos-querier-7c848fbc95-8jm99"
Oct 30 21:50:09.102716 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:09.102695    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ttgkm\" (UniqueName: \"kubernetes.io/projected/8c88b730-5cf1-4434-950f-4a3aa8f739cd-kube-api-access-ttgkm\") pod \"thanos-querier-7c848fbc95-8jm99\" (UID: \"8c88b730-5cf1-4434-950f-4a3aa8f739cd\") " pod="openshift-monitoring/thanos-querier-7c848fbc95-8jm99"
Oct 30 21:50:09.250598 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:09.250514    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-7c848fbc95-8jm99"
Oct 30 21:50:09.328896 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:09.328850    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:09.494953 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:09.494920    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6d5866d6cd-b6dpl" event={"ID":"42d54b8a-4f48-46a7-8a34-1f0f29cd1602","Type":"ContainerStarted","Data":"c8252358442af04b3522aa680915b0a81265fed2df35832d0ac243fc1ce490c0"}
Oct 30 21:50:09.596417 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:09.596384    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:09.615614 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:09.615563    2576 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/console-6d5866d6cd-b6dpl" podStartSLOduration=2.187204012 podStartE2EDuration="5.615550251s" totalImagesPullingTime="3.428346239s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:50:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:50:09.615093787 +0000 UTC m=+62.047042054" watchObservedRunningTime="2026-10-30 21:50:09.615550251 +0000 UTC m=+62.047498519"
Oct 30 21:50:09.724703 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:09.724679    2576 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-7c848fbc95-8jm99"]
Oct 30 21:50:09.726866 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:50:09.726838    2576 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c88b730_5cf1_4434_950f_4a3aa8f739cd.slice/crio-02e263b2978459859989e1cdc818ce3ef741757f7d74157e4fb4f4a83c1c0947 WatchSource:0}: Error finding container 02e263b2978459859989e1cdc818ce3ef741757f7d74157e4fb4f4a83c1c0947: Status 404 returned error can't find the container with id 02e263b2978459859989e1cdc818ce3ef741757f7d74157e4fb4f4a83c1c0947
Oct 30 21:50:09.837613 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:09.837582    2576 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-77f4f76594-bfnw9"]
Oct 30 21:50:09.842081 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:09.842059    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-77f4f76594-bfnw9"
Oct 30 21:50:09.898218 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:09.898190    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:09.903273 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:09.903250    2576 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-77f4f76594-bfnw9"]
Oct 30 21:50:09.917586 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:09.917564    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-xkkts\""
Oct 30 21:50:09.917724 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:09.917696    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Oct 30 21:50:09.948582 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:09.948555    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8883cb69-54c2-437c-a92e-3bc38547027f-monitoring-plugin-cert\") pod \"monitoring-plugin-77f4f76594-bfnw9\" (UID: \"8883cb69-54c2-437c-a92e-3bc38547027f\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-bfnw9"
Oct 30 21:50:09.948704 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:09.948593    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b2929\" (UniqueName: \"kubernetes.io/projected/8883cb69-54c2-437c-a92e-3bc38547027f-kube-api-access-b2929\") pod \"monitoring-plugin-77f4f76594-bfnw9\" (UID: \"8883cb69-54c2-437c-a92e-3bc38547027f\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-bfnw9"
Oct 30 21:50:10.049928 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:10.049896    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8883cb69-54c2-437c-a92e-3bc38547027f-monitoring-plugin-cert\") pod \"monitoring-plugin-77f4f76594-bfnw9\" (UID: \"8883cb69-54c2-437c-a92e-3bc38547027f\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-bfnw9"
Oct 30 21:50:10.050078 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:10.049944    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b2929\" (UniqueName: \"kubernetes.io/projected/8883cb69-54c2-437c-a92e-3bc38547027f-kube-api-access-b2929\") pod \"monitoring-plugin-77f4f76594-bfnw9\" (UID: \"8883cb69-54c2-437c-a92e-3bc38547027f\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-bfnw9"
Oct 30 21:50:10.052321 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:10.052296    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8883cb69-54c2-437c-a92e-3bc38547027f-monitoring-plugin-cert\") pod \"monitoring-plugin-77f4f76594-bfnw9\" (UID: \"8883cb69-54c2-437c-a92e-3bc38547027f\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-bfnw9"
Oct 30 21:50:10.108543 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:10.108477    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b2929\" (UniqueName: \"kubernetes.io/projected/8883cb69-54c2-437c-a92e-3bc38547027f-kube-api-access-b2929\") pod \"monitoring-plugin-77f4f76594-bfnw9\" (UID: \"8883cb69-54c2-437c-a92e-3bc38547027f\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-bfnw9"
Oct 30 21:50:10.152571 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:10.152543    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-77f4f76594-bfnw9"
Oct 30 21:50:10.154879 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:10.154854    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:10.498585 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:10.498506    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7c848fbc95-8jm99" event={"ID":"8c88b730-5cf1-4434-950f-4a3aa8f739cd","Type":"ContainerStarted","Data":"02e263b2978459859989e1cdc818ce3ef741757f7d74157e4fb4f4a83c1c0947"}
Oct 30 21:50:10.524740 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:10.524682    2576 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-77f4f76594-bfnw9"]
Oct 30 21:50:10.526924 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:50:10.526900    2576 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8883cb69_54c2_437c_a92e_3bc38547027f.slice/crio-e706fb1e7061ada9e54ae7abc07fc31473d443d12a0371f11a83d19444e35915 WatchSource:0}: Error finding container e706fb1e7061ada9e54ae7abc07fc31473d443d12a0371f11a83d19444e35915: Status 404 returned error can't find the container with id e706fb1e7061ada9e54ae7abc07fc31473d443d12a0371f11a83d19444e35915
Oct 30 21:50:10.618985 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:10.618947    2576 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6d5866d6cd-b6dpl"]
Oct 30 21:50:10.656063 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:10.656035    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:10.683021 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:10.682989    2576 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-748df94584-88cjt"]
Oct 30 21:50:10.686965 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:10.686944    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-748df94584-88cjt"
Oct 30 21:50:10.751423 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:10.751362    2576 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-748df94584-88cjt"]
Oct 30 21:50:10.755701 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:10.755677    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/31a389bb-f36f-4e53-9654-5f4648dae38a-console-oauth-config\") pod \"console-748df94584-88cjt\" (UID: \"31a389bb-f36f-4e53-9654-5f4648dae38a\") " pod="openshift-console/console-748df94584-88cjt"
Oct 30 21:50:10.755822 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:10.755708    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/31a389bb-f36f-4e53-9654-5f4648dae38a-tmp\") pod \"console-748df94584-88cjt\" (UID: \"31a389bb-f36f-4e53-9654-5f4648dae38a\") " pod="openshift-console/console-748df94584-88cjt"
Oct 30 21:50:10.755822 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:10.755806    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/31a389bb-f36f-4e53-9654-5f4648dae38a-oauth-serving-cert\") pod \"console-748df94584-88cjt\" (UID: \"31a389bb-f36f-4e53-9654-5f4648dae38a\") " pod="openshift-console/console-748df94584-88cjt"
Oct 30 21:50:10.755926 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:10.755825    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rb2wf\" (UniqueName: \"kubernetes.io/projected/31a389bb-f36f-4e53-9654-5f4648dae38a-kube-api-access-rb2wf\") pod \"console-748df94584-88cjt\" (UID: \"31a389bb-f36f-4e53-9654-5f4648dae38a\") " pod="openshift-console/console-748df94584-88cjt"
Oct 30 21:50:10.755926 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:10.755903    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/31a389bb-f36f-4e53-9654-5f4648dae38a-service-ca\") pod \"console-748df94584-88cjt\" (UID: \"31a389bb-f36f-4e53-9654-5f4648dae38a\") " pod="openshift-console/console-748df94584-88cjt"
Oct 30 21:50:10.756010 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:10.755946    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/31a389bb-f36f-4e53-9654-5f4648dae38a-console-serving-cert\") pod \"console-748df94584-88cjt\" (UID: \"31a389bb-f36f-4e53-9654-5f4648dae38a\") " pod="openshift-console/console-748df94584-88cjt"
Oct 30 21:50:10.756010 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:10.755963    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/31a389bb-f36f-4e53-9654-5f4648dae38a-trusted-ca-bundle\") pod \"console-748df94584-88cjt\" (UID: \"31a389bb-f36f-4e53-9654-5f4648dae38a\") " pod="openshift-console/console-748df94584-88cjt"
Oct 30 21:50:10.756010 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:10.755983    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/31a389bb-f36f-4e53-9654-5f4648dae38a-console-config\") pod \"console-748df94584-88cjt\" (UID: \"31a389bb-f36f-4e53-9654-5f4648dae38a\") " pod="openshift-console/console-748df94584-88cjt"
Oct 30 21:50:10.857014 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:10.856949    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/31a389bb-f36f-4e53-9654-5f4648dae38a-service-ca\") pod \"console-748df94584-88cjt\" (UID: \"31a389bb-f36f-4e53-9654-5f4648dae38a\") " pod="openshift-console/console-748df94584-88cjt"
Oct 30 21:50:10.857014 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:10.857010    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/31a389bb-f36f-4e53-9654-5f4648dae38a-console-serving-cert\") pod \"console-748df94584-88cjt\" (UID: \"31a389bb-f36f-4e53-9654-5f4648dae38a\") " pod="openshift-console/console-748df94584-88cjt"
Oct 30 21:50:10.857292 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:10.857041    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/31a389bb-f36f-4e53-9654-5f4648dae38a-trusted-ca-bundle\") pod \"console-748df94584-88cjt\" (UID: \"31a389bb-f36f-4e53-9654-5f4648dae38a\") " pod="openshift-console/console-748df94584-88cjt"
Oct 30 21:50:10.857292 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:10.857068    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/31a389bb-f36f-4e53-9654-5f4648dae38a-console-config\") pod \"console-748df94584-88cjt\" (UID: \"31a389bb-f36f-4e53-9654-5f4648dae38a\") " pod="openshift-console/console-748df94584-88cjt"
Oct 30 21:50:10.857292 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:10.857152    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/31a389bb-f36f-4e53-9654-5f4648dae38a-console-oauth-config\") pod \"console-748df94584-88cjt\" (UID: \"31a389bb-f36f-4e53-9654-5f4648dae38a\") " pod="openshift-console/console-748df94584-88cjt"
Oct 30 21:50:10.857292 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:10.857179    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/31a389bb-f36f-4e53-9654-5f4648dae38a-tmp\") pod \"console-748df94584-88cjt\" (UID: \"31a389bb-f36f-4e53-9654-5f4648dae38a\") " pod="openshift-console/console-748df94584-88cjt"
Oct 30 21:50:10.857292 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:10.857236    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/31a389bb-f36f-4e53-9654-5f4648dae38a-oauth-serving-cert\") pod \"console-748df94584-88cjt\" (UID: \"31a389bb-f36f-4e53-9654-5f4648dae38a\") " pod="openshift-console/console-748df94584-88cjt"
Oct 30 21:50:10.857292 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:10.857264    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rb2wf\" (UniqueName: \"kubernetes.io/projected/31a389bb-f36f-4e53-9654-5f4648dae38a-kube-api-access-rb2wf\") pod \"console-748df94584-88cjt\" (UID: \"31a389bb-f36f-4e53-9654-5f4648dae38a\") " pod="openshift-console/console-748df94584-88cjt"
Oct 30 21:50:10.858445 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:10.857882    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/31a389bb-f36f-4e53-9654-5f4648dae38a-tmp\") pod \"console-748df94584-88cjt\" (UID: \"31a389bb-f36f-4e53-9654-5f4648dae38a\") " pod="openshift-console/console-748df94584-88cjt"
Oct 30 21:50:10.858445 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:10.858051    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/31a389bb-f36f-4e53-9654-5f4648dae38a-trusted-ca-bundle\") pod \"console-748df94584-88cjt\" (UID: \"31a389bb-f36f-4e53-9654-5f4648dae38a\") " pod="openshift-console/console-748df94584-88cjt"
Oct 30 21:50:10.858445 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:10.858066    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/31a389bb-f36f-4e53-9654-5f4648dae38a-oauth-serving-cert\") pod \"console-748df94584-88cjt\" (UID: \"31a389bb-f36f-4e53-9654-5f4648dae38a\") " pod="openshift-console/console-748df94584-88cjt"
Oct 30 21:50:10.858445 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:10.858074    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/31a389bb-f36f-4e53-9654-5f4648dae38a-console-config\") pod \"console-748df94584-88cjt\" (UID: \"31a389bb-f36f-4e53-9654-5f4648dae38a\") " pod="openshift-console/console-748df94584-88cjt"
Oct 30 21:50:10.858636 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:10.858496    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/31a389bb-f36f-4e53-9654-5f4648dae38a-service-ca\") pod \"console-748df94584-88cjt\" (UID: \"31a389bb-f36f-4e53-9654-5f4648dae38a\") " pod="openshift-console/console-748df94584-88cjt"
Oct 30 21:50:10.860226 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:10.860198    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/31a389bb-f36f-4e53-9654-5f4648dae38a-console-oauth-config\") pod \"console-748df94584-88cjt\" (UID: \"31a389bb-f36f-4e53-9654-5f4648dae38a\") " pod="openshift-console/console-748df94584-88cjt"
Oct 30 21:50:10.860360 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:10.860237    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/31a389bb-f36f-4e53-9654-5f4648dae38a-console-serving-cert\") pod \"console-748df94584-88cjt\" (UID: \"31a389bb-f36f-4e53-9654-5f4648dae38a\") " pod="openshift-console/console-748df94584-88cjt"
Oct 30 21:50:10.920891 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:10.920851    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rb2wf\" (UniqueName: \"kubernetes.io/projected/31a389bb-f36f-4e53-9654-5f4648dae38a-kube-api-access-rb2wf\") pod \"console-748df94584-88cjt\" (UID: \"31a389bb-f36f-4e53-9654-5f4648dae38a\") " pod="openshift-console/console-748df94584-88cjt"
Oct 30 21:50:10.957789 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:10.957754    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:10.997968 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:10.997935    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-748df94584-88cjt"
Oct 30 21:50:11.023072 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:11.023047    2576 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-bb95b88bf-g9f94"]
Oct 30 21:50:11.027591 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:11.027569    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-bb95b88bf-g9f94"
Oct 30 21:50:11.063254 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:11.058174    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0e0d17a1-7742-42b0-9d9b-740bb945e454-serving-certs-ca-bundle\") pod \"telemeter-client-bb95b88bf-g9f94\" (UID: \"0e0d17a1-7742-42b0-9d9b-740bb945e454\") " pod="openshift-monitoring/telemeter-client-bb95b88bf-g9f94"
Oct 30 21:50:11.063254 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:11.058258    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/0e0d17a1-7742-42b0-9d9b-740bb945e454-secret-telemeter-client\") pod \"telemeter-client-bb95b88bf-g9f94\" (UID: \"0e0d17a1-7742-42b0-9d9b-740bb945e454\") " pod="openshift-monitoring/telemeter-client-bb95b88bf-g9f94"
Oct 30 21:50:11.063254 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:11.058287    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9gzpz\" (UniqueName: \"kubernetes.io/projected/0e0d17a1-7742-42b0-9d9b-740bb945e454-kube-api-access-9gzpz\") pod \"telemeter-client-bb95b88bf-g9f94\" (UID: \"0e0d17a1-7742-42b0-9d9b-740bb945e454\") " pod="openshift-monitoring/telemeter-client-bb95b88bf-g9f94"
Oct 30 21:50:11.063254 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:11.058322    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0e0d17a1-7742-42b0-9d9b-740bb945e454-telemeter-trusted-ca-bundle\") pod \"telemeter-client-bb95b88bf-g9f94\" (UID: \"0e0d17a1-7742-42b0-9d9b-740bb945e454\") " pod="openshift-monitoring/telemeter-client-bb95b88bf-g9f94"
Oct 30 21:50:11.063254 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:11.058372    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/0e0d17a1-7742-42b0-9d9b-740bb945e454-telemeter-client-tls\") pod \"telemeter-client-bb95b88bf-g9f94\" (UID: \"0e0d17a1-7742-42b0-9d9b-740bb945e454\") " pod="openshift-monitoring/telemeter-client-bb95b88bf-g9f94"
Oct 30 21:50:11.063254 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:11.058409    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0e0d17a1-7742-42b0-9d9b-740bb945e454-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-bb95b88bf-g9f94\" (UID: \"0e0d17a1-7742-42b0-9d9b-740bb945e454\") " pod="openshift-monitoring/telemeter-client-bb95b88bf-g9f94"
Oct 30 21:50:11.063254 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:11.058435    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0e0d17a1-7742-42b0-9d9b-740bb945e454-metrics-client-ca\") pod \"telemeter-client-bb95b88bf-g9f94\" (UID: \"0e0d17a1-7742-42b0-9d9b-740bb945e454\") " pod="openshift-monitoring/telemeter-client-bb95b88bf-g9f94"
Oct 30 21:50:11.063254 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:11.058484    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/0e0d17a1-7742-42b0-9d9b-740bb945e454-federate-client-tls\") pod \"telemeter-client-bb95b88bf-g9f94\" (UID: \"0e0d17a1-7742-42b0-9d9b-740bb945e454\") " pod="openshift-monitoring/telemeter-client-bb95b88bf-g9f94"
Oct 30 21:50:11.083717 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:11.080817    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Oct 30 21:50:11.127966 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:11.127938    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-28r105lbnph2c\""
Oct 30 21:50:11.129257 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:11.129228    2576 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-bb95b88bf-g9f94"]
Oct 30 21:50:11.148343 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:11.148314    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Oct 30 21:50:11.148343 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:11.148315    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Oct 30 21:50:11.148544 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:11.148436    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Oct 30 21:50:11.148625 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:11.148609    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Oct 30 21:50:11.148692 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:11.148647    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-pg2zk\""
Oct 30 21:50:11.159406 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:11.159379    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/0e0d17a1-7742-42b0-9d9b-740bb945e454-telemeter-client-tls\") pod \"telemeter-client-bb95b88bf-g9f94\" (UID: \"0e0d17a1-7742-42b0-9d9b-740bb945e454\") " pod="openshift-monitoring/telemeter-client-bb95b88bf-g9f94"
Oct 30 21:50:11.159548 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:11.159426    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0e0d17a1-7742-42b0-9d9b-740bb945e454-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-bb95b88bf-g9f94\" (UID: \"0e0d17a1-7742-42b0-9d9b-740bb945e454\") " pod="openshift-monitoring/telemeter-client-bb95b88bf-g9f94"
Oct 30 21:50:11.159548 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:11.159455    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0e0d17a1-7742-42b0-9d9b-740bb945e454-metrics-client-ca\") pod \"telemeter-client-bb95b88bf-g9f94\" (UID: \"0e0d17a1-7742-42b0-9d9b-740bb945e454\") " pod="openshift-monitoring/telemeter-client-bb95b88bf-g9f94"
Oct 30 21:50:11.159548 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:11.159504    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/0e0d17a1-7742-42b0-9d9b-740bb945e454-federate-client-tls\") pod \"telemeter-client-bb95b88bf-g9f94\" (UID: \"0e0d17a1-7742-42b0-9d9b-740bb945e454\") " pod="openshift-monitoring/telemeter-client-bb95b88bf-g9f94"
Oct 30 21:50:11.159725 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:11.159547    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0e0d17a1-7742-42b0-9d9b-740bb945e454-serving-certs-ca-bundle\") pod \"telemeter-client-bb95b88bf-g9f94\" (UID: \"0e0d17a1-7742-42b0-9d9b-740bb945e454\") " pod="openshift-monitoring/telemeter-client-bb95b88bf-g9f94"
Oct 30 21:50:11.159725 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:11.159610    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/0e0d17a1-7742-42b0-9d9b-740bb945e454-secret-telemeter-client\") pod \"telemeter-client-bb95b88bf-g9f94\" (UID: \"0e0d17a1-7742-42b0-9d9b-740bb945e454\") " pod="openshift-monitoring/telemeter-client-bb95b88bf-g9f94"
Oct 30 21:50:11.159725 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:11.159634    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9gzpz\" (UniqueName: \"kubernetes.io/projected/0e0d17a1-7742-42b0-9d9b-740bb945e454-kube-api-access-9gzpz\") pod \"telemeter-client-bb95b88bf-g9f94\" (UID: \"0e0d17a1-7742-42b0-9d9b-740bb945e454\") " pod="openshift-monitoring/telemeter-client-bb95b88bf-g9f94"
Oct 30 21:50:11.159725 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:11.159688    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0e0d17a1-7742-42b0-9d9b-740bb945e454-telemeter-trusted-ca-bundle\") pod \"telemeter-client-bb95b88bf-g9f94\" (UID: \"0e0d17a1-7742-42b0-9d9b-740bb945e454\") " pod="openshift-monitoring/telemeter-client-bb95b88bf-g9f94"
Oct 30 21:50:11.160222 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:11.160198    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0e0d17a1-7742-42b0-9d9b-740bb945e454-metrics-client-ca\") pod \"telemeter-client-bb95b88bf-g9f94\" (UID: \"0e0d17a1-7742-42b0-9d9b-740bb945e454\") " pod="openshift-monitoring/telemeter-client-bb95b88bf-g9f94"
Oct 30 21:50:11.160351 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:11.160272    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0e0d17a1-7742-42b0-9d9b-740bb945e454-serving-certs-ca-bundle\") pod \"telemeter-client-bb95b88bf-g9f94\" (UID: \"0e0d17a1-7742-42b0-9d9b-740bb945e454\") " pod="openshift-monitoring/telemeter-client-bb95b88bf-g9f94"
Oct 30 21:50:11.160528 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:11.160506    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0e0d17a1-7742-42b0-9d9b-740bb945e454-telemeter-trusted-ca-bundle\") pod \"telemeter-client-bb95b88bf-g9f94\" (UID: \"0e0d17a1-7742-42b0-9d9b-740bb945e454\") " pod="openshift-monitoring/telemeter-client-bb95b88bf-g9f94"
Oct 30 21:50:11.162507 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:11.162482    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0e0d17a1-7742-42b0-9d9b-740bb945e454-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-bb95b88bf-g9f94\" (UID: \"0e0d17a1-7742-42b0-9d9b-740bb945e454\") " pod="openshift-monitoring/telemeter-client-bb95b88bf-g9f94"
Oct 30 21:50:11.162870 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:11.162852    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/0e0d17a1-7742-42b0-9d9b-740bb945e454-federate-client-tls\") pod \"telemeter-client-bb95b88bf-g9f94\" (UID: \"0e0d17a1-7742-42b0-9d9b-740bb945e454\") " pod="openshift-monitoring/telemeter-client-bb95b88bf-g9f94"
Oct 30 21:50:11.164131 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:11.164110    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/0e0d17a1-7742-42b0-9d9b-740bb945e454-telemeter-client-tls\") pod \"telemeter-client-bb95b88bf-g9f94\" (UID: \"0e0d17a1-7742-42b0-9d9b-740bb945e454\") " pod="openshift-monitoring/telemeter-client-bb95b88bf-g9f94"
Oct 30 21:50:11.164131 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:11.164118    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/0e0d17a1-7742-42b0-9d9b-740bb945e454-secret-telemeter-client\") pod \"telemeter-client-bb95b88bf-g9f94\" (UID: \"0e0d17a1-7742-42b0-9d9b-740bb945e454\") " pod="openshift-monitoring/telemeter-client-bb95b88bf-g9f94"
Oct 30 21:50:11.222819 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:11.222758    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9gzpz\" (UniqueName: \"kubernetes.io/projected/0e0d17a1-7742-42b0-9d9b-740bb945e454-kube-api-access-9gzpz\") pod \"telemeter-client-bb95b88bf-g9f94\" (UID: \"0e0d17a1-7742-42b0-9d9b-740bb945e454\") " pod="openshift-monitoring/telemeter-client-bb95b88bf-g9f94"
Oct 30 21:50:11.345821 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:11.345780    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-bb95b88bf-g9f94"
Oct 30 21:50:11.378543 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:11.378511    2576 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-748df94584-88cjt"]
Oct 30 21:50:11.385311 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:50:11.385270    2576 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod31a389bb_f36f_4e53_9654_5f4648dae38a.slice/crio-d188c9559e229f137adf7ecb5417f5f9310e2d711a052c0aece44396a292cae9 WatchSource:0}: Error finding container d188c9559e229f137adf7ecb5417f5f9310e2d711a052c0aece44396a292cae9: Status 404 returned error can't find the container with id d188c9559e229f137adf7ecb5417f5f9310e2d711a052c0aece44396a292cae9
Oct 30 21:50:11.507912 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:11.507872    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-748df94584-88cjt" event={"ID":"31a389bb-f36f-4e53-9654-5f4648dae38a","Type":"ContainerStarted","Data":"be8d014ac774c705b67487af5cf43bac9c0460dc8d8be9cde6f1501fd7fdd9b5"}
Oct 30 21:50:11.508329 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:11.507938    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-748df94584-88cjt" event={"ID":"31a389bb-f36f-4e53-9654-5f4648dae38a","Type":"ContainerStarted","Data":"d188c9559e229f137adf7ecb5417f5f9310e2d711a052c0aece44396a292cae9"}
Oct 30 21:50:11.511384 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:11.511357    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-77f4f76594-bfnw9" event={"ID":"8883cb69-54c2-437c-a92e-3bc38547027f","Type":"ContainerStarted","Data":"e706fb1e7061ada9e54ae7abc07fc31473d443d12a0371f11a83d19444e35915"}
Oct 30 21:50:11.627763 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:11.627649    2576 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/console-748df94584-88cjt" podStartSLOduration=1.6276353669999999 podStartE2EDuration="1.627635367s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:50:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:50:11.626730547 +0000 UTC m=+64.058678815" watchObservedRunningTime="2026-10-30 21:50:11.627635367 +0000 UTC m=+64.059583613"
Oct 30 21:50:11.757487 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:11.757432    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:11.794856 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:11.794823    2576 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-bb95b88bf-g9f94"]
Oct 30 21:50:11.803208 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:50:11.799017    2576 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e0d17a1_7742_42b0_9d9b_740bb945e454.slice/crio-f4065939ad3a362fabd190f4f532a0ba04768adf8730414343ea996a097c52f5 WatchSource:0}: Error finding container f4065939ad3a362fabd190f4f532a0ba04768adf8730414343ea996a097c52f5: Status 404 returned error can't find the container with id f4065939ad3a362fabd190f4f532a0ba04768adf8730414343ea996a097c52f5
Oct 30 21:50:12.099908 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:12.099868    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:12.515865 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:12.515828    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7c848fbc95-8jm99" event={"ID":"8c88b730-5cf1-4434-950f-4a3aa8f739cd","Type":"ContainerStarted","Data":"ed2e50e22dc0a9b74e0a4e7fd8809adcd1b9c2615af58af4b78abbfff6138821"}
Oct 30 21:50:12.517216 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:12.517142    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-bb95b88bf-g9f94" event={"ID":"0e0d17a1-7742-42b0-9d9b-740bb945e454","Type":"ContainerStarted","Data":"f4065939ad3a362fabd190f4f532a0ba04768adf8730414343ea996a097c52f5"}
Oct 30 21:50:13.076531 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:13.076494    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/52eb01f8-e86a-4458-8875-639394a87b30-metrics-certs\") pod \"network-metrics-daemon-vmrkz\" (UID: \"52eb01f8-e86a-4458-8875-639394a87b30\") " pod="openshift-multus/network-metrics-daemon-vmrkz"
Oct 30 21:50:13.076749 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:13.076559    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e810bcd1-78da-409f-b86e-08c47031939e-original-pull-secret\") pod \"global-pull-secret-syncer-68jrf\" (UID: \"e810bcd1-78da-409f-b86e-08c47031939e\") " pod="kube-system/global-pull-secret-syncer-68jrf"
Oct 30 21:50:13.125375 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:13.125343    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Oct 30 21:50:13.125375 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:13.125362    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Oct 30 21:50:13.130356 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:13.130327    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e810bcd1-78da-409f-b86e-08c47031939e-original-pull-secret\") pod \"global-pull-secret-syncer-68jrf\" (UID: \"e810bcd1-78da-409f-b86e-08c47031939e\") " pod="kube-system/global-pull-secret-syncer-68jrf"
Oct 30 21:50:13.131531 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:13.131510    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/52eb01f8-e86a-4458-8875-639394a87b30-metrics-certs\") pod \"network-metrics-daemon-vmrkz\" (UID: \"52eb01f8-e86a-4458-8875-639394a87b30\") " pod="openshift-multus/network-metrics-daemon-vmrkz"
Oct 30 21:50:13.177709 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:13.177670    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fcjg4\" (UniqueName: \"kubernetes.io/projected/73c208eb-ffc0-46f2-8bd4-05ade5adb529-kube-api-access-fcjg4\") pod \"network-check-target-jmbng\" (UID: \"73c208eb-ffc0-46f2-8bd4-05ade5adb529\") " pod="openshift-network-diagnostics/network-check-target-jmbng"
Oct 30 21:50:13.226630 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:13.226595    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Oct 30 21:50:13.268552 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:13.268514    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-jbflf\""
Oct 30 21:50:13.270756 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:13.270724    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-68jrf"
Oct 30 21:50:13.277075 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:13.277052    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Oct 30 21:50:13.282096 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:13.282066    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fcjg4\" (UniqueName: \"kubernetes.io/projected/73c208eb-ffc0-46f2-8bd4-05ade5adb529-kube-api-access-fcjg4\") pod \"network-check-target-jmbng\" (UID: \"73c208eb-ffc0-46f2-8bd4-05ade5adb529\") " pod="openshift-network-diagnostics/network-check-target-jmbng"
Oct 30 21:50:13.368248 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:13.368222    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-9bjlv\""
Oct 30 21:50:13.370293 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:13.370258    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vmrkz"
Oct 30 21:50:13.523976 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:13.523937    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7c848fbc95-8jm99" event={"ID":"8c88b730-5cf1-4434-950f-4a3aa8f739cd","Type":"ContainerStarted","Data":"580b9dc30380273e85acbe6fa44e1cd5649705b9c86711745c0d340481b5a44c"}
Oct 30 21:50:13.524428 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:13.524035    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7c848fbc95-8jm99" event={"ID":"8c88b730-5cf1-4434-950f-4a3aa8f739cd","Type":"ContainerStarted","Data":"dc14aaeedd0fafd608be285f00ec53a290989bfd58295deac0474ce0258bcaf1"}
Oct 30 21:50:13.525617 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:13.525586    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-77f4f76594-bfnw9" event={"ID":"8883cb69-54c2-437c-a92e-3bc38547027f","Type":"ContainerStarted","Data":"3d0dfe117a14ed8b806dce2cd85fd6fabf9f16b3494641505c674836b9e48dfd"}
Oct 30 21:50:13.525908 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:13.525883    2576 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-77f4f76594-bfnw9"
Oct 30 21:50:13.533044 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:13.533021    2576 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-77f4f76594-bfnw9"
Oct 30 21:50:13.543092 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:13.543050    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:13.572498 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:13.572473    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-9mgc8\""
Oct 30 21:50:13.574491 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:13.574466    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jmbng"
Oct 30 21:50:13.636692 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:13.636045    2576 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-77f4f76594-bfnw9" podStartSLOduration=2.602292421 podStartE2EDuration="4.636024899s" totalImagesPullingTime="2.033732478s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:50:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:50:13.635468331 +0000 UTC m=+66.067416600" watchObservedRunningTime="2026-10-30 21:50:13.636024899 +0000 UTC m=+66.067973169"
Oct 30 21:50:13.651918 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:13.651686    2576 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-68jrf"]
Oct 30 21:50:13.655210 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:50:13.655185    2576 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode810bcd1_78da_409f_b86e_08c47031939e.slice/crio-ce29c22093919e23269e62f2db4fc0b855f665df8a6eb65b9945172a1594cc33 WatchSource:0}: Error finding container ce29c22093919e23269e62f2db4fc0b855f665df8a6eb65b9945172a1594cc33: Status 404 returned error can't find the container with id ce29c22093919e23269e62f2db4fc0b855f665df8a6eb65b9945172a1594cc33
Oct 30 21:50:13.746268 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:13.746231    2576 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-vmrkz"]
Oct 30 21:50:13.752847 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:50:13.752804    2576 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52eb01f8_e86a_4458_8875_639394a87b30.slice/crio-76daddeb7d1867d1fe0158bb897cbf216243b7ff1593fa67f48849c8d3d8ddab WatchSource:0}: Error finding container 76daddeb7d1867d1fe0158bb897cbf216243b7ff1593fa67f48849c8d3d8ddab: Status 404 returned error can't find the container with id 76daddeb7d1867d1fe0158bb897cbf216243b7ff1593fa67f48849c8d3d8ddab
Oct 30 21:50:13.758406 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:13.758376    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:13.975889 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:13.975804    2576 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-jmbng"]
Oct 30 21:50:13.982333 ip-10-0-136-210 kubenswrapper[2576]: W1030 21:50:13.982303    2576 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73c208eb_ffc0_46f2_8bd4_05ade5adb529.slice/crio-d90f0f601a7259b584b3cceb562a51226b1daf12d6df216a04e38d8680c26a8e WatchSource:0}: Error finding container d90f0f601a7259b584b3cceb562a51226b1daf12d6df216a04e38d8680c26a8e: Status 404 returned error can't find the container with id d90f0f601a7259b584b3cceb562a51226b1daf12d6df216a04e38d8680c26a8e
Oct 30 21:50:14.142434 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:14.142406    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:14.340026 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:14.339995    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:14.532635 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:14.532601    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7c848fbc95-8jm99" event={"ID":"8c88b730-5cf1-4434-950f-4a3aa8f739cd","Type":"ContainerStarted","Data":"952f49367ed3358a3fc704fae4f3fc29a3c650dae9fc5af32df73246dffc85ba"}
Oct 30 21:50:14.533066 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:14.532694    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7c848fbc95-8jm99" event={"ID":"8c88b730-5cf1-4434-950f-4a3aa8f739cd","Type":"ContainerStarted","Data":"9784d9a70b0bf67c236fff44de24e505bacaaf157e021c8ed14f9259dc4f9ce1"}
Oct 30 21:50:14.533066 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:14.532733    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7c848fbc95-8jm99" event={"ID":"8c88b730-5cf1-4434-950f-4a3aa8f739cd","Type":"ContainerStarted","Data":"2cdf8fb1c29e46df03138e8f8135b9183a1876f4bb4b46efb4c4fc93add4e719"}
Oct 30 21:50:14.533066 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:14.532895    2576 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-7c848fbc95-8jm99"
Oct 30 21:50:14.533986 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:14.533962    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-vmrkz" event={"ID":"52eb01f8-e86a-4458-8875-639394a87b30","Type":"ContainerStarted","Data":"76daddeb7d1867d1fe0158bb897cbf216243b7ff1593fa67f48849c8d3d8ddab"}
Oct 30 21:50:14.535115 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:14.535092    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-jmbng" event={"ID":"73c208eb-ffc0-46f2-8bd4-05ade5adb529","Type":"ContainerStarted","Data":"d90f0f601a7259b584b3cceb562a51226b1daf12d6df216a04e38d8680c26a8e"}
Oct 30 21:50:14.536193 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:14.536174    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-68jrf" event={"ID":"e810bcd1-78da-409f-b86e-08c47031939e","Type":"ContainerStarted","Data":"ce29c22093919e23269e62f2db4fc0b855f665df8a6eb65b9945172a1594cc33"}
Oct 30 21:50:14.537634 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:14.537611    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-bb95b88bf-g9f94" event={"ID":"0e0d17a1-7742-42b0-9d9b-740bb945e454","Type":"ContainerStarted","Data":"7665eba614cf5fda6a1f1386018c825c1a5503a217baffbaea14655e5132686c"}
Oct 30 21:50:14.617211 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:14.617180    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:14.626645 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:14.626609    2576 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6d5866d6cd-b6dpl"
Oct 30 21:50:14.662681 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:14.662605    2576 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-7c848fbc95-8jm99" podStartSLOduration=2.544303373 podStartE2EDuration="6.662588238s" totalImagesPullingTime="4.118284865s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:50:08 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:50:14.661289706 +0000 UTC m=+67.093237985" watchObservedRunningTime="2026-10-30 21:50:14.662588238 +0000 UTC m=+67.094536506"
Oct 30 21:50:16.549823 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:16.549784    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-vmrkz" event={"ID":"52eb01f8-e86a-4458-8875-639394a87b30","Type":"ContainerStarted","Data":"6d01a17315c626f583b2d7d3e888d56f2c00ebab5d4ed06d2a7e8bf3492ff2ba"}
Oct 30 21:50:16.550325 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:16.549846    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-vmrkz" event={"ID":"52eb01f8-e86a-4458-8875-639394a87b30","Type":"ContainerStarted","Data":"c57caf9fb4036abd3cf7745a13f2a8cbc6269faf3e7ecc65fcc1b9a37616e65f"}
Oct 30 21:50:16.553277 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:16.553225    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-bb95b88bf-g9f94" event={"ID":"0e0d17a1-7742-42b0-9d9b-740bb945e454","Type":"ContainerStarted","Data":"469b0659c63ae3d8d22744da3ffb912c0ae5a72512eb49b97218ed97fdca189a"}
Oct 30 21:50:16.553426 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:16.553312    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-bb95b88bf-g9f94" event={"ID":"0e0d17a1-7742-42b0-9d9b-740bb945e454","Type":"ContainerStarted","Data":"e67a81eda45e06aafd5d8c68c0fd57f185ad5719f95198a24cbb5b14fdb0a3c4"}
Oct 30 21:50:16.666570 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:16.666473    2576 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-vmrkz" podStartSLOduration=66.912885514 podStartE2EDuration="1m8.666450393s" totalImagesPullingTime="1.753564879s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:49:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:50:16.662477318 +0000 UTC m=+69.094425586" watchObservedRunningTime="2026-10-30 21:50:16.666450393 +0000 UTC m=+69.098398655"
Oct 30 21:50:16.789447 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:16.789373    2576 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-bb95b88bf-g9f94" podStartSLOduration=2.659801583 podStartE2EDuration="6.789355677s" totalImagesPullingTime="4.129554094s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:50:10 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:50:16.787291352 +0000 UTC m=+69.219239619" watchObservedRunningTime="2026-10-30 21:50:16.789355677 +0000 UTC m=+69.221303947"
Oct 30 21:50:17.704497 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:17.704454    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:17.934648 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:17.934610    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:18.563153 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:18.563115    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-jmbng" event={"ID":"73c208eb-ffc0-46f2-8bd4-05ade5adb529","Type":"ContainerStarted","Data":"33792ea768f3d4f3c5f7bef6ba5a8d252ea7dbb846490dc63f3cb72e04e23764"}
Oct 30 21:50:18.563388 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:18.563367    2576 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-jmbng"
Oct 30 21:50:18.675466 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:18.675411    2576 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-jmbng" podStartSLOduration=66.754044774 podStartE2EDuration="1m10.675395353s" totalImagesPullingTime="3.921350579s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:49:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:50:18.674415926 +0000 UTC m=+71.106364196" watchObservedRunningTime="2026-10-30 21:50:18.675395353 +0000 UTC m=+71.107343621"
Oct 30 21:50:19.212161 ip-10-0-136-210 kubenswrapper[2576]: I1030 21:50:19.212131    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:19.386710 ip-10-0-136-210 systemd[1]: Stopping Kubernetes Kubelet...
Oct 30 21:50:19.397100 ip-10-0-136-210 systemd[1]: kubelet.service: Deactivated successfully.
Oct 30 21:50:19.397274 ip-10-0-136-210 systemd[1]: Stopped Kubernetes Kubelet.
Oct 30 21:50:19.397324 ip-10-0-136-210 systemd[1]: kubelet.service: Consumed 4.100s CPU time.
Oct 30 21:50:19.407467 ip-10-0-136-210 systemd[1]: Starting Kubernetes Kubelet...
Oct 30 21:50:19.469094 ip-10-0-136-210 kubenswrapper[5510]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 21:50:19.469094 ip-10-0-136-210 kubenswrapper[5510]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Oct 30 21:50:19.469094 ip-10-0-136-210 kubenswrapper[5510]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 21:50:19.469094 ip-10-0-136-210 kubenswrapper[5510]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 21:50:19.471141 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.471052    5510 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:50:19.471141 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.471138    5510 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:50:19.471141 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.471142    5510 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:50:19.471141 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.471145    5510 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:50:19.471141 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.471148    5510 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:50:19.471372 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.471151    5510 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:50:19.471372 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.471154    5510 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:50:19.471372 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.471157    5510 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:50:19.471372 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.471159    5510 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:50:19.471372 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.471162    5510 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:50:19.471372 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.471164    5510 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:50:19.471372 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.471168    5510 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:50:19.471372 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.471171    5510 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:50:19.471372 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.471178    5510 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:50:19.471372 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.471181    5510 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:50:19.471372 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.471184    5510 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:50:19.471372 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.471186    5510 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:50:19.471372 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.471189    5510 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:50:19.471372 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.471192    5510 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:50:19.471372 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.471195    5510 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:50:19.471372 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.471197    5510 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:50:19.471372 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.471200    5510 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:50:19.471372 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.471202    5510 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:50:19.471372 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.471205    5510 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:50:19.472144 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.471208    5510 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:50:19.472144 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.471210    5510 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:50:19.472144 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.471213    5510 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:50:19.472144 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.471216    5510 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:50:19.472144 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.471218    5510 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:50:19.472144 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.471221    5510 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:50:19.472144 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.471224    5510 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:50:19.472144 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.471227    5510 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:50:19.472144 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.471229    5510 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:50:19.472144 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.471232    5510 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:50:19.472144 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.471235    5510 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:50:19.472144 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.471237    5510 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:50:19.472144 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.471240    5510 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:50:19.472144 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.471244    5510 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:50:19.472144 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.471246    5510 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:50:19.472144 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.471249    5510 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:50:19.472144 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.471252    5510 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:50:19.472144 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.471255    5510 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:50:19.472144 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.471257    5510 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:50:19.472144 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.471260    5510 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:50:19.472681 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.471262    5510 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:50:19.472681 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.471266    5510 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:50:19.472681 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.471269    5510 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:50:19.472681 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.471272    5510 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:50:19.472681 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.471276    5510 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:50:19.472681 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.471280    5510 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:50:19.472681 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.471283    5510 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:50:19.472681 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.471286    5510 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:50:19.472681 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.471289    5510 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:50:19.472681 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.471293    5510 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:50:19.472681 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.471295    5510 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:50:19.472681 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.471298    5510 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:50:19.472681 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.471301    5510 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:50:19.472681 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.471305    5510 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:50:19.472681 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.471308    5510 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:50:19.472681 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.471312    5510 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:50:19.472681 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.471314    5510 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:50:19.472681 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.471318    5510 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:50:19.472681 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.471321    5510 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:50:19.473176 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.471323    5510 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:50:19.473176 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.471326    5510 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:50:19.473176 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.471329    5510 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:50:19.473176 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.471332    5510 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:50:19.473176 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.471334    5510 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:50:19.473176 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.471337    5510 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:50:19.473176 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.471339    5510 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:50:19.473176 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.471342    5510 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:50:19.473176 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.471345    5510 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:50:19.473176 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.471350    5510 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:50:19.473176 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.471354    5510 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:50:19.473176 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.471358    5510 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:50:19.473176 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.471362    5510 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:50:19.473176 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.471366    5510 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:50:19.473176 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.471370    5510 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:50:19.473176 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.471374    5510 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:50:19.473176 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.471380    5510 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:50:19.473176 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.471384    5510 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:50:19.473176 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.471388    5510 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:50:19.473176 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.471393    5510 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:50:19.473743 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.471397    5510 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:50:19.473743 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.471401    5510 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:50:19.473743 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.471407    5510 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:50:19.473743 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.471411    5510 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:50:19.473743 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.471415    5510 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:50:19.473743 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.471422    5510 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:50:19.473743 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.471425    5510 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:50:19.473743 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.471428    5510 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:50:19.473743 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.471430    5510 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:50:19.473743 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.471433    5510 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:50:19.473743 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.471436    5510 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:50:19.473743 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.471438    5510 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:50:19.473743 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.471441    5510 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:50:19.473743 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.471443    5510 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:50:19.473743 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.471446    5510 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:50:19.473743 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.471449    5510 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:50:19.473743 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.471452    5510 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:50:19.473743 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.471455    5510 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:50:19.473743 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.471458    5510 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:50:19.473743 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.471460    5510 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:50:19.474246 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.471464    5510 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:50:19.474246 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.471467    5510 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:50:19.474246 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.471470    5510 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:50:19.474246 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.472438    5510 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:50:19.474246 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.472445    5510 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:50:19.474246 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.472448    5510 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:50:19.474246 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.472451    5510 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:50:19.474246 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.472454    5510 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:50:19.474246 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.472457    5510 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:50:19.474246 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.472461    5510 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:50:19.474246 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.472463    5510 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:50:19.474246 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.472467    5510 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:50:19.474246 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.472469    5510 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:50:19.474246 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.472472    5510 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:50:19.474246 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.472475    5510 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:50:19.474246 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.472478    5510 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:50:19.474246 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.472481    5510 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:50:19.474246 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.472485    5510 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:50:19.474246 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.472488    5510 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:50:19.474793 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.472493    5510 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:50:19.474793 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.472496    5510 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:50:19.474793 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.472498    5510 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:50:19.474793 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.472501    5510 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:50:19.474793 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.472503    5510 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:50:19.474793 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.472506    5510 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:50:19.474793 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.472509    5510 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:50:19.474793 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.472512    5510 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:50:19.474793 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.472515    5510 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:50:19.474793 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.472517    5510 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:50:19.474793 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.472520    5510 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:50:19.474793 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.472522    5510 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:50:19.474793 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.472525    5510 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:50:19.474793 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.472528    5510 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:50:19.474793 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.472530    5510 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:50:19.474793 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.472533    5510 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:50:19.474793 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.472536    5510 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:50:19.474793 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.472538    5510 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:50:19.474793 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.472540    5510 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:50:19.474793 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.472543    5510 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:50:19.475285 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.472546    5510 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:50:19.475285 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.472549    5510 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:50:19.475285 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.472551    5510 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:50:19.475285 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.472554    5510 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:50:19.475285 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.472557    5510 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:50:19.475285 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.472560    5510 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:50:19.475285 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.472562    5510 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:50:19.475285 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.472566    5510 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:50:19.475285 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.472569    5510 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:50:19.475285 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.472572    5510 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:50:19.475285 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.472574    5510 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:50:19.475285 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.472577    5510 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:50:19.475285 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.472579    5510 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:50:19.475285 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.472582    5510 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:50:19.475285 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.472585    5510 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:50:19.475285 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.472587    5510 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:50:19.475285 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.472590    5510 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:50:19.475285 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.472592    5510 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:50:19.475285 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.472595    5510 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:50:19.475781 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.472597    5510 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:50:19.475781 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.472600    5510 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:50:19.475781 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.472602    5510 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:50:19.475781 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.472605    5510 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:50:19.475781 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.472609    5510 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:50:19.475781 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.472613    5510 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:50:19.475781 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.472616    5510 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:50:19.475781 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.472618    5510 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:50:19.475781 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.472621    5510 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:50:19.475781 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.472624    5510 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:50:19.475781 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.472627    5510 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:50:19.475781 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.472629    5510 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:50:19.475781 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.472632    5510 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:50:19.475781 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.472635    5510 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:50:19.475781 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.472637    5510 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:50:19.475781 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.472640    5510 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:50:19.475781 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.472642    5510 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:50:19.475781 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.472646    5510 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:50:19.475781 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.472649    5510 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:50:19.476299 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.472666    5510 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:50:19.476299 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.472670    5510 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:50:19.476299 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.472673    5510 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:50:19.476299 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.472676    5510 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:50:19.476299 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.472678    5510 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:50:19.476299 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.472681    5510 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:50:19.476299 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.472684    5510 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:50:19.476299 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.472687    5510 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:50:19.476299 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.472690    5510 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:50:19.476299 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.472692    5510 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:50:19.476299 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.472695    5510 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:50:19.476299 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.472698    5510 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:50:19.476299 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.472700    5510 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:50:19.476299 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.472703    5510 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:50:19.476299 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.472706    5510 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:50:19.476299 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.472709    5510 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:50:19.476299 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.472711    5510 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:50:19.476299 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.472714    5510 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:50:19.476299 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.472717    5510 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:50:19.476299 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.472719    5510 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:50:19.476822 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.472722    5510 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:50:19.476822 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.472724    5510 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:50:19.476822 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.472727    5510 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:50:19.476822 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.472730    5510 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:50:19.476822 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.472733    5510 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:50:19.476822 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.472735    5510 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:50:19.476822 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.472738    5510 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:50:19.476822 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.472741    5510 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:50:19.476822 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.472743    5510 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:50:19.476822 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.472746    5510 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:50:19.476822 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.472750    5510 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:50:19.476822 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.472753    5510 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:50:19.476822 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.473259    5510 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:50:19.476822 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.473265    5510 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:50:19.476822 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.473270    5510 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:50:19.476822 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.473273    5510 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:50:19.476822 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.473276    5510 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:50:19.476822 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.473280    5510 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:50:19.476822 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.473284    5510 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:50:19.477340 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.473287    5510 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:50:19.477340 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.473291    5510 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:50:19.477340 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.473293    5510 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:50:19.477340 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.473296    5510 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:50:19.477340 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.473299    5510 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:50:19.477340 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.473302    5510 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:50:19.477340 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.473304    5510 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:50:19.477340 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.473307    5510 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:50:19.477340 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.473310    5510 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:50:19.477340 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.473313    5510 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:50:19.477340 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.473315    5510 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:50:19.477340 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.473320    5510 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:50:19.477340 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.473323    5510 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:50:19.477340 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.473325    5510 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:50:19.477340 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.473328    5510 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:50:19.477340 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.473332    5510 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:50:19.477340 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.473335    5510 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:50:19.477340 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.473337    5510 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:50:19.477340 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.473340    5510 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:50:19.477844 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.473343    5510 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:50:19.477844 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.473345    5510 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:50:19.477844 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.473348    5510 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:50:19.477844 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.473350    5510 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:50:19.477844 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.473353    5510 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:50:19.477844 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.473356    5510 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:50:19.477844 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.473359    5510 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:50:19.477844 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.473362    5510 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:50:19.477844 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.473365    5510 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:50:19.477844 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.473369    5510 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:50:19.477844 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.473373    5510 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:50:19.477844 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.473376    5510 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:50:19.477844 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.473378    5510 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:50:19.477844 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.473381    5510 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:50:19.477844 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.473384    5510 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:50:19.477844 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.473386    5510 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:50:19.477844 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.473389    5510 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:50:19.477844 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.473392    5510 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:50:19.477844 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.473395    5510 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:50:19.477844 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.473397    5510 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:50:19.478347 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.473400    5510 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:50:19.478347 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.473403    5510 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:50:19.478347 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.473406    5510 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:50:19.478347 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.473408    5510 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:50:19.478347 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.473412    5510 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:50:19.478347 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.473415    5510 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:50:19.478347 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.473417    5510 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:50:19.478347 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.473420    5510 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:50:19.478347 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.473423    5510 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:50:19.478347 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.473425    5510 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:50:19.478347 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.473428    5510 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:50:19.478347 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.473430    5510 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:50:19.478347 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.473433    5510 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:50:19.478347 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.473436    5510 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:50:19.478347 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.473438    5510 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:50:19.478347 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.473441    5510 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:50:19.478347 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.473444    5510 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:50:19.478347 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.473446    5510 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:50:19.478347 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.473455    5510 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:50:19.478845 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.473460    5510 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:50:19.478845 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.473462    5510 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:50:19.478845 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.473465    5510 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:50:19.478845 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.473468    5510 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:50:19.478845 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.473471    5510 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:50:19.478845 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.473474    5510 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:50:19.478845 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.473476    5510 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:50:19.478845 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.473479    5510 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:50:19.478845 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.473481    5510 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:50:19.478845 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.473484    5510 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:50:19.478845 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.473487    5510 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:50:19.478845 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.473489    5510 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:50:19.478845 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.473492    5510 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:50:19.478845 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.473495    5510 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:50:19.478845 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.473497    5510 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:50:19.478845 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.473500    5510 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:50:19.478845 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.473502    5510 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:50:19.478845 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.473506    5510 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:50:19.478845 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.473509    5510 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:50:19.478845 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.473512    5510 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:50:19.479448 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.473515    5510 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:50:19.479448 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.473517    5510 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:50:19.479448 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.473520    5510 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:50:19.479448 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.473523    5510 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:50:19.479448 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.473525    5510 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:50:19.479448 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.473528    5510 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:50:19.479448 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.473531    5510 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:50:19.479448 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.473533    5510 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:50:19.479448 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.473536    5510 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:50:19.479448 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.473538    5510 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:50:19.479448 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.473541    5510 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:50:19.479448 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.473544    5510 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:50:19.479448 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.473548    5510 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:50:19.479448 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.473551    5510 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:50:19.479448 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.473554    5510 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:50:19.479448 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.473556    5510 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:50:19.479448 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.473560    5510 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:50:19.479448 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.473563    5510 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:50:19.479448 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.473565    5510 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:50:19.479957 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.473568    5510 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:50:19.479957 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.473570    5510 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:50:19.479957 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.473673    5510 flags.go:64] FLAG: --address="0.0.0.0"
Oct 30 21:50:19.479957 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.473681    5510 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Oct 30 21:50:19.479957 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.473688    5510 flags.go:64] FLAG: --anonymous-auth="true"
Oct 30 21:50:19.479957 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.473692    5510 flags.go:64] FLAG: --application-metrics-count-limit="100"
Oct 30 21:50:19.479957 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.473697    5510 flags.go:64] FLAG: --authentication-token-webhook="false"
Oct 30 21:50:19.479957 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.473700    5510 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Oct 30 21:50:19.479957 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.473705    5510 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Oct 30 21:50:19.479957 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.473709    5510 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Oct 30 21:50:19.479957 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.473714    5510 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Oct 30 21:50:19.479957 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.473717    5510 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Oct 30 21:50:19.479957 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.473721    5510 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Oct 30 21:50:19.479957 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.473724    5510 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Oct 30 21:50:19.479957 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.473727    5510 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Oct 30 21:50:19.479957 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.473731    5510 flags.go:64] FLAG: --cgroup-root=""
Oct 30 21:50:19.479957 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.473734    5510 flags.go:64] FLAG: --cgroups-per-qos="true"
Oct 30 21:50:19.479957 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.473737    5510 flags.go:64] FLAG: --client-ca-file=""
Oct 30 21:50:19.479957 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.473740    5510 flags.go:64] FLAG: --cloud-provider="external"
Oct 30 21:50:19.479957 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.473743    5510 flags.go:64] FLAG: --cluster-dns="[]"
Oct 30 21:50:19.479957 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.473748    5510 flags.go:64] FLAG: --cluster-domain=""
Oct 30 21:50:19.479957 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.473750    5510 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Oct 30 21:50:19.479957 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.473754    5510 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Oct 30 21:50:19.480521 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.473757    5510 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Oct 30 21:50:19.480521 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.473761    5510 flags.go:64] FLAG: --container-log-max-files="5"
Oct 30 21:50:19.480521 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.473765    5510 flags.go:64] FLAG: --container-log-max-size="10Mi"
Oct 30 21:50:19.480521 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.473769    5510 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Oct 30 21:50:19.480521 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.473772    5510 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Oct 30 21:50:19.480521 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.473776    5510 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Oct 30 21:50:19.480521 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.473779    5510 flags.go:64] FLAG: --contention-profiling="false"
Oct 30 21:50:19.480521 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.473782    5510 flags.go:64] FLAG: --cpu-cfs-quota="true"
Oct 30 21:50:19.480521 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.473785    5510 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Oct 30 21:50:19.480521 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.473788    5510 flags.go:64] FLAG: --cpu-manager-policy="none"
Oct 30 21:50:19.480521 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.473791    5510 flags.go:64] FLAG: --cpu-manager-policy-options=""
Oct 30 21:50:19.480521 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.473795    5510 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Oct 30 21:50:19.480521 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.473799    5510 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Oct 30 21:50:19.480521 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.473802    5510 flags.go:64] FLAG: --enable-debugging-handlers="true"
Oct 30 21:50:19.480521 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.473805    5510 flags.go:64] FLAG: --enable-load-reader="false"
Oct 30 21:50:19.480521 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.473808    5510 flags.go:64] FLAG: --enable-server="true"
Oct 30 21:50:19.480521 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.473811    5510 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Oct 30 21:50:19.480521 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.473815    5510 flags.go:64] FLAG: --event-burst="100"
Oct 30 21:50:19.480521 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.473818    5510 flags.go:64] FLAG: --event-qps="50"
Oct 30 21:50:19.480521 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.473823    5510 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Oct 30 21:50:19.480521 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.473826    5510 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Oct 30 21:50:19.480521 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.473829    5510 flags.go:64] FLAG: --eviction-hard=""
Oct 30 21:50:19.480521 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.473833    5510 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Oct 30 21:50:19.480521 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.473836    5510 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Oct 30 21:50:19.481149 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.473839    5510 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Oct 30 21:50:19.481149 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.473843    5510 flags.go:64] FLAG: --eviction-soft=""
Oct 30 21:50:19.481149 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.473846    5510 flags.go:64] FLAG: --eviction-soft-grace-period=""
Oct 30 21:50:19.481149 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.473849    5510 flags.go:64] FLAG: --exit-on-lock-contention="false"
Oct 30 21:50:19.481149 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.473852    5510 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Oct 30 21:50:19.481149 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.473855    5510 flags.go:64] FLAG: --experimental-mounter-path=""
Oct 30 21:50:19.481149 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.473858    5510 flags.go:64] FLAG: --fail-cgroupv1="true"
Oct 30 21:50:19.481149 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.473861    5510 flags.go:64] FLAG: --fail-swap-on="true"
Oct 30 21:50:19.481149 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.473864    5510 flags.go:64] FLAG: --feature-gates=""
Oct 30 21:50:19.481149 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.473868    5510 flags.go:64] FLAG: --file-check-frequency="20s"
Oct 30 21:50:19.481149 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.473871    5510 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Oct 30 21:50:19.481149 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.473874    5510 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Oct 30 21:50:19.481149 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.473878    5510 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Oct 30 21:50:19.481149 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.473881    5510 flags.go:64] FLAG: --healthz-port="10248"
Oct 30 21:50:19.481149 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.473885    5510 flags.go:64] FLAG: --help="false"
Oct 30 21:50:19.481149 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.473888    5510 flags.go:64] FLAG: --hostname-override="ip-10-0-136-210.us-west-2.compute.internal"
Oct 30 21:50:19.481149 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.473891    5510 flags.go:64] FLAG: --housekeeping-interval="10s"
Oct 30 21:50:19.481149 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.473894    5510 flags.go:64] FLAG: --http-check-frequency="20s"
Oct 30 21:50:19.481149 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.473897    5510 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Oct 30 21:50:19.481149 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.473901    5510 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Oct 30 21:50:19.481149 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.473904    5510 flags.go:64] FLAG: --image-gc-high-threshold="85"
Oct 30 21:50:19.481149 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.473907    5510 flags.go:64] FLAG: --image-gc-low-threshold="80"
Oct 30 21:50:19.481149 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.473910    5510 flags.go:64] FLAG: --image-service-endpoint=""
Oct 30 21:50:19.481779 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.473914    5510 flags.go:64] FLAG: --kernel-memcg-notification="false"
Oct 30 21:50:19.481779 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.473917    5510 flags.go:64] FLAG: --kube-api-burst="100"
Oct 30 21:50:19.481779 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.473920    5510 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Oct 30 21:50:19.481779 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.473923    5510 flags.go:64] FLAG: --kube-api-qps="50"
Oct 30 21:50:19.481779 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.473927    5510 flags.go:64] FLAG: --kube-reserved=""
Oct 30 21:50:19.481779 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.473930    5510 flags.go:64] FLAG: --kube-reserved-cgroup=""
Oct 30 21:50:19.481779 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.473933    5510 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Oct 30 21:50:19.481779 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.473937    5510 flags.go:64] FLAG: --kubelet-cgroups=""
Oct 30 21:50:19.481779 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.473940    5510 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Oct 30 21:50:19.481779 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.473943    5510 flags.go:64] FLAG: --lock-file=""
Oct 30 21:50:19.481779 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.473946    5510 flags.go:64] FLAG: --log-cadvisor-usage="false"
Oct 30 21:50:19.481779 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.473949    5510 flags.go:64] FLAG: --log-flush-frequency="5s"
Oct 30 21:50:19.481779 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.473952    5510 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Oct 30 21:50:19.481779 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.473958    5510 flags.go:64] FLAG: --log-json-split-stream="false"
Oct 30 21:50:19.481779 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.473961    5510 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Oct 30 21:50:19.481779 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.473964    5510 flags.go:64] FLAG: --log-text-split-stream="false"
Oct 30 21:50:19.481779 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.473967    5510 flags.go:64] FLAG: --logging-format="text"
Oct 30 21:50:19.481779 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.473970    5510 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Oct 30 21:50:19.481779 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.473973    5510 flags.go:64] FLAG: --make-iptables-util-chains="true"
Oct 30 21:50:19.481779 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.473976    5510 flags.go:64] FLAG: --manifest-url=""
Oct 30 21:50:19.481779 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.473980    5510 flags.go:64] FLAG: --manifest-url-header=""
Oct 30 21:50:19.481779 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.473984    5510 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Oct 30 21:50:19.481779 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.473988    5510 flags.go:64] FLAG: --max-open-files="1000000"
Oct 30 21:50:19.481779 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.473995    5510 flags.go:64] FLAG: --max-pods="110"
Oct 30 21:50:19.481779 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.473998    5510 flags.go:64] FLAG: --maximum-dead-containers="-1"
Oct 30 21:50:19.482444 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.474001    5510 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Oct 30 21:50:19.482444 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.474004    5510 flags.go:64] FLAG: --memory-manager-policy="None"
Oct 30 21:50:19.482444 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.474007    5510 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Oct 30 21:50:19.482444 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.474010    5510 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Oct 30 21:50:19.482444 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.474013    5510 flags.go:64] FLAG: --node-ip="0.0.0.0"
Oct 30 21:50:19.482444 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.474016    5510 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Oct 30 21:50:19.482444 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.474023    5510 flags.go:64] FLAG: --node-status-max-images="50"
Oct 30 21:50:19.482444 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.474026    5510 flags.go:64] FLAG: --node-status-update-frequency="10s"
Oct 30 21:50:19.482444 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.474029    5510 flags.go:64] FLAG: --oom-score-adj="-999"
Oct 30 21:50:19.482444 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.474033    5510 flags.go:64] FLAG: --pod-cidr=""
Oct 30 21:50:19.482444 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.474036    5510 flags.go:64] FLAG: --pod-manifest-path=""
Oct 30 21:50:19.482444 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.474040    5510 flags.go:64] FLAG: --pod-max-pids="-1"
Oct 30 21:50:19.482444 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.474043    5510 flags.go:64] FLAG: --pods-per-core="0"
Oct 30 21:50:19.482444 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.474047    5510 flags.go:64] FLAG: --port="10250"
Oct 30 21:50:19.482444 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.474050    5510 flags.go:64] FLAG: --protect-kernel-defaults="false"
Oct 30 21:50:19.482444 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.474053    5510 flags.go:64] FLAG: --provider-id="aws:///us-west-2a/i-0d3218457a2c9bb70"
Oct 30 21:50:19.482444 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.474056    5510 flags.go:64] FLAG: --qos-reserved=""
Oct 30 21:50:19.482444 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.474059    5510 flags.go:64] FLAG: --read-only-port="10255"
Oct 30 21:50:19.482444 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.474062    5510 flags.go:64] FLAG: --register-node="true"
Oct 30 21:50:19.482444 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.474065    5510 flags.go:64] FLAG: --register-with-taints=""
Oct 30 21:50:19.482444 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.474069    5510 flags.go:64] FLAG: --registry-burst="10"
Oct 30 21:50:19.482444 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.474072    5510 flags.go:64] FLAG: --registry-qps="5"
Oct 30 21:50:19.482444 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.474075    5510 flags.go:64] FLAG: --reserved-cpus=""
Oct 30 21:50:19.482444 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.474078    5510 flags.go:64] FLAG: --reserved-memory=""
Oct 30 21:50:19.482444 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.474082    5510 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Oct 30 21:50:19.482444 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.474085    5510 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Oct 30 21:50:19.483521 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.474088    5510 flags.go:64] FLAG: --rotate-certificates="false"
Oct 30 21:50:19.483521 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.474091    5510 flags.go:64] FLAG: --rotate-server-certificates="false"
Oct 30 21:50:19.483521 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.474094    5510 flags.go:64] FLAG: --runonce="false"
Oct 30 21:50:19.483521 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.474097    5510 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Oct 30 21:50:19.483521 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.474101    5510 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Oct 30 21:50:19.483521 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.474104    5510 flags.go:64] FLAG: --seccomp-default="false"
Oct 30 21:50:19.483521 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.474107    5510 flags.go:64] FLAG: --serialize-image-pulls="true"
Oct 30 21:50:19.483521 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.474110    5510 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Oct 30 21:50:19.483521 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.474113    5510 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Oct 30 21:50:19.483521 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.474116    5510 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Oct 30 21:50:19.483521 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.474119    5510 flags.go:64] FLAG: --storage-driver-password="root"
Oct 30 21:50:19.483521 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.474122    5510 flags.go:64] FLAG: --storage-driver-secure="false"
Oct 30 21:50:19.483521 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.474125    5510 flags.go:64] FLAG: --storage-driver-table="stats"
Oct 30 21:50:19.483521 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.474128    5510 flags.go:64] FLAG: --storage-driver-user="root"
Oct 30 21:50:19.483521 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.474131    5510 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Oct 30 21:50:19.483521 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.474136    5510 flags.go:64] FLAG: --sync-frequency="1m0s"
Oct 30 21:50:19.483521 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.474139    5510 flags.go:64] FLAG: --system-cgroups=""
Oct 30 21:50:19.483521 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.474143    5510 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Oct 30 21:50:19.483521 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.474149    5510 flags.go:64] FLAG: --system-reserved-cgroup=""
Oct 30 21:50:19.483521 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.474152    5510 flags.go:64] FLAG: --tls-cert-file=""
Oct 30 21:50:19.483521 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.474155    5510 flags.go:64] FLAG: --tls-cipher-suites="[]"
Oct 30 21:50:19.483521 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.474160    5510 flags.go:64] FLAG: --tls-min-version=""
Oct 30 21:50:19.483521 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.474163    5510 flags.go:64] FLAG: --tls-private-key-file=""
Oct 30 21:50:19.483521 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.474166    5510 flags.go:64] FLAG: --topology-manager-policy="none"
Oct 30 21:50:19.483521 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.474169    5510 flags.go:64] FLAG: --topology-manager-policy-options=""
Oct 30 21:50:19.484317 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.474172    5510 flags.go:64] FLAG: --topology-manager-scope="container"
Oct 30 21:50:19.484317 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.474175    5510 flags.go:64] FLAG: --v="2"
Oct 30 21:50:19.484317 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.474179    5510 flags.go:64] FLAG: --version="false"
Oct 30 21:50:19.484317 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.474183    5510 flags.go:64] FLAG: --vmodule=""
Oct 30 21:50:19.484317 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.474187    5510 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Oct 30 21:50:19.484317 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.474191    5510 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Oct 30 21:50:19.484317 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474332    5510 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:50:19.484317 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474337    5510 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:50:19.484317 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474340    5510 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:50:19.484317 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474343    5510 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:50:19.484317 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474346    5510 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:50:19.484317 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474348    5510 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:50:19.484317 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474352    5510 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:50:19.484317 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474354    5510 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:50:19.484317 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474357    5510 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:50:19.484317 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474359    5510 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:50:19.484317 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474362    5510 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:50:19.484317 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474365    5510 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:50:19.484317 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474368    5510 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:50:19.484317 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474370    5510 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:50:19.484317 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474373    5510 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:50:19.484859 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474376    5510 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:50:19.484859 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474380    5510 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:50:19.484859 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474382    5510 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:50:19.484859 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474386    5510 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:50:19.484859 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474388    5510 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:50:19.484859 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474391    5510 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:50:19.484859 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474394    5510 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:50:19.484859 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474398    5510 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:50:19.484859 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474401    5510 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:50:19.484859 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474404    5510 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:50:19.484859 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474407    5510 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:50:19.484859 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474410    5510 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:50:19.484859 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474412    5510 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:50:19.484859 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474415    5510 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:50:19.484859 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474417    5510 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:50:19.484859 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474420    5510 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:50:19.484859 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474423    5510 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:50:19.484859 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474426    5510 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:50:19.484859 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474429    5510 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:50:19.484859 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474431    5510 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:50:19.485358 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474434    5510 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:50:19.485358 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474436    5510 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:50:19.485358 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474439    5510 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:50:19.485358 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474441    5510 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:50:19.485358 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474445    5510 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:50:19.485358 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474447    5510 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:50:19.485358 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474450    5510 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:50:19.485358 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474452    5510 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:50:19.485358 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474455    5510 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:50:19.485358 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474458    5510 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:50:19.485358 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474460    5510 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:50:19.485358 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474463    5510 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:50:19.485358 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474465    5510 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:50:19.485358 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474469    5510 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:50:19.485358 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474473    5510 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:50:19.485358 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474477    5510 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:50:19.485358 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474481    5510 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:50:19.485358 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474483    5510 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:50:19.485358 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474487    5510 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:50:19.485946 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474490    5510 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:50:19.485946 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474493    5510 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:50:19.485946 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474496    5510 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:50:19.485946 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474499    5510 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:50:19.485946 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474501    5510 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:50:19.485946 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474504    5510 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:50:19.485946 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474507    5510 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:50:19.485946 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474509    5510 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:50:19.485946 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474512    5510 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:50:19.485946 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474514    5510 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:50:19.485946 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474517    5510 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:50:19.485946 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474520    5510 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:50:19.485946 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474522    5510 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:50:19.485946 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474525    5510 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:50:19.485946 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474528    5510 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:50:19.485946 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474530    5510 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:50:19.485946 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474533    5510 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:50:19.485946 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474535    5510 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:50:19.485946 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474539    5510 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:50:19.486505 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474542    5510 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:50:19.486505 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474544    5510 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:50:19.486505 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474547    5510 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:50:19.486505 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474549    5510 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:50:19.486505 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474552    5510 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:50:19.486505 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474554    5510 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:50:19.486505 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474557    5510 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:50:19.486505 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474561    5510 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:50:19.486505 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474564    5510 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:50:19.486505 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474566    5510 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:50:19.486505 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474569    5510 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:50:19.486505 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474572    5510 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:50:19.486505 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474575    5510 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:50:19.486505 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474577    5510 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:50:19.486505 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474580    5510 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:50:19.486505 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474583    5510 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:50:19.486505 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474585    5510 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:50:19.486505 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474588    5510 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:50:19.486505 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474591    5510 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:50:19.487469 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474593    5510 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:50:19.487469 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474596    5510 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:50:19.487469 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474598    5510 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:50:19.487469 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474601    5510 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:50:19.487469 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474604    5510 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:50:19.487469 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474606    5510 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:50:19.487469 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474609    5510 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:50:19.487469 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474611    5510 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:50:19.487469 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474614    5510 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:50:19.487469 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474617    5510 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:50:19.487469 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474619    5510 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:50:19.487469 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474622    5510 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:50:19.487469 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474624    5510 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:50:19.487469 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474627    5510 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:50:19.487469 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.474676    5510 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 21:50:19.487469 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474737    5510 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:50:19.488224 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474742    5510 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:50:19.488224 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474746    5510 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:50:19.488224 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474749    5510 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:50:19.488224 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474752    5510 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:50:19.488224 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474758    5510 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:50:19.488224 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474761    5510 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:50:19.488224 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474763    5510 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:50:19.488224 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474766    5510 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:50:19.488224 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474769    5510 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:50:19.488224 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474772    5510 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:50:19.488224 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474775    5510 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:50:19.488224 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474777    5510 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:50:19.488224 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474780    5510 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:50:19.488224 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474782    5510 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:50:19.488224 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474785    5510 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:50:19.488224 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474787    5510 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:50:19.488224 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474790    5510 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:50:19.488224 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474793    5510 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:50:19.488224 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474795    5510 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:50:19.488925 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474798    5510 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:50:19.488925 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474801    5510 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:50:19.488925 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474804    5510 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:50:19.488925 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474807    5510 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:50:19.488925 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474809    5510 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:50:19.488925 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474812    5510 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:50:19.488925 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474814    5510 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:50:19.488925 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474817    5510 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:50:19.488925 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474819    5510 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:50:19.488925 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474822    5510 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:50:19.488925 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474825    5510 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:50:19.488925 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474828    5510 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:50:19.488925 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474830    5510 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:50:19.488925 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474833    5510 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:50:19.488925 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474835    5510 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:50:19.488925 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474838    5510 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:50:19.488925 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474841    5510 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:50:19.488925 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474846    5510 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:50:19.488925 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474849    5510 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:50:19.488925 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474851    5510 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:50:19.489458 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474854    5510 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:50:19.489458 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474857    5510 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:50:19.489458 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474860    5510 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:50:19.489458 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474863    5510 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:50:19.489458 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474867    5510 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:50:19.489458 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474870    5510 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:50:19.489458 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474873    5510 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:50:19.489458 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474876    5510 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:50:19.489458 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474878    5510 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:50:19.489458 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474881    5510 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:50:19.489458 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474884    5510 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:50:19.489458 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474886    5510 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:50:19.489458 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474889    5510 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:50:19.489458 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474892    5510 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:50:19.489458 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474895    5510 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:50:19.489458 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474897    5510 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:50:19.489458 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474900    5510 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:50:19.489458 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474902    5510 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:50:19.489458 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474905    5510 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:50:19.489458 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474907    5510 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:50:19.490007 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474910    5510 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:50:19.490007 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474913    5510 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:50:19.490007 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474915    5510 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:50:19.490007 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474918    5510 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:50:19.490007 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474921    5510 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:50:19.490007 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474924    5510 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:50:19.490007 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474927    5510 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:50:19.490007 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474930    5510 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:50:19.490007 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474932    5510 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:50:19.490007 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474939    5510 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:50:19.490007 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474943    5510 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:50:19.490007 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474947    5510 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:50:19.490007 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474950    5510 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:50:19.490007 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474953    5510 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:50:19.490007 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474956    5510 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:50:19.490007 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474959    5510 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:50:19.490007 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474962    5510 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:50:19.490007 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474964    5510 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:50:19.490007 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474967    5510 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:50:19.490473 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474970    5510 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:50:19.490473 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474972    5510 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:50:19.490473 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474975    5510 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:50:19.490473 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474978    5510 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:50:19.490473 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474980    5510 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:50:19.490473 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474983    5510 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:50:19.490473 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474985    5510 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:50:19.490473 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474988    5510 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:50:19.490473 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474991    5510 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:50:19.490473 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474994    5510 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:50:19.490473 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474996    5510 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:50:19.490473 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.474999    5510 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:50:19.490473 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.475001    5510 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:50:19.490473 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.475004    5510 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:50:19.490473 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.475007    5510 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:50:19.490473 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.475009    5510 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:50:19.490473 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.475012    5510 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:50:19.490473 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.475014    5510 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:50:19.490473 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.475017    5510 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:50:19.490959 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.475020    5510 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:50:19.490959 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.475023    5510 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:50:19.490959 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.475025    5510 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:50:19.490959 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.475032    5510 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:50:19.490959 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.475036    5510 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:50:19.490959 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.475039    5510 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:50:19.490959 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.475042    5510 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:50:19.490959 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.475044    5510 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:50:19.490959 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.479031    5510 server.go:525] "Kubelet version" kubeletVersion="v1.35.9"
Oct 30 21:50:19.490959 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.479045    5510 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Oct 30 21:50:19.490959 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.479064    5510 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Oct 30 21:50:19.490959 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.479068    5510 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Oct 30 21:50:19.490959 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479141    5510 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:50:19.490959 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479147    5510 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:50:19.490959 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479150    5510 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:50:19.490959 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479155    5510 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:50:19.490959 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479158    5510 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:50:19.490959 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479161    5510 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:50:19.490959 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479164    5510 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:50:19.491423 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479167    5510 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:50:19.491423 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479170    5510 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:50:19.491423 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479173    5510 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:50:19.491423 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479175    5510 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:50:19.491423 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479178    5510 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:50:19.491423 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479181    5510 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:50:19.491423 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479183    5510 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:50:19.491423 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479186    5510 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:50:19.491423 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479189    5510 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:50:19.491423 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479191    5510 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:50:19.491423 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479194    5510 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:50:19.491423 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479197    5510 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:50:19.491423 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479205    5510 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:50:19.491423 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479208    5510 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:50:19.491423 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479211    5510 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:50:19.491423 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479214    5510 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:50:19.491423 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479217    5510 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:50:19.491423 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479220    5510 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:50:19.491423 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479222    5510 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:50:19.491423 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479225    5510 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:50:19.491928 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479227    5510 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:50:19.491928 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479230    5510 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:50:19.491928 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479233    5510 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:50:19.491928 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479236    5510 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:50:19.491928 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479238    5510 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:50:19.491928 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479241    5510 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:50:19.491928 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479244    5510 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:50:19.491928 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479247    5510 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:50:19.491928 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479249    5510 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:50:19.491928 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479252    5510 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:50:19.491928 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479254    5510 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:50:19.491928 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479263    5510 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:50:19.491928 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479266    5510 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:50:19.491928 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479268    5510 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:50:19.491928 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479271    5510 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:50:19.491928 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479273    5510 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:50:19.491928 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479276    5510 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:50:19.491928 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479278    5510 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:50:19.491928 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479281    5510 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:50:19.491928 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479284    5510 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:50:19.492411 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479286    5510 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:50:19.492411 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479289    5510 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:50:19.492411 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479291    5510 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:50:19.492411 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479294    5510 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:50:19.492411 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479296    5510 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:50:19.492411 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479305    5510 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:50:19.492411 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479310    5510 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:50:19.492411 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479314    5510 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:50:19.492411 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479318    5510 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:50:19.492411 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479321    5510 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:50:19.492411 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479324    5510 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:50:19.492411 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479326    5510 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:50:19.492411 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479329    5510 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:50:19.492411 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479332    5510 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:50:19.492411 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479335    5510 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:50:19.492411 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479338    5510 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:50:19.492411 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479340    5510 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:50:19.492411 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479343    5510 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:50:19.492411 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479346    5510 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:50:19.492916 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479348    5510 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:50:19.492916 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479351    5510 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:50:19.492916 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479354    5510 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:50:19.492916 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479356    5510 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:50:19.492916 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479359    5510 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:50:19.492916 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479362    5510 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:50:19.492916 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479365    5510 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:50:19.492916 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479367    5510 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:50:19.492916 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479370    5510 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:50:19.492916 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479373    5510 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:50:19.492916 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479375    5510 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:50:19.492916 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479378    5510 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:50:19.492916 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479380    5510 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:50:19.492916 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479383    5510 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:50:19.492916 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479386    5510 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:50:19.492916 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479388    5510 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:50:19.492916 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479391    5510 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:50:19.492916 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479393    5510 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:50:19.492916 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479396    5510 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:50:19.492916 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479405    5510 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:50:19.493412 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479408    5510 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:50:19.493412 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479411    5510 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:50:19.493412 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479413    5510 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:50:19.493412 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479416    5510 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:50:19.493412 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479419    5510 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:50:19.493412 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479423    5510 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:50:19.493412 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479426    5510 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:50:19.493412 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479430    5510 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:50:19.493412 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479433    5510 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:50:19.493412 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479436    5510 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:50:19.493412 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479439    5510 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:50:19.493412 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479442    5510 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:50:19.493412 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479444    5510 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:50:19.493412 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479447    5510 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:50:19.493412 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479450    5510 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:50:19.493412 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479453    5510 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:50:19.493412 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479456    5510 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:50:19.493412 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479458    5510 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:50:19.493412 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479462    5510 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:50:19.493884 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479464    5510 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:50:19.493884 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.479491    5510 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 21:50:19.493884 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479694    5510 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:50:19.493884 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479700    5510 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:50:19.493884 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479703    5510 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:50:19.493884 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479706    5510 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:50:19.493884 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479709    5510 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:50:19.493884 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479711    5510 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:50:19.493884 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479714    5510 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:50:19.493884 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479716    5510 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:50:19.493884 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479719    5510 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:50:19.493884 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479721    5510 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:50:19.493884 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479724    5510 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:50:19.493884 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479732    5510 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:50:19.493884 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479735    5510 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:50:19.493884 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479738    5510 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:50:19.494273 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479740    5510 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:50:19.494273 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479743    5510 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:50:19.494273 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479745    5510 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:50:19.494273 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479748    5510 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:50:19.494273 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479751    5510 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:50:19.494273 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479754    5510 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:50:19.494273 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479756    5510 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:50:19.494273 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479759    5510 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:50:19.494273 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479761    5510 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:50:19.494273 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479764    5510 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:50:19.494273 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479766    5510 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:50:19.494273 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479770    5510 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:50:19.494273 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479774    5510 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:50:19.494273 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479778    5510 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:50:19.494273 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479780    5510 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:50:19.494273 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479783    5510 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:50:19.494273 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479786    5510 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:50:19.494273 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479789    5510 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:50:19.494273 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479791    5510 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:50:19.494797 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479794    5510 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:50:19.494797 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479796    5510 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:50:19.494797 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479799    5510 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:50:19.494797 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479801    5510 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:50:19.494797 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479804    5510 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:50:19.494797 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479807    5510 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:50:19.494797 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479809    5510 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:50:19.494797 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479812    5510 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:50:19.494797 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479814    5510 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:50:19.494797 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479817    5510 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:50:19.494797 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479819    5510 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:50:19.494797 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479827    5510 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:50:19.494797 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479830    5510 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:50:19.494797 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479833    5510 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:50:19.494797 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479835    5510 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:50:19.494797 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479838    5510 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:50:19.494797 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479840    5510 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:50:19.494797 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479843    5510 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:50:19.494797 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479846    5510 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:50:19.495268 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479848    5510 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:50:19.495268 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479851    5510 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:50:19.495268 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479853    5510 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:50:19.495268 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479856    5510 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:50:19.495268 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479858    5510 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:50:19.495268 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479861    5510 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:50:19.495268 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479864    5510 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:50:19.495268 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479866    5510 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:50:19.495268 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479869    5510 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:50:19.495268 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479871    5510 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:50:19.495268 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479874    5510 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:50:19.495268 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479876    5510 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:50:19.495268 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479879    5510 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:50:19.495268 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479882    5510 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:50:19.495268 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479884    5510 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:50:19.495268 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479887    5510 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:50:19.495268 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479889    5510 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:50:19.495268 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479892    5510 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:50:19.495268 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479894    5510 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:50:19.495836 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479897    5510 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:50:19.495836 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479900    5510 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:50:19.495836 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479902    5510 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:50:19.495836 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479904    5510 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:50:19.495836 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479907    5510 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:50:19.495836 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479910    5510 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:50:19.495836 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479918    5510 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:50:19.495836 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479921    5510 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:50:19.495836 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479923    5510 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:50:19.495836 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479926    5510 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:50:19.495836 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479928    5510 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:50:19.495836 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479931    5510 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:50:19.495836 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479934    5510 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:50:19.495836 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479936    5510 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:50:19.495836 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479939    5510 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:50:19.495836 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479941    5510 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:50:19.495836 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479944    5510 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:50:19.495836 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479947    5510 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:50:19.495836 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479953    5510 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:50:19.495836 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479956    5510 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:50:19.496378 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479958    5510 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:50:19.496378 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479962    5510 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:50:19.496378 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479966    5510 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:50:19.496378 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479969    5510 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:50:19.496378 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479972    5510 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:50:19.496378 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479975    5510 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:50:19.496378 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479978    5510 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:50:19.496378 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479980    5510 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:50:19.496378 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479983    5510 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:50:19.496378 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479986    5510 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:50:19.496378 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479988    5510 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:50:19.496378 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479991    5510 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:50:19.496378 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479994    5510 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:50:19.496378 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479997    5510 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:50:19.496378 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.479999    5510 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:50:19.496378 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.480024    5510 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 21:50:19.496787 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.480084    5510 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:50:19.496787 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.480088    5510 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:50:19.496787 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.480091    5510 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:50:19.496787 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.480101    5510 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:50:19.496787 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.480104    5510 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:50:19.496787 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.480108    5510 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:50:19.496787 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.480111    5510 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:50:19.496787 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.480114    5510 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:50:19.496787 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.480116    5510 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:50:19.496787 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.480119    5510 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:50:19.496787 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.480122    5510 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:50:19.496787 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.480125    5510 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:50:19.496787 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.480128    5510 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:50:19.496787 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.480130    5510 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:50:19.496787 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.480133    5510 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:50:19.496787 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.480137    5510 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:50:19.496787 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.480140    5510 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:50:19.496787 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.480143    5510 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:50:19.496787 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.480145    5510 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:50:19.496787 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.480148    5510 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:50:19.497282 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.480151    5510 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:50:19.497282 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.480153    5510 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:50:19.497282 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.480156    5510 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:50:19.497282 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.480159    5510 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:50:19.497282 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.480161    5510 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:50:19.497282 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.480164    5510 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:50:19.497282 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.480167    5510 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:50:19.497282 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.480170    5510 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:50:19.497282 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.480172    5510 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:50:19.497282 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.480175    5510 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:50:19.497282 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.480177    5510 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:50:19.497282 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.480180    5510 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:50:19.497282 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.480183    5510 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:50:19.497282 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.480185    5510 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:50:19.497282 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.480188    5510 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:50:19.497282 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.480190    5510 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:50:19.497282 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.480198    5510 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:50:19.497282 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.480201    5510 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:50:19.497282 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.480204    5510 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:50:19.497282 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.480208    5510 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:50:19.497788 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.480211    5510 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:50:19.497788 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.480214    5510 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:50:19.497788 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.480216    5510 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:50:19.497788 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.480219    5510 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:50:19.497788 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.480222    5510 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:50:19.497788 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.480224    5510 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:50:19.497788 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.480227    5510 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:50:19.497788 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.480231    5510 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:50:19.497788 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.480242    5510 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:50:19.497788 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.480245    5510 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:50:19.497788 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.480248    5510 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:50:19.497788 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.480251    5510 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:50:19.497788 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.480253    5510 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:50:19.497788 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.480256    5510 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:50:19.497788 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.480259    5510 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:50:19.497788 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.480262    5510 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:50:19.497788 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.480266    5510 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:50:19.497788 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.480269    5510 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:50:19.497788 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.480273    5510 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:50:19.498247 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.480276    5510 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:50:19.498247 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.480279    5510 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:50:19.498247 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.480282    5510 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:50:19.498247 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.480285    5510 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:50:19.498247 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.480287    5510 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:50:19.498247 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.480290    5510 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:50:19.498247 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.480293    5510 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:50:19.498247 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.480295    5510 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:50:19.498247 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.480298    5510 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:50:19.498247 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.480301    5510 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:50:19.498247 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.480310    5510 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:50:19.498247 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.480313    5510 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:50:19.498247 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.480317    5510 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:50:19.498247 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.480319    5510 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:50:19.498247 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.480322    5510 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:50:19.498247 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.480325    5510 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:50:19.498247 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.480328    5510 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:50:19.498247 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.480330    5510 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:50:19.498247 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.480333    5510 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:50:19.498247 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.480335    5510 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:50:19.498792 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.480339    5510 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:50:19.498792 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.480342    5510 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:50:19.498792 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.480345    5510 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:50:19.498792 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.480347    5510 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:50:19.498792 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.480350    5510 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:50:19.498792 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.480352    5510 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:50:19.498792 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.480355    5510 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:50:19.498792 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.480358    5510 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:50:19.498792 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.480360    5510 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:50:19.498792 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.480363    5510 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:50:19.498792 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.480366    5510 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:50:19.498792 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.480368    5510 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:50:19.498792 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.480371    5510 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:50:19.498792 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.480373    5510 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:50:19.498792 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.480376    5510 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:50:19.498792 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.480380    5510 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:50:19.498792 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.480384    5510 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:50:19.498792 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.480387    5510 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:50:19.498792 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.480390    5510 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:50:19.499251 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.480393    5510 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:50:19.499251 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.480395    5510 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:50:19.499251 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.480399    5510 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:50:19.499251 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.480402    5510 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:50:19.499251 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.480410    5510 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:50:19.499251 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.480414    5510 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:50:19.499251 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.480417    5510 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:50:19.499251 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:19.480420    5510 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:50:19.499251 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.480482    5510 server.go:957] "Client rotation is on, will bootstrap in background"
Oct 30 21:50:19.499251 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.481472    5510 bootstrap.go:85] "Current kubeconfig file contents are still valid, no bootstrap necessary"
Oct 30 21:50:19.499251 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.481518    5510 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-client-current.pem"
Oct 30 21:50:19.499251 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.481726    5510 server.go:1014] "Starting client certificate rotation"
Oct 30 21:50:19.499251 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.481913    5510 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Oct 30 21:50:19.499251 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.481965    5510 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-10-29 21:44:08 +0000 UTC" deadline="2028-07-23 20:49:20.48725777 +0000 UTC"
Oct 30 21:50:19.499251 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.482026    5510 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15166h59m1.005233595s"
Oct 30 21:50:19.499251 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.482319    5510 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 30 21:50:19.499641 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.485063    5510 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 30 21:50:19.499641 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.486288    5510 log.go:25] "Validated CRI v1 runtime API"
Oct 30 21:50:19.499641 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.488590    5510 log.go:25] "Validated CRI v1 image API"
Oct 30 21:50:19.499641 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.489325    5510 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Oct 30 21:50:19.499641 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.491345    5510 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 825034c1-cd56-4c9c-ad44-16d53b1e630b:/dev/nvme0n1p4 c7fc0637-553e-4526-a80b-df66fd1465c4:/dev/nvme0n1p3]
Oct 30 21:50:19.499929 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.491365    5510 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/02e263b2978459859989e1cdc818ce3ef741757f7d74157e4fb4f4a83c1c0947/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/02e263b2978459859989e1cdc818ce3ef741757f7d74157e4fb4f4a83c1c0947/userdata/shm major:0 minor:310 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/078dbad40157a479574f1da113133d4990f89749660a3686f1fd0e1ff2882507/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/078dbad40157a479574f1da113133d4990f89749660a3686f1fd0e1ff2882507/userdata/shm major:0 minor:90 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/26d84296d1d451363382e3d3f1fd68416a311f8b1ba5919ac8230687bef7460c/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/26d84296d1d451363382e3d3f1fd68416a311f8b1ba5919ac8230687bef7460c/userdata/shm major:0 minor:78 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/29097bc9c91c0bbb3e5034ccc9cf5e145105cca40a1dfb3dee33c7d2ad65ea31/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/29097bc9c91c0bbb3e5034ccc9cf5e145105cca40a1dfb3dee33c7d2ad65ea31/userdata/shm major:0 minor:68 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/2bdc41beaf5186f2d39137449d5d2a9606e952cfcb48cef9006ad1fab03049da/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/2bdc41beaf5186f2d39137449d5d2a9606e952cfcb48cef9006ad1fab03049da/userdata/shm major:0 minor:82 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/327a89302c3fd96103b398b276678511aec903499e1c71d72134532152be1296/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/327a89302c3fd96103b398b276678511aec903499e1c71d72134532152be1296/userdata/shm major:0 minor:61 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/4d6cb4d22f1a3341a2a3e4f90510ae7fbec361fb13b62997a07cb2f4f9fe864b/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/4d6cb4d22f1a3341a2a3e4f90510ae7fbec361fb13b62997a07cb2f4f9fe864b/userdata/shm major:0 minor:205 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/5791c43c54a24e8372a1e21b3010621359dc2caa4fbdac86975465bbfa40f7b3/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/5791c43c54a24e8372a1e21b3010621359dc2caa4fbdac86975465bbfa40f7b3/userdata/shm major:0 minor:91 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/76daddeb7d1867d1fe0158bb897cbf216243b7ff1593fa67f48849c8d3d8ddab/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/76daddeb7d1867d1fe0158bb897cbf216243b7ff1593fa67f48849c8d3d8ddab/userdata/shm major:0 minor:369 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/7c658481457a80544871f4902bf0c0cb431e33d2e3c54401546755d56b0716b4/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/7c658481457a80544871f4902bf0c0cb431e33d2e3c54401546755d56b0716b4/userdata/shm major:0 minor:262 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/847fb3c0d4eb226ac6a8ad5fdd1e6fb20a842ef516173dfee5213674f9f0d651/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/847fb3c0d4eb226ac6a8ad5fdd1e6fb20a842ef516173dfee5213674f9f0d651/userdata/shm major:0 minor:86 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/c1581eb1a638205af61c045effda2bf3708d1f9f54df9b7a6c6c4d69436f7817/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/c1581eb1a638205af61c045effda2bf3708d1f9f54df9b7a6c6c4d69436f7817/userdata/shm major:0 minor:60 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/cc6e2905d1c15d26a3a375d45ff83a2a3e6b21a41741fe6c57f315c1ea464d58/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/cc6e2905d1c15d26a3a375d45ff83a2a3e6b21a41741fe6c57f315c1ea464d58/userdata/shm major:0 minor:74 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/ce29c22093919e23269e62f2db4fc0b855f665df8a6eb65b9945172a1594cc33/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/ce29c22093919e23269e62f2db4fc0b855f665df8a6eb65b9945172a1594cc33/userdata/shm major:0 minor:366 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/ce973673f940234e7bde5a0d7a85118c6da9bd4ed38d54bac859dc43fe25920b/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/ce973673f940234e7bde5a0d7a85118c6da9bd4ed38d54bac859dc43fe25920b/userdata/shm major:0 minor:98 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/d188c9559e229f137adf7ecb5417f5f9310e2d711a052c0aece44396a292cae9/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/d188c9559e229f137adf7ecb5417f5f9310e2d711a052c0aece44396a292cae9/userdata/shm major:0 minor:323 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/d90f0f601a7259b584b3cceb562a51226b1daf12d6df216a04e38d8680c26a8e/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/d90f0f601a7259b584b3cceb562a51226b1daf12d6df216a04e38d8680c26a8e/userdata/shm major:0 minor:371 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/e706fb1e7061ada9e54ae7abc07fc31473d443d12a0371f11a83d19444e35915/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/e706fb1e7061ada9e54ae7abc07fc31473d443d12a0371f11a83d19444e35915/userdata/shm major:0 minor:316 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/f4065939ad3a362fabd190f4f532a0ba04768adf8730414343ea996a097c52f5/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/f4065939ad3a362fabd190f4f532a0ba04768adf8730414343ea996a097c52f5/userdata/shm major:0 minor:330 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/f45d4e00313976fd643259c26716f35ec4f66325b137a79e47d646a0ddaf4cd0/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/f45d4e00313976fd643259c26716f35ec4f66325b137a79e47d646a0ddaf4cd0/userdata/shm major:0 minor:265 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/fa5a20193e2a7edb9a384afb717ec20cb79c193302573d94ad22f8eecfbb3a16/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/fa5a20193e2a7edb9a384afb717ec20cb79c193302573d94ad22f8eecfbb3a16/userdata/shm major:0 minor:209 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/fbffb78323609cf8dd21b10efbe35374ed8184fc5b3e616fe9f4b87fd69b0936/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/fbffb78323609cf8dd21b10efbe35374ed8184fc5b3e616fe9f4b87fd69b0936/userdata/shm major:0 minor:253 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/ff49e81b45715f352095c16b17b12a8c2cc3ccc92322711b2615d651155d2dbb/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/ff49e81b45715f352095c16b17b12a8c2cc3ccc92322711b2615d651155d2dbb/userdata/shm major:0 minor:207 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/04465eda-e6db-42d3-8d5f-c9be668038a9/volumes/kubernetes.io~projected/kube-api-access-hrxwg:{mountpoint:/var/lib/kubelet/pods/04465eda-e6db-42d3-8d5f-c9be668038a9/volumes/kubernetes.io~projected/kube-api-access-hrxwg major:0 minor:59 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0ab70415-9f1d-4988-a228-df6ed621f0d8/volumes/kubernetes.io~projected/kube-api-access-td6rr:{mountpoint:/var/lib/kubelet/pods/0ab70415-9f1d-4988-a228-df6ed621f0d8/volumes/kubernetes.io~projected/kube-api-access-td6rr major:0 minor:53 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0e0d17a1-7742-42b0-9d9b-740bb945e454/volumes/kubernetes.io~projected/kube-api-access-9gzpz:{mountpoint:/var/lib/kubelet/pods/0e0d17a1-7742-42b0-9d9b-740bb945e454/volumes/kubernetes.io~projected/kube-api-access-9gzpz major:0 minor:329 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0e0d17a1-7742-42b0-9d9b-740bb945e454/volumes/kubernetes.io~secret/federate-client-tls:{mountpoint:/var/lib/kubelet/pods/0e0d17a1-7742-42b0-9d9b-740bb945e454/volumes/kubernetes.io~secret/federate-client-tls major:0 minor:326 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0e0d17a1-7742-42b0-9d9b-740bb945e454/volumes/kubernetes.io~secret/secret-telemeter-client:{mountpoint:/var/lib/kubelet/pods/0e0d17a1-7742-42b0-9d9b-740bb945e454/volumes/kubernetes.io~secret/secret-telemeter-client major:0 minor:328 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0e0d17a1-7742-42b0-9d9b-740bb945e454/volumes/kubernetes.io~secret/secret-telemeter-client-kube-rbac-proxy-config:{mountpoint:/var/lib/kubelet/pods/0e0d17a1-7742-42b0-9d9b-740bb945e454/volumes/kubernetes.io~secret/secret-telemeter-client-kube-rbac-proxy-config major:0 minor:325 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0e0d17a1-7742-42b0-9d9b-740bb945e454/volumes/kubernetes.io~secret/telemeter-client-tls:{mountpoint:/var/lib/kubelet/pods/0e0d17a1-7742-42b0-9d9b-740bb945e454/volumes/kubernetes.io~secret/telemeter-client-tls major:0 minor:327 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/2fe50918-ee11-4d17-b0fe-27c7898912c6/volumes/kubernetes.io~projected/kube-api-access-25ggc:{mountpoint:/var/lib/kubelet/pods/2fe50918-ee11-4d17-b0fe-27c7898912c6/volumes/kubernetes.io~projected/kube-api-access-25ggc major:0 minor:204 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/2fe50918-ee11-4d17-b0fe-27c7898912c6/volumes/kubernetes.io~secret/metrics-tls:{mountpoint:/var/lib/kubelet/pods/2fe50918-ee11-4d17-b0fe-27c7898912c6/volumes/kubernetes.io~secret/metrics-tls major:0 minor:197 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/31a389bb-f36f-4e53-9654-5f4648dae38a/volumes/kubernetes.io~projected/kube-api-access-rb2wf:{mountpoint:/var/lib/kubelet/pods/31a389bb-f36f-4e53-9654-5f4648dae38a/volumes/kubernetes.io~projected/kube-api-access-rb2wf major:0 minor:322 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/31a389bb-f36f-4e53-9654-5f4648dae38a/volumes/kubernetes.io~secret/console-oauth-config:{mountpoint:/var/lib/kubelet/pods/31a389bb-f36f-4e53-9654-5f4648dae38a/volumes/kubernetes.io~secret/console-oauth-config major:0 minor:321 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/31a389bb-f36f-4e53-9654-5f4648dae38a/volumes/kubernetes.io~secret/console-serving-cert:{mountpoint:/var/lib/kubelet/pods/31a389bb-f36f-4e53-9654-5f4648dae38a/volumes/kubernetes.io~secret/console-serving-cert major:0 minor:320 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3abd55eb-d1d8-4fdf-b82c-d35894b652ac/volumes/kubernetes.io~projected/kube-api-access-f2qbm:{mountpoint:/var/lib/kubelet/pods/3abd55eb-d1d8-4fdf-b82c-d35894b652ac/volumes/kubernetes.io~projected/kube-api-access-f2qbm major:0 minor:259 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3abd55eb-d1d8-4fdf-b82c-d35894b652ac/volumes/kubernetes.io~secret/openshift-state-metrics-kube-rbac-proxy-config:{mountpoint:/var/lib/kubelet/pods/3abd55eb-d1d8-4fdf-b82c-d35894b652ac/volumes/kubernetes.io~secret/openshift-state-metrics-kube-rbac-proxy-config major:0 minor:258 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3abd55eb-d1d8-4fdf-b82c-d35894b652ac/volumes/kubernetes.io~secret/openshift-state-metrics-tls:{mountpoint:/var/lib/kubelet/pods/3abd55eb-d1d8-4fdf-b82c-d35894b652ac/volumes/kubernetes.io~secret/openshift-state-metrics-tls major:0 minor:257 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/42d54b8a-4f48-46a7-8a34-1f0f29cd1602/volumes/kubernetes.io~projected/kube-api-access-5c4fj:{mountpoint:/var/lib/kubelet/pods/42d54b8a-4f48-46a7-8a34-1f0f29cd1602/volumes/kubernetes.io~projected/kube-api-access-5c4fj major:0 minor:252 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/42d54b8a-4f48-46a7-8a34-1f0f29cd1602/volumes/kubernetes.io~secret/console-oauth-config:{mountpoint:/var/lib/kubelet/pods/42d54b8a-4f48-46a7-8a34-1f0f29cd1602/volumes/kubernetes.io~secret/console-oauth-config major:0 minor:244 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/42d54b8a-4f48-46a7-8a34-1f0f29cd1602/volumes/kubernetes.io~secret/console-serving-cert:{mountpoint:/var/lib/kubelet/pods/42d54b8a-4f48-46a7-8a34-1f0f29cd1602/volumes/kubernetes.io~secret/console-serving-cert major:0 minor:243 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/52eb01f8-e86a-4458-8875-639394a87b30/volumes/kubernetes.io~projected/kube-api-access-zxwjt:{mountpoint:/var/lib/kubelet/pods/52eb01f8-e86a-4458-8875-639394a87b30/volumes/kubernetes.io~projected/kube-api-access-zxwjt major:0 minor:54 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/52eb01f8-e86a-4458-8875-639394a87b30/volumes/kubernetes.io~secret/metrics-certs:{mountpoint:/var/lib/kubelet/pods/52eb01f8-e86a-4458-8875-639394a87b30/volumes/kubernetes.io~secret/metrics-certs major:0 minor:360 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/73c208eb-ffc0-46f2-8bd4-05ade5adb529/volumes/kubernetes.io~projected/kube-api-access-fcjg4:{mountpoint:/var/lib/kubelet/pods/73c208eb-ffc0-46f2-8bd4-05ade5adb529/volumes/kubernetes.io~projected/kube-api-access-fcjg4 major:0 minor:367 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7d6190d5-6574-4704-97cb-2241398548e3/volumes/kubernetes.io~projected/kube-api-access-ggf4d:{mountpoint:/var/lib/kubelet/pods/7d6190d5-6574-4704-97cb-2241398548e3/volumes/kubernetes.io~projected/kube-api-access-ggf4d major:0 minor:202 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7d6190d5-6574-4704-97cb-2241398548e3/volumes/kubernetes.io~secret/insights-runtime-extractor-tls:{mountpoint:/var/lib/kubelet/pods/7d6190d5-6574-4704-97cb-2241398548e3/volumes/kubernetes.io~secret/insights-runtime-extractor-tls major:0 minor:201 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8883cb69-54c2-437c-a92e-3bc38547027f/volumes/kubernetes.io~projected/kube-api-access-b2929:{mountpoint:/var/lib/kubelet/pods/8883cb69-54c2-437c-a92e-3bc38547027f/volumes/kubernetes.io~projected/kube-api-access-b2929 major:0 minor:315 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8883cb69-54c2-437c-a92e-3bc38547027f/volumes/kubernetes.io~secret/monitoring-plugin-cert:{mountpoint:/var/lib/kubelet/pods/8883cb69-54c2-437c-a92e-3bc38547027f/volumes/kubernetes.io~secret/monitoring-plugin-cert major:0 minor:314 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8c88b730-5cf1-4434-950f-4a3aa8f739cd/volumes/kubernetes.io~projected/kube-api-access-ttgkm:{mountpoint:/var/lib/kubelet/pods/8c88b730-5cf1-4434-950f-4a3aa8f739cd/volumes/kubernetes.io~projected/kube-api-access-ttgkm major:0 minor:309 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8c88b730-5cf1-4434-950f-4a3aa8f739cd/volumes/kubernetes.io~secret/secret-grpc-tls:{mountpoint:/var/lib/kubelet/pods/8c88b730-5cf1-4434-950f-4a3aa8f739cd/volumes/kubernetes.io~secret/secret-grpc-tls major:0 minor:306 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8c88b730-5cf1-4434-950f-4a3aa8f739cd/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy:{mountpoint:/var/lib/kubelet/pods/8c88b730-5cf1-4434-950f-4a3aa8f739cd/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy major:0 minor:308 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8c88b730-5cf1-4434-950f-4a3aa8f739cd/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-metrics:{mountpoint:/var/lib/kubelet/pods/8c88b730-5cf1-4434-950f-4a3aa8f739cd/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-metrics major:0 minor:305 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8c88b730-5cf1-4434-950f-4a3aa8f739cd/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-rules:{mountpoint:/var/lib/kubelet/pods/8c88b730-5cf1-4434-950f-4a3aa8f739cd/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-rules major:0 minor:307 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8c88b730-5cf1-4434-950f-4a3aa8f739cd/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-web:{mountpoint:/var/lib/kubelet/pods/8c88b730-5cf1-4434-950f-4a3aa8f739cd/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-web major:0 minor:299 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8c88b730-5cf1-4434-950f-4a3aa8f739cd/volumes/kubernetes.io~secret/secret-thanos-querier-tls:{mountpoint:/var/lib/kubelet/pods/8c88b730-5cf1-4434-950f-4a3aa8f739cd/volumes/kubernetes.io~secret/secret-thanos-querier-tls major:0 minor:304 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8f2f95c4-21b2-477b-a2cb-7c8590ec340d/volumes/kubernetes.io~empty-dir/etc-tuned:{mountpoint:/var/lib/kubelet/pods/8f2f95c4-21b2-477b-a2cb-7c8590ec340d/volumes/kubernetes.io~empty-dir/etc-tuned major:0 minor:47 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8f2f95c4-21b2-477b-a2cb-7c8590ec340d/volumes/kubernetes.io~empty-dir/tmp:{mountpoint:/var/lib/kubelet/pods/8f2f95c4-21b2-477b-a2cb-7c8590ec340d/volumes/kubernetes.io~empty-dir/tmp major:0 minor:48 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8f2f95c4-21b2-477b-a2cb-7c8590ec340d/volumes/kubernetes.io~projected/kube-api-access-t7ldj:{mountpoint:/var/lib/kubelet/pods/8f2f95c4-21b2-477b-a2cb-7c8590ec340d/volumes/kubernetes.io~projected/kube-api-access-t7ldj major:0 minor:55 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/c8b13e55-b3c7-4656-9845-16749704265f/volumes/kubernetes.io~projected/kube-api-access-v4d5j:{mountpoint:/var/lib/kubelet/pods/c8b13e55-b3c7-4656-9845-16749704265f/volumes/kubernetes.io~projected/kube-api-access-v4d5j major:0 minor:57 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d414339d-1a64-4a2a-ad15-805b71ebfd1b/volumes/kubernetes.io~projected/kube-api-access-rq9p9:{mountpoint:/var/lib/kubelet/pods/d414339d-1a64-4a2a-ad15-805b71ebfd1b/volumes/kubernetes.io~projected/kube-api-access-rq9p9 major:0 minor:264 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d414339d-1a64-4a2a-ad15-805b71ebfd1b/volumes/kubernetes.io~secret/node-exporter-kube-rbac-proxy-config:{mountpoint:/var/lib/kubelet/pods/d414339d-1a64-4a2a-ad15-805b71ebfd1b/volumes/kubernetes.io~secret/node-exporter-kube-rbac-proxy-config major:0 minor:261 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d414339d-1a64-4a2a-ad15-805b71ebfd1b/volumes/kubernetes.io~secret/node-exporter-tls:{mountpoint:/var/lib/kubelet/pods/d414339d-1a64-4a2a-ad15-805b71ebfd1b/volumes/kubernetes.io~secret/node-exporter-tls major:0 minor:260 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d5e5ac6e-d37d-457d-9ca7-93065f5925ff/volume-subpaths/run-systemd/ovnkube-controller/6:{mountpoint:/var/lib/kubelet/pods/d5e5ac6e-d37d-457d-9ca7-93065f5925ff/volume-subpaths/run-systemd/ovnkube-controller/6 major:0 minor:26 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d5e5ac6e-d37d-457d-9ca7-93065f5925ff/volumes/kubernetes.io~projected/kube-api-access-z2p79:{mountpoint:/var/lib/kubelet/pods/d5e5ac6e-d37d-457d-9ca7-93065f5925ff/volumes/kubernetes.io~projected/kube-api-access-z2p79 major:0 minor:56 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d5e5ac6e-d37d-457d-9ca7-93065f5925ff/volumes/kubernetes.io~secret/ovn-node-metrics-cert:{mountpoint:/var/lib/kubelet/pods/d5e5ac6e-d37d-457d-9ca7-93065f5925ff/volumes/kubernetes.io~secret/ovn-node-metrics-cert major:0 minor:49 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d7c355fa-2141-463e-8cbc-bae3c44a16f1/volumes/kubernetes.io~projected/kube-api-access-vg89l:{mountpoint:/var/lib/kubelet/pods/d7c355fa-2141-463e-8cbc-bae3c44a16f1/volumes/kubernetes.io~projected/kube-api-access-vg89l major:0 minor:52 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d8dc5e17-81b4-413a-a48f-6175d0ed3ec0/volumes/kubernetes.io~projected/kube-api-access-vh5d6:{mountpoint:/var/lib/kubelet/pods/d8dc5e17-81b4-413a-a48f-6175d0ed3ec0/volumes/kubernetes.io~projected/kube-api-access-vh5d6 major:0 minor:203 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d8dc5e17-81b4-413a-a48f-6175d0ed3ec0/volumes/kubernetes.io~secret/cert:{mountpoint:/var/lib/kubelet/pods/d8dc5e17-81b4-413a-a48f-6175d0ed3ec0/volumes/kubernetes.io~secret/cert major:0 minor:45 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/db4b06a3-45e5-4d45-9d7a-407d117d9454/volumes/kubernetes.io~projected/kube-api-access-9krk9:{mountpoint:/var/lib/kubelet/pods/db4b06a3-45e5-4d45-9d7a-407d117d9454/volumes/kubernetes.io~projected/kube-api-access-9krk9 major:0 minor:58 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/dc220f0e-8a21-4d7f-96b3-e7055c242f42/volumes/kubernetes.io~projected/kube-api-access-tq25h:{mountpoint:/var/lib/kubelet/pods/dc220f0e-8a21-4d7f-96b3-e7055c242f42/volumes/kubernetes.io~projected/kube-api-access-tq25h major:0 minor:51 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e810bcd1-78da-409f-b86e-08c47031939e/volumes/kubernetes.io~secret/original-pull-secret:{mountpoint:/var/lib/kubelet/pods/e810bcd1-78da-409f-b86e-08c47031939e/volumes/kubernetes.io~secret/original-pull-secret major:0 minor:357 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/f69b6a2e-d33f-4cc8-8e0a-598b20948f38/volumes/kubernetes.io~secret/agent-certs:{mountpoint:/var/lib/kubelet/pods/f69b6a2e-d33f-4cc8-8e0a-598b20948f38/volumes/kubernetes.io~secret/agent-certs major:0 minor:50 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0} overlay_0-100:{mountpoint:/var/lib/containers/storage/overlay/130d17af1630fcd61789af67128c0873f250c1b555551bf42d7a155f5d30fa4c/merged major:0 minor:100 fsType:overlay blockSize:0} overlay_0-102:{mountpoint:/var/lib/containers/storage/overlay/8e08ea87c0e5880cee99545d07e3403e45f924885552b03b310032f4bb03937d/merged major:0 minor:102 fsType:overlay blockSize:0} overlay_0-104:{mountpoint:/var/lib/containers/storage/overlay/0d94147bd6095d5c978abc7f453d30ae8b9f39f719082cb31f89c05e7183796e/merged major:0 minor:104 fsType:overlay blockSize:0} overlay_0-106:{mountpoint:/var/lib/containers/storage/overlay/0d0680c6268d8e570e1f0c28a4a4573775c9af4850213df492554213cb0ec9ee/merged major:0 minor:106 fsType:overlay blockSize:0} overlay_0-114:{mountpoint:/var/lib/containers/storage/overlay/9b7c6e3e9829259201f6520925fc6cf5f99e026678f67fb5d865e7865c9c1dc0/merged major:0 minor:114 fsType:overlay blockSize:0} overlay_0-119:{mountpoint:/var/lib/containers/storage/overlay/a48e91d9a2a825d54d6a1bbac8019d79eb0bca5c5ae6350cc2e25274a2370f72/merged major:0 minor:119 fsType:overlay blockSize:0} overlay_0-124:{mountpoint:/var/lib/containers/storage/overlay/b313e752ecf0f3a1eaa7971dc238638fba27bf7986aab9804d3b2dabe0cf2b67/merged major:0 minor:124 fsType:overlay blockSize:0} overlay_0-127:{mountpoint:/var/lib/containers/storage/overlay/36239499931dbc478931e827a1a412ef275858e1b363cb186a7c84b57d46c051/merged major:0 minor:127 fsType:overlay blockSize:0} overlay_0-132:{mountpoint:/var/lib/containers/storage/overlay/de6965527db32f14a96d54aa7bd0ad0659044a4b374781142cd969f2c938c170/merged major:0 minor:132 fsType:overlay blockSize:0} overlay_0-133:{mountpoint:/var/lib/containers/storage/overlay/c1337827c1c70d3429aa129d5ad5ce5cbf7e5f0778decbb20e3e95a44bb7f8d5/merged major:0 minor:133 fsType:overlay blockSize:0} overlay_0-142:{mountpoint:/var/lib/containers/storage/overlay/32c895261dbc9c97aa1e575e8d3f87e957236beb4a99d18baeb830d936330a47/merged major:0 minor:142 fsType:overlay blockSize:0} overlay_0-146:{mountpoint:/var/lib/containers/storage/overlay/1e305841f9f1aa4ed69f94cd5e97d275ade2256ba76bdc313b505b6906c7e50e/merged major:0 minor:146 fsType:overlay blockSize:0} overlay_0-152:{mountpoint:/var/lib/containers/storage/overlay/d76cf469f671767d59e8bae6a0c68be8ae47122d720443b8cbeba4c288d02d0a/merged major:0 minor:152 fsType:overlay blockSize:0} overlay_0-157:{mountpoint:/var/lib/containers/storage/overlay/4809705f25b7bfa1da11751af9ab804e7497c165155ab6b0bcf6fb148138fa1c/merged major:0 minor:157 fsType:overlay blockSize:0} overlay_0-162:{mountpoint:/var/lib/containers/storage/overlay/34ae0504eb45cca3803898fe19d85ee59c83347abdebcd84a4e6ea9895fbc670/merged major:0 minor:162 fsType:overlay blockSize:0} overlay_0-167:{mountpoint:/var/lib/containers/storage/overlay/87cef41b1e0f6e167ba6f8765905b71458ed9b7d9541e9543c0ab600bf02b2f8/merged major:0 minor:167 fsType:overlay blockSize:0} overlay_0-172:{mountpoint:/var/lib/containers/storage/overlay/9e7792ff054e96828d87e294ec1ad54106c897f94ce4564e2d0b1ba1e2ba8c47/merged major:0 minor:172 fsType:overlay blockSize:0} overlay_0-177:{mountpoint:/var/lib/containers/storage/overlay/4f739120923e089adb83fd2d2a241d5e2e7b22ff31ee15fe2028651aeb907bc9/merged major:0 minor:177 fsType:overlay blockSize:0} overlay_0-182:{mountpoint:/var/lib/containers/storage/overlay/0c17cbacd576f854111d15d29d01cbe6e2b43ed8a96e58bb54badda3c343da37/merged major:0 minor:182 fsType:overlay blockSize:0} overlay_0-187:{mountpoint:/var/lib/containers/storage/overlay/2d8bc240f61ccd7ada4a327e1bd4cc1a61974811a683cc4366a38de8d31d0eaf/merged major:0 minor:187 fsType:overlay blockSize:0} overlay_0-192:{mountpoint:/var/lib/containers/storage/overlay/f1832dfc2352e8b2188f486aabe231565903b5dffff1a0b61fe2ba0909e521c9/merged major:0 minor:192 fsType:overlay blockSize:0} overlay_0-211:{mountpoint:/var/lib/containers/storage/overlay/9fd94211a71380796ae0ae7037dc439910f4ffedc922021ee9654331cc9372a8/merged major:0 minor:211 fsType:overlay blockSize:0} overlay_0-213:{mountpoint:/var/lib/containers/storage/overlay/d4332e92f0ad2a44b90dc677215a86373735f7b0f384c6ff9f65f27f36268100/merged major:0 minor:213 fsType:overlay blockSize:0} overlay_0-215:{mountpoint:/var/lib/containers/storage/overlay/e3c2298704131efcad6369ab4cceadb0b23a225079373d83f8314520d6fd2d7e/merged major:0 minor:215 fsType:overlay blockSize:0} overlay_0-217:{mountpoint:/var/lib/containers/storage/overlay/d4252904a4b8936e5e9e619b74d152a39d4d7be157b377aa33959ab4c2a878bb/merged major:0 minor:217 fsType:overlay blockSize:0} overlay_0-219:{mountpoint:/var/lib/containers/storage/overlay/3e1cac10db1e5944a0b0ff8e803f068d5ffad53341bb4cdeff565cd9afb01746/merged major:0 minor:219 fsType:overlay blockSize:0} overlay_0-225:{mountpoint:/var/lib/containers/storage/overlay/faf902155a624416a7bbe1c83ad0fb378b7d281be992e6538769ebb307772bba/merged major:0 minor:225 fsType:overlay blockSize:0} overlay_0-230:{mountpoint:/var/lib/containers/storage/overlay/d552d991aef222c1ffebe886eafc8fd75fab1eb4bbbcc7329d7753116263fe24/merged major:0 minor:230 fsType:overlay blockSize:0} overlay_0-236:{mountpoint:/var/lib/containers/storage/overlay/fec6ced483c534c2c326155c09b860180740971ab8ccf9cdba811b3be110a104/merged major:0 minor:236 fsType:overlay blockSize:0} overlay_0-241:{mountpoint:/var/lib/containers/storage/overlay/49ac8716eeb8c0cac29e7901f1450131789b4cba1e8e54db533f2346ca98dbcd/merged major:0 minor:241 fsType:overlay blockSize:0} overlay_0-255:{mountpoint:/var/lib/containers/storage/overlay/55ced85fffb4d672ca858acef09d1f15f817b5cd4b54352680413db9c6e2de5e/merged major:0 minor:255 fsType:overlay blockSize:0} overlay_0-267:{mountpoint:/var/lib/containers/storage/overlay/06558dd0f59cc7c4df645d64f02e6356a8edfac1d0c926c6054c8f58e19fb648/merged major:0 minor:267 fsType:overlay blockSize:0} overlay_0-269:{mountpoint:/var/lib/containers/storage/overlay/efc02a06c860848c334ee6dd93b8aae126a4ed4b4c0a29ef5a97682d5499179f/merged major:0 minor:269 fsType:overlay blockSize:0} overlay_0-271:{mountpoint:/var/lib/containers/storage/overlay/f9458816f6fd284dd2f548bbb9504167d60edbab305b1ab1df37b2d7fc0d8031/merged major:0 minor:271 fsType:overlay blockSize:0} overlay_0-273:{mountpoint:/var/lib/containers/storage/overlay/db5a0c9d51224dd8643dcb7aeccfcee7c251f0c8dff5af906efc44a5dc9b6c07/merged major:0 minor:273 fsType:overlay blockSize:0} overlay_0-279:{mountpoint:/var/lib/containers/storage/overlay/a9c631ec14c3dbeffbf99da1b612f2495cec51f9cb3284419f58a8b9a58eca00/merged major:0 minor:279 fsType:overlay blockSize:0} overlay_0-284:{mountpoint:/var/lib/containers/storage/overlay/3a531e9ad99089f054c8449c572a917a5427ff593dcd6914fe12a5cd2a04dc9b/merged major:0 minor:284 fsType:overlay blockSize:0} overlay_0-286:{mountpoint:/var/lib/containers/storage/overlay/42ea50619c64061d87d37a7eb937ce9e30834882c6c7f499cecfd73d2485d05b/merged major:0 minor:286 fsType:overlay blockSize:0} overlay_0-294:{mountpoint:/var/lib/containers/storage/overlay/fdac679e77b9688b0a5712a8bcc02c657dcc1bb188a3f66b5c75e3385be35941/merged major:0 minor:294 fsType:overlay blockSize:0} overlay_0-312:{mountpoint:/var/lib/containers/storage/overlay/8c5f961e16d149a492aefd7d3a1a667b9cc8915927684d54ff9cd3dfd3dee825/merged major:0 minor:312 fsType:overlay blockSize:0} overlay_0-318:{mountpoint:/var/lib/containers/storage/overlay/99d5cc0f9d375fb6a176c80b06e37270275da67e1cba0388c7ee7d04b3f992b7/merged major:0 minor:318 fsType:overlay blockSize:0} overlay_0-332:{mountpoint:/var/lib/containers/storage/overlay/b86c6dc40b1f00b4d533baf2184b16c343e4f4621472ebc4876273abcc35cca3/merged major:0 minor:332 fsType:overlay blockSize:0} overlay_0-334:{mountpoint:/var/lib/containers/storage/overlay/6639c16225c22e875e84f6da65ce92b6fced0f1003462af3a88ab2b15b05d02c/merged major:0 minor:334 fsType:overlay blockSize:0} overlay_0-336:{mountpoint:/var/lib/containers/storage/overlay/fcd7a1495fefe8cabdb9c8f153a3efe38e0307fbe8c185cb87c26f541650d7a2/merged major:0 minor:336 fsType:overlay blockSize:0} overlay_0-342:{mountpoint:/var/lib/containers/storage/overlay/d432084895a54a0e0c67063487c4a7ac46854631de97e36b331ca21ddd466b4d/merged major:0 minor:342 fsType:overlay blockSize:0} overlay_0-344:{mountpoint:/var/lib/containers/storage/overlay/e326432c26317b7a72d3493f6394e0ea2d3d8126c9b6968dccaf8cf12143accb/merged major:0 minor:344 fsType:overlay blockSize:0} overlay_0-350:{mountpoint:/var/lib/containers/storage/overlay/55a46bccd5835591b3f2eb18916f9de2454b93b128a621040c463b306767c01b/merged major:0 minor:350 fsType:overlay blockSize:0} overlay_0-355:{mountpoint:/var/lib/containers/storage/overlay/22d6c89783f7e605c3214fe8c5e54b2fffd573e6c41ef4706f3bb41a53bb5679/merged major:0 minor:355 fsType:overlay blockSize:0} overlay_0-373:{mountpoint:/var/lib/containers/storage/overlay/dd30046dd9ad7888c8724681f945ee812cdcb2d8341747ae9be641065ca150f5/merged major:0 minor:373 fsType:overlay blockSize:0} overlay_0-375:{mountpoint:/var/lib/containers/storage/overlay/02c9e9006929f699d9a1d9d4c852d51f9e2b6d0dd76b32eca92ab9842d25af33/merged major:0 minor:375 fsType:overlay blockSize:0} overlay_0-377:{mountpoint:/var/lib/containers/storage/overlay/4443ad078d5693cf7f003a4c92a76c7a18bb04583e93345c0903125dfb2d066b/merged major:0 minor:377 fsType:overlay blockSize:0} overlay_0-379:{mountpoint:/var/lib/containers/storage/overlay/532c507d6bfe8fd2661f67af31570e3194c861ca736f9704377cbe8bfec77a60/merged major:0 minor:379 fsType:overlay blockSize:0} overlay_0-381:{mountpoint:/var/lib/containers/storage/overlay/17cf599b166792d2124df3addb2da295103dd3b3d79604b182bbf004ae09ac58/merged major:0 minor:381 fsType:overlay blockSize:0} overlay_0-386:{mountpoint:/var/lib/containers/storage/overlay/91b39c7241fa8ab73e882231a67340c8c83fbaaf5e43a6e4c9d95c20a24d520f/merged major:0 minor:386 fsType:overlay blockSize:0} overlay_0-392:{mountpoint:/var/lib/containers/storage/overlay/1358fe64076de1179eb2c37c06967e751f64238fe29b94a825d0d23e9e387832/merged major:0 minor:392 fsType:overlay blockSize:0} overlay_0-397:{mountpoint:/var/lib/containers/storage/overlay/c6b052fe619b03982914efce5e112fea6846616eef5b5724d1f20c62f796e462/merged major:0 minor:397 fsType:overlay blockSize:0} overlay_0-402:{mountpoint:/var/lib/containers/storage/overlay/65dacffbc60a3bce875d6e54fac9beb2e8fef46128c2c6b915c00d1687079617/merged major:0 minor:402 fsType:overlay blockSize:0} overlay_0-408:{mountpoint:/var/lib/containers/storage/overlay/39b3e105da77482a1865e9ad6138adbe9aa95818be63d5470bfa60d132a199ec/merged major:0 minor:408 fsType:overlay blockSize:0} overlay_0-413:{mountpoint:/var/lib/containers/storage/overlay/59f42120c2e5ee07b2a7539e57c127f7bdec50ac9dbc8910e88fe473a4341d13/merged major:0 minor:413 fsType:overlay blockSize:0} overlay_0-418:{mountpoint:/var/lib/containers/storage/overlay/e589cdd1c8344fb9b5975a5975e866e9f77b92fcd55262a23b9e82beddc58d29/merged major:0 minor:418 fsType:overlay blockSize:0} overlay_0-423:{mountpoint:/var/lib/containers/storage/overlay/9df7341cffe6d41713c7ffe6653f9d96e017089c0dd581bad6b3b92b25f66efa/merged major:0 minor:423 fsType:overlay blockSize:0} overlay_0-64:{mountpoint:/var/lib/containers/storage/overlay/e548df97506f67dec1e2bc1ceb41a98890c4da982b3b3e481ec65e2542388396/merged major:0 minor:64 fsType:overlay blockSize:0} overlay_0-66:{mountpoint:/var/lib/containers/storage/overlay/eadf59143a077d7489e34055a22e771ba6315ab5af3909fe4d82a5866249dbc8/merged major:0 minor:66 fsType:overlay blockSize:0} overlay_0-69:{mountpoint:/var/lib/containers/storage/overlay/936a0c26a561fa6e590536613124682595c1992ac95350ffe482e1c8fb7d4d58/merged major:0 minor:69 fsType:overlay blockSize:0} overlay_0-72:{mountpoint:/var/lib/containers/storage/overlay/64b213c437ae1e6d38570023852fb2cfcbe62369d4015cc066c31ff42da343ed/merged major:0 minor:72 fsType:overlay blockSize:0} overlay_0-76:{mountpoint:/var/lib/containers/storage/overlay/95533819089ecaa9b203addb8e9269cf7f73163c0794d7e7b78f0a5a7f3191d4/merged major:0 minor:76 fsType:overlay blockSize:0} overlay_0-80:{mountpoint:/var/lib/containers/storage/overlay/79dd55082c49f8d34db9e178f62eeb284c0d03ad2d1d3e2751d466d71f7b526b/merged major:0 minor:80 fsType:overlay blockSize:0} overlay_0-84:{mountpoint:/var/lib/containers/storage/overlay/b20cc3d03bb1d066852cad45d53760d7a2cc451848ef5dd13fc6ab7e8eb80686/merged major:0 minor:84 fsType:overlay blockSize:0} overlay_0-88:{mountpoint:/var/lib/containers/storage/overlay/f4c6c6268a4b872053e6264fc4eab8584d683ed37d70cbff62bbf32289e2e9f6/merged major:0 minor:88 fsType:overlay blockSize:0} overlay_0-93:{mountpoint:/var/lib/containers/storage/overlay/9cdbfb0c8a1514cb3e89f784b54bd03dd5ae88bf613759314c024da1c44b5df0/merged major:0 minor:93 fsType:overlay blockSize:0} overlay_0-96:{mountpoint:/var/lib/containers/storage/overlay/b55a8183f195de641f49537aa2b57067b0b1c1857e35707fa0d109687694594f/merged major:0 minor:96 fsType:overlay blockSize:0}]
Oct 30 21:50:19.500320 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.499940    5510 manager.go:217] Machine: {Timestamp:2026-10-30 21:50:19.499430428 +0000 UTC m=+0.060240424 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100351 MemoryCapacity:33164337152 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec26b90f21bde3a07705d9f8f834ca89 SystemUUID:ec26b90f-21bd-e3a0-7705-d9f8f834ca89 BootID:c916d274-bf09-4d8a-a59c-d198f21ad112 Filesystems:[{Device:/run/containers/storage/overlay-containers/02e263b2978459859989e1cdc818ce3ef741757f7d74157e4fb4f4a83c1c0947/userdata/shm DeviceMajor:0 DeviceMinor:310 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/0ab70415-9f1d-4988-a228-df6ed621f0d8/volumes/kubernetes.io~projected/kube-api-access-td6rr DeviceMajor:0 DeviceMinor:53 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/c1581eb1a638205af61c045effda2bf3708d1f9f54df9b7a6c6c4d69436f7817/userdata/shm DeviceMajor:0 DeviceMinor:60 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-76 DeviceMajor:0 DeviceMinor:76 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/4d6cb4d22f1a3341a2a3e4f90510ae7fbec361fb13b62997a07cb2f4f9fe864b/userdata/shm DeviceMajor:0 DeviceMinor:205 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-213 DeviceMajor:0 DeviceMinor:213 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-279 DeviceMajor:0 DeviceMinor:279 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/0e0d17a1-7742-42b0-9d9b-740bb945e454/volumes/kubernetes.io~secret/secret-telemeter-client-kube-rbac-proxy-config DeviceMajor:0 DeviceMinor:325 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true} {Device:/run/containers/storage/overlay-containers/327a89302c3fd96103b398b276678511aec903499e1c71d72134532152be1296/userdata/shm DeviceMajor:0 DeviceMinor:61 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/8c88b730-5cf1-4434-950f-4a3aa8f739cd/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-metrics DeviceMajor:0 DeviceMinor:305 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/8c88b730-5cf1-4434-950f-4a3aa8f739cd/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy DeviceMajor:0 DeviceMinor:308 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-336 DeviceMajor:0 DeviceMinor:336 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-344 DeviceMajor:0 DeviceMinor:344 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16582168576 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-177 DeviceMajor:0 DeviceMinor:177 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-66 DeviceMajor:0 DeviceMinor:66 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-119 DeviceMajor:0 DeviceMinor:119 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-172 DeviceMajor:0 DeviceMinor:172 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-187 DeviceMajor:0 DeviceMinor:187 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/42d54b8a-4f48-46a7-8a34-1f0f29cd1602/volumes/kubernetes.io~secret/console-oauth-config DeviceMajor:0 DeviceMinor:244 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/fbffb78323609cf8dd21b10efbe35374ed8184fc5b3e616fe9f4b87fd69b0936/userdata/shm DeviceMajor:0 DeviceMinor:253 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/d414339d-1a64-4a2a-ad15-805b71ebfd1b/volumes/kubernetes.io~secret/node-exporter-tls DeviceMajor:0 DeviceMinor:260 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/cc6e2905d1c15d26a3a375d45ff83a2a3e6b21a41741fe6c57f315c1ea464d58/userdata/shm DeviceMajor:0 DeviceMinor:74 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/2bdc41beaf5186f2d39137449d5d2a9606e952cfcb48cef9006ad1fab03049da/userdata/shm DeviceMajor:0 DeviceMinor:82 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/2fe50918-ee11-4d17-b0fe-27c7898912c6/volumes/kubernetes.io~projected/kube-api-access-25ggc DeviceMajor:0 DeviceMinor:204 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-215 DeviceMajor:0 DeviceMinor:215 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/3abd55eb-d1d8-4fdf-b82c-d35894b652ac/volumes/kubernetes.io~secret/openshift-state-metrics-tls DeviceMajor:0 DeviceMinor:257 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-271 DeviceMajor:0 DeviceMinor:271 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-318 DeviceMajor:0 DeviceMinor:318 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-392 DeviceMajor:0 DeviceMinor:392 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/847fb3c0d4eb226ac6a8ad5fdd1e6fb20a842ef516173dfee5213674f9f0d651/userdata/shm DeviceMajor:0 DeviceMinor:86 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/078dbad40157a479574f1da113133d4990f89749660a3686f1fd0e1ff2882507/userdata/shm DeviceMajor:0 DeviceMinor:90 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-132 DeviceMajor:0 DeviceMinor:132 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/7d6190d5-6574-4704-97cb-2241398548e3/volumes/kubernetes.io~projected/kube-api-access-ggf4d DeviceMajor:0 DeviceMinor:202 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-286 DeviceMajor:0 DeviceMinor:286 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/8c88b730-5cf1-4434-950f-4a3aa8f739cd/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-rules DeviceMajor:0 DeviceMinor:307 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/0e0d17a1-7742-42b0-9d9b-740bb945e454/volumes/kubernetes.io~secret/federate-client-tls DeviceMajor:0 DeviceMinor:326 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:overlay_0-114 DeviceMajor:0 DeviceMinor:114 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/d414339d-1a64-4a2a-ad15-805b71ebfd1b/volumes/kubernetes.io~projected/kube-api-access-rq9p9 DeviceMajor:0 DeviceMinor:264 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/8c88b730-5cf1-4434-950f-4a3aa8f739cd/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-web DeviceMajor:0 DeviceMinor:299 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-423 DeviceMajor:0 DeviceMinor:423 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-142 DeviceMajor:0 DeviceMinor:142 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-413 DeviceMajor:0 DeviceMinor:413 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-312 DeviceMajor:0 DeviceMinor:312 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-167 DeviceMajor:0 DeviceMinor:167 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/5791c43c54a24e8372a1e21b3010621359dc2caa4fbdac86975465bbfa40f7b3/userdata/shm DeviceMajor:0 DeviceMinor:91 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/d8dc5e17-81b4-413a-a48f-6175d0ed3ec0/volumes/kubernetes.io~projected/kube-api-access-vh5d6 DeviceMajor:0 DeviceMinor:203 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-267 DeviceMajor:0 DeviceMinor:267 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/31a389bb-f36f-4e53-9654-5f4648dae38a/volumes/kubernetes.io~secret/console-oauth-config DeviceMajor:0 DeviceMinor:321 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-69 DeviceMajor:0 DeviceMinor:69 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-96 DeviceMajor:0 DeviceMinor:96 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/dc220f0e-8a21-4d7f-96b3-e7055c242f42/volumes/kubernetes.io~projected/kube-api-access-tq25h DeviceMajor:0 DeviceMinor:51 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-219 DeviceMajor:0 DeviceMinor:219 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-332 DeviceMajor:0 DeviceMinor:332 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-334 DeviceMajor:0 DeviceMinor:334 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/52eb01f8-e86a-4458-8875-639394a87b30/volumes/kubernetes.io~secret/metrics-certs DeviceMajor:0 DeviceMinor:360 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/76daddeb7d1867d1fe0158bb897cbf216243b7ff1593fa67f48849c8d3d8ddab/userdata/shm DeviceMajor:0 DeviceMinor:369 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-381 DeviceMajor:0 DeviceMinor:381 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-84 DeviceMajor:0 DeviceMinor:84 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-104 DeviceMajor:0 DeviceMinor:104 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-152 DeviceMajor:0 DeviceMinor:152 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/d5e5ac6e-d37d-457d-9ca7-93065f5925ff/volume-subpaths/run-systemd/ovnkube-controller/6 DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true} {Device:/var/lib/kubelet/pods/2fe50918-ee11-4d17-b0fe-27c7898912c6/volumes/kubernetes.io~secret/metrics-tls DeviceMajor:0 DeviceMinor:197 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/f45d4e00313976fd643259c26716f35ec4f66325b137a79e47d646a0ddaf4cd0/userdata/shm DeviceMajor:0 DeviceMinor:265 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/31a389bb-f36f-4e53-9654-5f4648dae38a/volumes/kubernetes.io~secret/console-serving-cert DeviceMajor:0 DeviceMinor:320 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/f4065939ad3a362fabd190f4f532a0ba04768adf8730414343ea996a097c52f5/userdata/shm DeviceMajor:0 DeviceMinor:330 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/26d84296d1d451363382e3d3f1fd68416a311f8b1ba5919ac8230687bef7460c/userdata/shm DeviceMajor:0 DeviceMinor:78 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-102 DeviceMajor:0 DeviceMinor:102 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-124 DeviceMajor:0 DeviceMinor:124 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-192 DeviceMajor:0 DeviceMinor:192 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/d188c9559e229f137adf7ecb5417f5f9310e2d711a052c0aece44396a292cae9/userdata/shm DeviceMajor:0 DeviceMinor:323 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/var/lib/kubelet/pods/8f2f95c4-21b2-477b-a2cb-7c8590ec340d/volumes/kubernetes.io~empty-dir/tmp DeviceMajor:0 DeviceMinor:48 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-146 DeviceMajor:0 DeviceMinor:146 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-182 DeviceMajor:0 DeviceMinor:182 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-284 DeviceMajor:0 DeviceMinor:284 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-294 DeviceMajor:0 DeviceMinor:294 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-342 DeviceMajor:0 DeviceMinor:342 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16582168576 Type:vfs Inodes:1048576 HasInodes:true} {Device:overlay_0-64 DeviceMajor:0 DeviceMinor:64 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/ff49e81b45715f352095c16b17b12a8c2cc3ccc92322711b2615d651155d2dbb/userdata/shm DeviceMajor:0 DeviceMinor:207 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/42d54b8a-4f48-46a7-8a34-1f0f29cd1602/volumes/kubernetes.io~secret/console-serving-cert DeviceMajor:0 DeviceMinor:243 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-350 DeviceMajor:0 DeviceMinor:350 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/d8dc5e17-81b4-413a-a48f-6175d0ed3ec0/volumes/kubernetes.io~secret/cert DeviceMajor:0 DeviceMinor:45 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-211 DeviceMajor:0 DeviceMinor:211 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/42d54b8a-4f48-46a7-8a34-1f0f29cd1602/volumes/kubernetes.io~projected/kube-api-access-5c4fj DeviceMajor:0 DeviceMinor:252 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/7c658481457a80544871f4902bf0c0cb431e33d2e3c54401546755d56b0716b4/userdata/shm DeviceMajor:0 DeviceMinor:262 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/ce29c22093919e23269e62f2db4fc0b855f665df8a6eb65b9945172a1594cc33/userdata/shm DeviceMajor:0 DeviceMinor:366 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-379 DeviceMajor:0 DeviceMinor:379 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/c8b13e55-b3c7-4656-9845-16749704265f/volumes/kubernetes.io~projected/kube-api-access-v4d5j DeviceMajor:0 DeviceMinor:57 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-133 DeviceMajor:0 DeviceMinor:133 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-217 DeviceMajor:0 DeviceMinor:217 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-255 DeviceMajor:0 DeviceMinor:255 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-408 DeviceMajor:0 DeviceMinor:408 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/52eb01f8-e86a-4458-8875-639394a87b30/volumes/kubernetes.io~projected/kube-api-access-zxwjt DeviceMajor:0 DeviceMinor:54 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-80 DeviceMajor:0 DeviceMinor:80 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-88 DeviceMajor:0 DeviceMinor:88 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/3abd55eb-d1d8-4fdf-b82c-d35894b652ac/volumes/kubernetes.io~projected/kube-api-access-f2qbm DeviceMajor:0 DeviceMinor:259 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-72 DeviceMajor:0 DeviceMinor:72 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-236 DeviceMajor:0 DeviceMinor:236 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/d414339d-1a64-4a2a-ad15-805b71ebfd1b/volumes/kubernetes.io~secret/node-exporter-kube-rbac-proxy-config DeviceMajor:0 DeviceMinor:261 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-418 DeviceMajor:0 DeviceMinor:418 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/8883cb69-54c2-437c-a92e-3bc38547027f/volumes/kubernetes.io~projected/kube-api-access-b2929 DeviceMajor:0 DeviceMinor:315 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-373 DeviceMajor:0 DeviceMinor:373 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/8c88b730-5cf1-4434-950f-4a3aa8f739cd/volumes/kubernetes.io~projected/kube-api-access-ttgkm DeviceMajor:0 DeviceMinor:309 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/ce973673f940234e7bde5a0d7a85118c6da9bd4ed38d54bac859dc43fe25920b/userdata/shm DeviceMajor:0 DeviceMinor:98 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-127 DeviceMajor:0 DeviceMinor:127 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/8c88b730-5cf1-4434-950f-4a3aa8f739cd/volumes/kubernetes.io~secret/secret-thanos-querier-tls DeviceMajor:0 DeviceMinor:304 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/db4b06a3-45e5-4d45-9d7a-407d117d9454/volumes/kubernetes.io~projected/kube-api-access-9krk9 DeviceMajor:0 DeviceMinor:58 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-162 DeviceMajor:0 DeviceMinor:162 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/7d6190d5-6574-4704-97cb-2241398548e3/volumes/kubernetes.io~secret/insights-runtime-extractor-tls DeviceMajor:0 DeviceMinor:201 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-377 DeviceMajor:0 DeviceMinor:377 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/29097bc9c91c0bbb3e5034ccc9cf5e145105cca40a1dfb3dee33c7d2ad65ea31/userdata/shm DeviceMajor:0 DeviceMinor:68 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-225 DeviceMajor:0 DeviceMinor:225 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/e706fb1e7061ada9e54ae7abc07fc31473d443d12a0371f11a83d19444e35915/userdata/shm DeviceMajor:0 DeviceMinor:316 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/d5e5ac6e-d37d-457d-9ca7-93065f5925ff/volumes/kubernetes.io~secret/ovn-node-metrics-cert DeviceMajor:0 DeviceMinor:49 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/8f2f95c4-21b2-477b-a2cb-7c8590ec340d/volumes/kubernetes.io~projected/kube-api-access-t7ldj DeviceMajor:0 DeviceMinor:55 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/04465eda-e6db-42d3-8d5f-c9be668038a9/volumes/kubernetes.io~projected/kube-api-access-hrxwg DeviceMajor:0 DeviceMinor:59 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-100 DeviceMajor:0 DeviceMinor:100 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-397 DeviceMajor:0 DeviceMinor:397 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/d7c355fa-2141-463e-8cbc-bae3c44a16f1/volumes/kubernetes.io~projected/kube-api-access-vg89l DeviceMajor:0 DeviceMinor:52 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/31a389bb-f36f-4e53-9654-5f4648dae38a/volumes/kubernetes.io~projected/kube-api-access-rb2wf DeviceMajor:0 DeviceMinor:322 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/0e0d17a1-7742-42b0-9d9b-740bb945e454/volumes/kubernetes.io~secret/telemeter-client-tls DeviceMajor:0 DeviceMinor:327 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/e810bcd1-78da-409f-b86e-08c47031939e/volumes/kubernetes.io~secret/original-pull-secret DeviceMajor:0 DeviceMinor:357 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/73c208eb-ffc0-46f2-8bd4-05ade5adb529/volumes/kubernetes.io~projected/kube-api-access-fcjg4 DeviceMajor:0 DeviceMinor:367 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-386 DeviceMajor:0 DeviceMinor:386 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/f69b6a2e-d33f-4cc8-8e0a-598b20948f38/volumes/kubernetes.io~secret/agent-certs DeviceMajor:0 DeviceMinor:50 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-93 DeviceMajor:0 DeviceMinor:93 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-269 DeviceMajor:0 DeviceMinor:269 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/8c88b730-5cf1-4434-950f-4a3aa8f739cd/volumes/kubernetes.io~secret/secret-grpc-tls DeviceMajor:0 DeviceMinor:306 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-375 DeviceMajor:0 DeviceMinor:375 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/d5e5ac6e-d37d-457d-9ca7-93065f5925ff/volumes/kubernetes.io~projected/kube-api-access-z2p79 DeviceMajor:0 DeviceMinor:56 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-230 DeviceMajor:0 DeviceMinor:230 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/8f2f95c4-21b2-477b-a2cb-7c8590ec340d/volumes/kubernetes.io~empty-dir/etc-tuned DeviceMajor:0 DeviceMinor:47 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/fa5a20193e2a7edb9a384afb717ec20cb79c193302573d94ad22f8eecfbb3a16/userdata/shm DeviceMajor:0 DeviceMinor:209 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/8883cb69-54c2-437c-a92e-3bc38547027f/volumes/kubernetes.io~secret/monitoring-plugin-cert DeviceMajor:0 DeviceMinor:314 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/0e0d17a1-7742-42b0-9d9b-740bb945e454/volumes/kubernetes.io~projected/kube-api-access-9gzpz DeviceMajor:0 DeviceMinor:329 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/d90f0f601a7259b584b3cceb562a51226b1daf12d6df216a04e38d8680c26a8e/userdata/shm DeviceMajor:0 DeviceMinor:371 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-106 DeviceMajor:0 DeviceMinor:106 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-157 DeviceMajor:0 DeviceMinor:157 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-241 DeviceMajor:0 DeviceMinor:241 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/3abd55eb-d1d8-4fdf-b82c-d35894b652ac/volumes/kubernetes.io~secret/openshift-state-metrics-kube-rbac-proxy-config DeviceMajor:0 DeviceMinor:258 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-273 DeviceMajor:0 DeviceMinor:273 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-402 DeviceMajor:0 DeviceMinor:402 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/0e0d17a1-7742-42b0-9d9b-740bb945e454/volumes/kubernetes.io~secret/secret-telemeter-client DeviceMajor:0 DeviceMinor:328 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-355 DeviceMajor:0 DeviceMinor:355 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:02e263b29784598 MacAddress:7a:98:c7:33:76:b4 Speed:10000 Mtu:8901} {Name:4d6cb4d22f1a334 MacAddress:02:63:3e:7b:ec:57 Speed:10000 Mtu:8901} {Name:76daddeb7d1867d MacAddress:9a:c3:18:ce:d7:ca Speed:10000 Mtu:8901} {Name:7c658481457a805 MacAddress:5a:c1:81:ba:2e:21 Speed:10000 Mtu:8901} {Name:br-ex MacAddress:06:ff:cf:f6:cd:09 Speed:0 Mtu:9001} {Name:br-int MacAddress:32:28:a7:1a:e0:ab Speed:0 Mtu:8901} {Name:ce29c22093919e2 MacAddress:6a:2a:50:6a:d3:f9 Speed:10000 Mtu:8901} {Name:d188c9559e229f1 MacAddress:72:c9:66:d5:9c:71 Speed:10000 Mtu:8901} {Name:d90f0f601a7259b MacAddress:e2:dd:9b:15:15:18 Speed:10000 Mtu:8901} {Name:e706fb1e7061ada MacAddress:ee:42:1d:5c:54:b7 Speed:10000 Mtu:8901} {Name:ens5 MacAddress:06:ff:cf:f6:cd:09 Speed:0 Mtu:9001} {Name:f4065939ad3a362 MacAddress:be:4b:22:88:69:1c Speed:10000 Mtu:8901} {Name:fa5a20193e2a7ed MacAddress:82:29:55:a1:b9:01 Speed:10000 Mtu:8901} {Name:fbffb78323609cf MacAddress:2e:d5:22:2b:a7:53 Speed:10000 Mtu:8901} {Name:ff49e81b45715f3 MacAddress:5a:1b:f6:34:77:09 Speed:10000 Mtu:8901} {Name:genev_sys_6081 MacAddress:a2:63:1b:26:79:8a Speed:0 Mtu:65000} {Name:ovn-k8s-mp0 MacAddress:0a:58:0a:85:00:02 Speed:0 Mtu:8901} {Name:ovs-system MacAddress:36:ea:c8:db:7a:c5 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164337152 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Oct 30 21:50:19.500499 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.500325    5510 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Oct 30 21:50:19.500499 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.500367    5510 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Oct 30 21:50:19.500714 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.500688    5510 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Oct 30 21:50:19.500839 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.500715    5510 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-136-210.us-west-2.compute.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Oct 30 21:50:19.500885 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.500841    5510 topology_manager.go:143] "Creating topology manager with none policy"
Oct 30 21:50:19.500885 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.500849    5510 container_manager_linux.go:311] "Creating device plugin manager"
Oct 30 21:50:19.500885 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.500860    5510 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 30 21:50:19.500885 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.500878    5510 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 30 21:50:19.501000 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.500944    5510 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager"
Oct 30 21:50:19.501101 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.501091    5510 state_mem.go:41] "Initialized" logger="CPUManager state memory"
Oct 30 21:50:19.501197 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.501188    5510 server.go:1235] "Using root directory" path="/var/lib/kubelet"
Oct 30 21:50:19.501228 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.501223    5510 kubelet.go:493] "Attempting to sync node with API server"
Oct 30 21:50:19.501255 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.501234    5510 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests"
Oct 30 21:50:19.501255 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.501244    5510 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Oct 30 21:50:19.501255 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.501253    5510 kubelet.go:405] "Adding apiserver pod source"
Oct 30 21:50:19.501347 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.501261    5510 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Oct 30 21:50:19.503061 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.502746    5510 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint"
Oct 30 21:50:19.503061 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.502766    5510 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 30 21:50:19.503552 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.503529    5510 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1"
Oct 30 21:50:19.506373 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.506359    5510 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint"
Oct 30 21:50:19.506373 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.506373    5510 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 30 21:50:19.506466 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.506456    5510 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-server-current.pem"
Oct 30 21:50:19.506693 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.506681    5510 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Oct 30 21:50:19.506725 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.506711    5510 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Oct 30 21:50:19.506809 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.506800    5510 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Oct 30 21:50:19.506838 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.506815    5510 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Oct 30 21:50:19.506838 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.506822    5510 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Oct 30 21:50:19.506838 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.506828    5510 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Oct 30 21:50:19.506838 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.506833    5510 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Oct 30 21:50:19.506838 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.506838    5510 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Oct 30 21:50:19.506976 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.506845    5510 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Oct 30 21:50:19.506976 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.506850    5510 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Oct 30 21:50:19.506976 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.506856    5510 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Oct 30 21:50:19.506976 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.506862    5510 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Oct 30 21:50:19.506976 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.506878    5510 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Oct 30 21:50:19.506976 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.506887    5510 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Oct 30 21:50:19.506976 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.506916    5510 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Oct 30 21:50:19.506976 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.506921    5510 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Oct 30 21:50:19.510122 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.510110    5510 server.go:1263] "Started kubelet"
Oct 30 21:50:19.510518 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.510202    5510 server.go:182] "Starting to listen" address="0.0.0.0" port=10250
Oct 30 21:50:19.510518 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.510237    5510 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Oct 30 21:50:19.510518 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.510310    5510 server_v1.go:49] "podresources" method="list" useActivePods=true
Oct 30 21:50:19.510714 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.510602    5510 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Oct 30 21:50:19.510609 ip-10-0-136-210 systemd[1]: Started Kubernetes Kubelet.
Oct 30 21:50:19.511471 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.511452    5510 server.go:317] "Adding debug handlers to kubelet server"
Oct 30 21:50:19.513743 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.513726    5510 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer"
Oct 30 21:50:19.515781 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.513756    5510 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Oct 30 21:50:19.515781 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.513804    5510 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-29 21:44:09 +0000 UTC" deadline="2028-04-06 01:02:07.914094141 +0000 UTC"
Oct 30 21:50:19.515781 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.513835    5510 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12555h11m48.4002623s"
Oct 30 21:50:19.515781 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:19.513857    5510 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-136-210.us-west-2.compute.internal\" not found"
Oct 30 21:50:19.515781 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.514496    5510 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Oct 30 21:50:19.515781 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.513826    5510 volume_manager.go:309] "The desired_state_of_world populator starts"
Oct 30 21:50:19.515781 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.514540    5510 factory.go:55] Registering systemd factory
Oct 30 21:50:19.515781 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.514556    5510 factory.go:223] Registration of the systemd container factory successfully
Oct 30 21:50:19.515781 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.514545    5510 volume_manager.go:311] "Starting Kubelet Volume Manager"
Oct 30 21:50:19.519983 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.517646    5510 factory.go:153] Registering CRI-O factory
Oct 30 21:50:19.519983 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.517681    5510 factory.go:223] Registration of the crio container factory successfully
Oct 30 21:50:19.519983 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.517745    5510 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Oct 30 21:50:19.519983 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.517763    5510 factory.go:103] Registering Raw factory
Oct 30 21:50:19.519983 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.517779    5510 manager.go:1199] Started watching for new ooms in manager
Oct 30 21:50:19.519983 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.518525    5510 manager.go:319] Starting recovery of all containers
Oct 30 21:50:19.519983 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.519588    5510 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2fe50918-ee11-4d17-b0fe-27c7898912c6" volumeName="kubernetes.io/projected/2fe50918-ee11-4d17-b0fe-27c7898912c6-kube-api-access-25ggc" seLinuxMountContext=""
Oct 30 21:50:19.519983 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.519626    5510 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="31a389bb-f36f-4e53-9654-5f4648dae38a" volumeName="kubernetes.io/configmap/31a389bb-f36f-4e53-9654-5f4648dae38a-service-ca" seLinuxMountContext=""
Oct 30 21:50:19.519983 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.519649    5510 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7d6190d5-6574-4704-97cb-2241398548e3" volumeName="kubernetes.io/configmap/7d6190d5-6574-4704-97cb-2241398548e3-kube-rbac-proxy-cm" seLinuxMountContext=""
Oct 30 21:50:19.519983 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.519679    5510 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c8b13e55-b3c7-4656-9845-16749704265f" volumeName="kubernetes.io/projected/c8b13e55-b3c7-4656-9845-16749704265f-kube-api-access-v4d5j" seLinuxMountContext=""
Oct 30 21:50:19.519983 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.519699    5510 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d5e5ac6e-d37d-457d-9ca7-93065f5925ff" volumeName="kubernetes.io/configmap/d5e5ac6e-d37d-457d-9ca7-93065f5925ff-ovnkube-script-lib" seLinuxMountContext=""
Oct 30 21:50:19.519983 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.519714    5510 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="31a389bb-f36f-4e53-9654-5f4648dae38a" volumeName="kubernetes.io/empty-dir/31a389bb-f36f-4e53-9654-5f4648dae38a-tmp" seLinuxMountContext=""
Oct 30 21:50:19.519983 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.519732    5510 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="31a389bb-f36f-4e53-9654-5f4648dae38a" volumeName="kubernetes.io/secret/31a389bb-f36f-4e53-9654-5f4648dae38a-console-serving-cert" seLinuxMountContext=""
Oct 30 21:50:19.519983 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.519747    5510 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0ab70415-9f1d-4988-a228-df6ed621f0d8" volumeName="kubernetes.io/configmap/0ab70415-9f1d-4988-a228-df6ed621f0d8-iptables-alerter-script" seLinuxMountContext=""
Oct 30 21:50:19.519983 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.519764    5510 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0e0d17a1-7742-42b0-9d9b-740bb945e454" volumeName="kubernetes.io/configmap/0e0d17a1-7742-42b0-9d9b-740bb945e454-telemeter-trusted-ca-bundle" seLinuxMountContext=""
Oct 30 21:50:19.519983 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.519780    5510 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="31a389bb-f36f-4e53-9654-5f4648dae38a" volumeName="kubernetes.io/configmap/31a389bb-f36f-4e53-9654-5f4648dae38a-trusted-ca-bundle" seLinuxMountContext=""
Oct 30 21:50:19.519983 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.519800    5510 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="42d54b8a-4f48-46a7-8a34-1f0f29cd1602" volumeName="kubernetes.io/empty-dir/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-tmp" seLinuxMountContext=""
Oct 30 21:50:19.519983 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.519820    5510 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8c88b730-5cf1-4434-950f-4a3aa8f739cd" volumeName="kubernetes.io/secret/8c88b730-5cf1-4434-950f-4a3aa8f739cd-secret-grpc-tls" seLinuxMountContext=""
Oct 30 21:50:19.519983 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.519833    5510 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8f2f95c4-21b2-477b-a2cb-7c8590ec340d" volumeName="kubernetes.io/empty-dir/8f2f95c4-21b2-477b-a2cb-7c8590ec340d-tmp" seLinuxMountContext=""
Oct 30 21:50:19.519983 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.519847    5510 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0e0d17a1-7742-42b0-9d9b-740bb945e454" volumeName="kubernetes.io/configmap/0e0d17a1-7742-42b0-9d9b-740bb945e454-metrics-client-ca" seLinuxMountContext=""
Oct 30 21:50:19.519983 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.519870    5510 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3abd55eb-d1d8-4fdf-b82c-d35894b652ac" volumeName="kubernetes.io/configmap/3abd55eb-d1d8-4fdf-b82c-d35894b652ac-metrics-client-ca" seLinuxMountContext=""
Oct 30 21:50:19.519983 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.519890    5510 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3abd55eb-d1d8-4fdf-b82c-d35894b652ac" volumeName="kubernetes.io/projected/3abd55eb-d1d8-4fdf-b82c-d35894b652ac-kube-api-access-f2qbm" seLinuxMountContext=""
Oct 30 21:50:19.519983 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.519910    5510 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8f2f95c4-21b2-477b-a2cb-7c8590ec340d" volumeName="kubernetes.io/empty-dir/8f2f95c4-21b2-477b-a2cb-7c8590ec340d-etc-tuned" seLinuxMountContext=""
Oct 30 21:50:19.519983 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.519922    5510 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d5e5ac6e-d37d-457d-9ca7-93065f5925ff" volumeName="kubernetes.io/configmap/d5e5ac6e-d37d-457d-9ca7-93065f5925ff-env-overrides" seLinuxMountContext=""
Oct 30 21:50:19.519983 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.519939    5510 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f69b6a2e-d33f-4cc8-8e0a-598b20948f38" volumeName="kubernetes.io/secret/f69b6a2e-d33f-4cc8-8e0a-598b20948f38-agent-certs" seLinuxMountContext=""
Oct 30 21:50:19.519983 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.519950    5510 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2fe50918-ee11-4d17-b0fe-27c7898912c6" volumeName="kubernetes.io/secret/2fe50918-ee11-4d17-b0fe-27c7898912c6-metrics-tls" seLinuxMountContext=""
Oct 30 21:50:19.519983 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.519966    5510 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="42d54b8a-4f48-46a7-8a34-1f0f29cd1602" volumeName="kubernetes.io/configmap/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-service-ca" seLinuxMountContext=""
Oct 30 21:50:19.519983 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.519977    5510 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="31a389bb-f36f-4e53-9654-5f4648dae38a" volumeName="kubernetes.io/configmap/31a389bb-f36f-4e53-9654-5f4648dae38a-console-config" seLinuxMountContext=""
Oct 30 21:50:19.519983 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.519990    5510 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="42d54b8a-4f48-46a7-8a34-1f0f29cd1602" volumeName="kubernetes.io/configmap/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-trusted-ca-bundle" seLinuxMountContext=""
Oct 30 21:50:19.521481 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.520006    5510 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="73c208eb-ffc0-46f2-8bd4-05ade5adb529" volumeName="kubernetes.io/projected/73c208eb-ffc0-46f2-8bd4-05ade5adb529-kube-api-access-fcjg4" seLinuxMountContext=""
Oct 30 21:50:19.521481 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.520017    5510 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d414339d-1a64-4a2a-ad15-805b71ebfd1b" volumeName="kubernetes.io/configmap/d414339d-1a64-4a2a-ad15-805b71ebfd1b-metrics-client-ca" seLinuxMountContext=""
Oct 30 21:50:19.521481 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.520032    5510 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d414339d-1a64-4a2a-ad15-805b71ebfd1b" volumeName="kubernetes.io/configmap/d414339d-1a64-4a2a-ad15-805b71ebfd1b-node-exporter-accelerators-collector-config" seLinuxMountContext=""
Oct 30 21:50:19.521481 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.520044    5510 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d414339d-1a64-4a2a-ad15-805b71ebfd1b" volumeName="kubernetes.io/secret/d414339d-1a64-4a2a-ad15-805b71ebfd1b-node-exporter-kube-rbac-proxy-config" seLinuxMountContext=""
Oct 30 21:50:19.521481 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.520056    5510 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d8dc5e17-81b4-413a-a48f-6175d0ed3ec0" volumeName="kubernetes.io/secret/d8dc5e17-81b4-413a-a48f-6175d0ed3ec0-cert" seLinuxMountContext=""
Oct 30 21:50:19.521481 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.520083    5510 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="db4b06a3-45e5-4d45-9d7a-407d117d9454" volumeName="kubernetes.io/projected/db4b06a3-45e5-4d45-9d7a-407d117d9454-kube-api-access-9krk9" seLinuxMountContext=""
Oct 30 21:50:19.521481 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.520101    5510 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="04465eda-e6db-42d3-8d5f-c9be668038a9" volumeName="kubernetes.io/configmap/04465eda-e6db-42d3-8d5f-c9be668038a9-serviceca" seLinuxMountContext=""
Oct 30 21:50:19.521481 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.520126    5510 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0ab70415-9f1d-4988-a228-df6ed621f0d8" volumeName="kubernetes.io/projected/0ab70415-9f1d-4988-a228-df6ed621f0d8-kube-api-access-td6rr" seLinuxMountContext=""
Oct 30 21:50:19.521481 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.520139    5510 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="31a389bb-f36f-4e53-9654-5f4648dae38a" volumeName="kubernetes.io/configmap/31a389bb-f36f-4e53-9654-5f4648dae38a-oauth-serving-cert" seLinuxMountContext=""
Oct 30 21:50:19.521481 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.520156    5510 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="42d54b8a-4f48-46a7-8a34-1f0f29cd1602" volumeName="kubernetes.io/configmap/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-oauth-serving-cert" seLinuxMountContext=""
Oct 30 21:50:19.521481 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.520168    5510 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d5e5ac6e-d37d-457d-9ca7-93065f5925ff" volumeName="kubernetes.io/secret/d5e5ac6e-d37d-457d-9ca7-93065f5925ff-ovn-node-metrics-cert" seLinuxMountContext=""
Oct 30 21:50:19.521481 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.520185    5510 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="42d54b8a-4f48-46a7-8a34-1f0f29cd1602" volumeName="kubernetes.io/projected/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-kube-api-access-5c4fj" seLinuxMountContext=""
Oct 30 21:50:19.521481 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.520198    5510 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="52eb01f8-e86a-4458-8875-639394a87b30" volumeName="kubernetes.io/projected/52eb01f8-e86a-4458-8875-639394a87b30-kube-api-access-zxwjt" seLinuxMountContext=""
Oct 30 21:50:19.521481 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.520210    5510 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8c88b730-5cf1-4434-950f-4a3aa8f739cd" volumeName="kubernetes.io/configmap/8c88b730-5cf1-4434-950f-4a3aa8f739cd-metrics-client-ca" seLinuxMountContext=""
Oct 30 21:50:19.521481 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.520227    5510 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8c88b730-5cf1-4434-950f-4a3aa8f739cd" volumeName="kubernetes.io/projected/8c88b730-5cf1-4434-950f-4a3aa8f739cd-kube-api-access-ttgkm" seLinuxMountContext=""
Oct 30 21:50:19.521481 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.520241    5510 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8f2f95c4-21b2-477b-a2cb-7c8590ec340d" volumeName="kubernetes.io/projected/8f2f95c4-21b2-477b-a2cb-7c8590ec340d-kube-api-access-t7ldj" seLinuxMountContext=""
Oct 30 21:50:19.521481 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.520258    5510 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2fe50918-ee11-4d17-b0fe-27c7898912c6" volumeName="kubernetes.io/configmap/2fe50918-ee11-4d17-b0fe-27c7898912c6-config-volume" seLinuxMountContext=""
Oct 30 21:50:19.521481 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.520271    5510 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0e0d17a1-7742-42b0-9d9b-740bb945e454" volumeName="kubernetes.io/secret/0e0d17a1-7742-42b0-9d9b-740bb945e454-secret-telemeter-client" seLinuxMountContext=""
Oct 30 21:50:19.521481 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.520283    5510 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="42d54b8a-4f48-46a7-8a34-1f0f29cd1602" volumeName="kubernetes.io/secret/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-console-oauth-config" seLinuxMountContext=""
Oct 30 21:50:19.521481 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.520299    5510 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="52eb01f8-e86a-4458-8875-639394a87b30" volumeName="kubernetes.io/secret/52eb01f8-e86a-4458-8875-639394a87b30-metrics-certs" seLinuxMountContext=""
Oct 30 21:50:19.521481 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.520312    5510 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8c88b730-5cf1-4434-950f-4a3aa8f739cd" volumeName="kubernetes.io/secret/8c88b730-5cf1-4434-950f-4a3aa8f739cd-secret-thanos-querier-kube-rbac-proxy-web" seLinuxMountContext=""
Oct 30 21:50:19.521481 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.520327    5510 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d414339d-1a64-4a2a-ad15-805b71ebfd1b" volumeName="kubernetes.io/projected/d414339d-1a64-4a2a-ad15-805b71ebfd1b-kube-api-access-rq9p9" seLinuxMountContext=""
Oct 30 21:50:19.521481 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.520339    5510 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d5e5ac6e-d37d-457d-9ca7-93065f5925ff" volumeName="kubernetes.io/projected/d5e5ac6e-d37d-457d-9ca7-93065f5925ff-kube-api-access-z2p79" seLinuxMountContext=""
Oct 30 21:50:19.521481 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.520352    5510 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="31a389bb-f36f-4e53-9654-5f4648dae38a" volumeName="kubernetes.io/projected/31a389bb-f36f-4e53-9654-5f4648dae38a-kube-api-access-rb2wf" seLinuxMountContext=""
Oct 30 21:50:19.521481 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.520368    5510 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="42d54b8a-4f48-46a7-8a34-1f0f29cd1602" volumeName="kubernetes.io/secret/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-console-serving-cert" seLinuxMountContext=""
Oct 30 21:50:19.521481 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.520381    5510 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8883cb69-54c2-437c-a92e-3bc38547027f" volumeName="kubernetes.io/secret/8883cb69-54c2-437c-a92e-3bc38547027f-monitoring-plugin-cert" seLinuxMountContext=""
Oct 30 21:50:19.521481 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.520398    5510 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8c88b730-5cf1-4434-950f-4a3aa8f739cd" volumeName="kubernetes.io/secret/8c88b730-5cf1-4434-950f-4a3aa8f739cd-secret-thanos-querier-kube-rbac-proxy-rules" seLinuxMountContext=""
Oct 30 21:50:19.521481 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.520440    5510 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c8b13e55-b3c7-4656-9845-16749704265f" volumeName="kubernetes.io/configmap/c8b13e55-b3c7-4656-9845-16749704265f-multus-daemon-config" seLinuxMountContext=""
Oct 30 21:50:19.521481 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.520459    5510 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2fe50918-ee11-4d17-b0fe-27c7898912c6" volumeName="kubernetes.io/empty-dir/2fe50918-ee11-4d17-b0fe-27c7898912c6-tmp-dir" seLinuxMountContext=""
Oct 30 21:50:19.521481 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.520474    5510 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7d6190d5-6574-4704-97cb-2241398548e3" volumeName="kubernetes.io/empty-dir/7d6190d5-6574-4704-97cb-2241398548e3-data-volume" seLinuxMountContext=""
Oct 30 21:50:19.521481 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.520487    5510 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7d6190d5-6574-4704-97cb-2241398548e3" volumeName="kubernetes.io/projected/7d6190d5-6574-4704-97cb-2241398548e3-kube-api-access-ggf4d" seLinuxMountContext=""
Oct 30 21:50:19.521481 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.520505    5510 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8c88b730-5cf1-4434-950f-4a3aa8f739cd" volumeName="kubernetes.io/secret/8c88b730-5cf1-4434-950f-4a3aa8f739cd-secret-thanos-querier-kube-rbac-proxy" seLinuxMountContext=""
Oct 30 21:50:19.521481 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.520517    5510 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e810bcd1-78da-409f-b86e-08c47031939e" volumeName="kubernetes.io/secret/e810bcd1-78da-409f-b86e-08c47031939e-original-pull-secret" seLinuxMountContext=""
Oct 30 21:50:19.521481 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.520549    5510 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0e0d17a1-7742-42b0-9d9b-740bb945e454" volumeName="kubernetes.io/configmap/0e0d17a1-7742-42b0-9d9b-740bb945e454-serving-certs-ca-bundle" seLinuxMountContext=""
Oct 30 21:50:19.521481 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.520566    5510 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0e0d17a1-7742-42b0-9d9b-740bb945e454" volumeName="kubernetes.io/projected/0e0d17a1-7742-42b0-9d9b-740bb945e454-kube-api-access-9gzpz" seLinuxMountContext=""
Oct 30 21:50:19.521481 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.520579    5510 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8c88b730-5cf1-4434-950f-4a3aa8f739cd" volumeName="kubernetes.io/secret/8c88b730-5cf1-4434-950f-4a3aa8f739cd-secret-thanos-querier-kube-rbac-proxy-metrics" seLinuxMountContext=""
Oct 30 21:50:19.521481 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.520597    5510 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c8b13e55-b3c7-4656-9845-16749704265f" volumeName="kubernetes.io/configmap/c8b13e55-b3c7-4656-9845-16749704265f-cni-binary-copy" seLinuxMountContext=""
Oct 30 21:50:19.521481 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.520609    5510 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d8dc5e17-81b4-413a-a48f-6175d0ed3ec0" volumeName="kubernetes.io/projected/d8dc5e17-81b4-413a-a48f-6175d0ed3ec0-kube-api-access-vh5d6" seLinuxMountContext=""
Oct 30 21:50:19.521481 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.520622    5510 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="dc220f0e-8a21-4d7f-96b3-e7055c242f42" volumeName="kubernetes.io/configmap/dc220f0e-8a21-4d7f-96b3-e7055c242f42-cni-sysctl-allowlist" seLinuxMountContext=""
Oct 30 21:50:19.521481 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.520638    5510 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0e0d17a1-7742-42b0-9d9b-740bb945e454" volumeName="kubernetes.io/secret/0e0d17a1-7742-42b0-9d9b-740bb945e454-telemeter-client-tls" seLinuxMountContext=""
Oct 30 21:50:19.521481 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.520669    5510 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8c88b730-5cf1-4434-950f-4a3aa8f739cd" volumeName="kubernetes.io/secret/8c88b730-5cf1-4434-950f-4a3aa8f739cd-secret-thanos-querier-tls" seLinuxMountContext=""
Oct 30 21:50:19.523362 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.520689    5510 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d414339d-1a64-4a2a-ad15-805b71ebfd1b" volumeName="kubernetes.io/secret/d414339d-1a64-4a2a-ad15-805b71ebfd1b-node-exporter-tls" seLinuxMountContext=""
Oct 30 21:50:19.523362 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.522224    5510 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d5e5ac6e-d37d-457d-9ca7-93065f5925ff" volumeName="kubernetes.io/configmap/d5e5ac6e-d37d-457d-9ca7-93065f5925ff-ovnkube-config" seLinuxMountContext=""
Oct 30 21:50:19.523362 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.522245    5510 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="dc220f0e-8a21-4d7f-96b3-e7055c242f42" volumeName="kubernetes.io/projected/dc220f0e-8a21-4d7f-96b3-e7055c242f42-kube-api-access-tq25h" seLinuxMountContext=""
Oct 30 21:50:19.523362 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.522265    5510 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3abd55eb-d1d8-4fdf-b82c-d35894b652ac" volumeName="kubernetes.io/secret/3abd55eb-d1d8-4fdf-b82c-d35894b652ac-openshift-state-metrics-tls" seLinuxMountContext=""
Oct 30 21:50:19.523362 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.522277    5510 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d414339d-1a64-4a2a-ad15-805b71ebfd1b" volumeName="kubernetes.io/empty-dir/d414339d-1a64-4a2a-ad15-805b71ebfd1b-node-exporter-textfile" seLinuxMountContext=""
Oct 30 21:50:19.523362 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.522293    5510 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="04465eda-e6db-42d3-8d5f-c9be668038a9" volumeName="kubernetes.io/projected/04465eda-e6db-42d3-8d5f-c9be668038a9-kube-api-access-hrxwg" seLinuxMountContext=""
Oct 30 21:50:19.523362 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.522306    5510 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="42d54b8a-4f48-46a7-8a34-1f0f29cd1602" volumeName="kubernetes.io/configmap/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-console-config" seLinuxMountContext=""
Oct 30 21:50:19.523362 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.523245    5510 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8883cb69-54c2-437c-a92e-3bc38547027f" volumeName="kubernetes.io/projected/8883cb69-54c2-437c-a92e-3bc38547027f-kube-api-access-b2929" seLinuxMountContext=""
Oct 30 21:50:19.523362 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.523275    5510 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="dc220f0e-8a21-4d7f-96b3-e7055c242f42" volumeName="kubernetes.io/configmap/dc220f0e-8a21-4d7f-96b3-e7055c242f42-cni-binary-copy" seLinuxMountContext=""
Oct 30 21:50:19.523362 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.523288    5510 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0e0d17a1-7742-42b0-9d9b-740bb945e454" volumeName="kubernetes.io/secret/0e0d17a1-7742-42b0-9d9b-740bb945e454-federate-client-tls" seLinuxMountContext=""
Oct 30 21:50:19.523362 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.523300    5510 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="31a389bb-f36f-4e53-9654-5f4648dae38a" volumeName="kubernetes.io/secret/31a389bb-f36f-4e53-9654-5f4648dae38a-console-oauth-config" seLinuxMountContext=""
Oct 30 21:50:19.523362 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.523312    5510 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7d6190d5-6574-4704-97cb-2241398548e3" volumeName="kubernetes.io/secret/7d6190d5-6574-4704-97cb-2241398548e3-insights-runtime-extractor-tls" seLinuxMountContext=""
Oct 30 21:50:19.523362 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.523324    5510 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d7c355fa-2141-463e-8cbc-bae3c44a16f1" volumeName="kubernetes.io/projected/d7c355fa-2141-463e-8cbc-bae3c44a16f1-kube-api-access-vg89l" seLinuxMountContext=""
Oct 30 21:50:19.523362 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.523336    5510 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="db4b06a3-45e5-4d45-9d7a-407d117d9454" volumeName="kubernetes.io/empty-dir/db4b06a3-45e5-4d45-9d7a-407d117d9454-tmp-dir" seLinuxMountContext=""
Oct 30 21:50:19.523362 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.523347    5510 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f69b6a2e-d33f-4cc8-8e0a-598b20948f38" volumeName="kubernetes.io/configmap/f69b6a2e-d33f-4cc8-8e0a-598b20948f38-konnectivity-ca" seLinuxMountContext=""
Oct 30 21:50:19.523362 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.523358    5510 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0e0d17a1-7742-42b0-9d9b-740bb945e454" volumeName="kubernetes.io/secret/0e0d17a1-7742-42b0-9d9b-740bb945e454-secret-telemeter-client-kube-rbac-proxy-config" seLinuxMountContext=""
Oct 30 21:50:19.523362 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.523370    5510 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3abd55eb-d1d8-4fdf-b82c-d35894b652ac" volumeName="kubernetes.io/secret/3abd55eb-d1d8-4fdf-b82c-d35894b652ac-openshift-state-metrics-kube-rbac-proxy-config" seLinuxMountContext=""
Oct 30 21:50:19.524103 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.523380    5510 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8c88b730-5cf1-4434-950f-4a3aa8f739cd" volumeName="kubernetes.io/empty-dir/8c88b730-5cf1-4434-950f-4a3aa8f739cd-tmp" seLinuxMountContext=""
Oct 30 21:50:19.524103 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.523390    5510 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="dc220f0e-8a21-4d7f-96b3-e7055c242f42" volumeName="kubernetes.io/configmap/dc220f0e-8a21-4d7f-96b3-e7055c242f42-whereabouts-flatfile-configmap" seLinuxMountContext=""
Oct 30 21:50:19.524103 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.523401    5510 reconstruct.go:97] "Volume reconstruction finished"
Oct 30 21:50:19.524103 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.523408    5510 reconciler.go:29] "Reconciler: start to sync state"
Oct 30 21:50:19.543309 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.543263    5510 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Oct 30 21:50:19.544941 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.544921    5510 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Oct 30 21:50:19.545092 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.545082    5510 status_manager.go:249] "Starting to sync pod status with apiserver"
Oct 30 21:50:19.545174 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.545166    5510 kubelet.go:2521] "Starting kubelet main sync loop"
Oct 30 21:50:19.545299 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:19.545280    5510 kubelet.go:2545] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]"
Oct 30 21:50:19.565751 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.565713    5510 generic.go:356] "Generic (PLEG): container finished" podID="d414339d-1a64-4a2a-ad15-805b71ebfd1b" containerID="c9e285756bcb8985bd79e7662e7dbea3151694964916f6717ec31d02a33a4b43" exitCode=0
Oct 30 21:50:19.568920 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.568890    5510 generic.go:356] "Generic (PLEG): container finished" podID="dc220f0e-8a21-4d7f-96b3-e7055c242f42" containerID="5609661adcd1a00bf7e06cdf2ed4015b56e6d563020fdb03aded3854573f531d" exitCode=0
Oct 30 21:50:19.568920 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.568917    5510 generic.go:356] "Generic (PLEG): container finished" podID="dc220f0e-8a21-4d7f-96b3-e7055c242f42" containerID="78543d599edd4758c0436abb669aa311f082e55f713b57ab479ebddb035a78cc" exitCode=0
Oct 30 21:50:19.569047 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.568925    5510 generic.go:356] "Generic (PLEG): container finished" podID="dc220f0e-8a21-4d7f-96b3-e7055c242f42" containerID="bec9121f537bc927169b50f8de94a73a09a472c6289bf5d6f1b23300d9d9abe7" exitCode=0
Oct 30 21:50:19.569047 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.568932    5510 generic.go:356] "Generic (PLEG): container finished" podID="dc220f0e-8a21-4d7f-96b3-e7055c242f42" containerID="523893fe178fa198884903d08e16609ad04a306f22bf92f6c1cd30f9951ca09f" exitCode=0
Oct 30 21:50:19.569047 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.568937    5510 generic.go:356] "Generic (PLEG): container finished" podID="dc220f0e-8a21-4d7f-96b3-e7055c242f42" containerID="e39db2e6b3e23bfcd306c9f3a561f3eb57cd334e51521b9cb5cfd411da5980ec" exitCode=0
Oct 30 21:50:19.569047 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.568942    5510 generic.go:356] "Generic (PLEG): container finished" podID="dc220f0e-8a21-4d7f-96b3-e7055c242f42" containerID="9348fc680b899ad9da756ea1029e7953ef9fc6c9f44a85be282c7e73eb47d028" exitCode=0
Oct 30 21:50:19.572628 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.572587    5510 generic.go:356] "Generic (PLEG): container finished" podID="d91a9843014f769d901fa6aba545e397" containerID="3415a45fd4be33eac12c60f14895f6c19f67a6765533f5e8df07bc8b45eeaa3a" exitCode=0
Oct 30 21:50:19.613937 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:19.613908    5510 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-136-210.us-west-2.compute.internal\" not found"
Oct 30 21:50:19.620620 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.620599    5510 manager.go:324] Recovery completed
Oct 30 21:50:19.641417 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.641362    5510 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 21:50:19.643256 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.643239    5510 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-210.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 21:50:19.643337 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.643269    5510 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-210.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 21:50:19.643337 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.643279    5510 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-210.us-west-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 21:50:19.646068 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.646047    5510 cpu_manager.go:226] "Starting" policy="none"
Oct 30 21:50:19.646068 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.646064    5510 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s"
Oct 30 21:50:19.646068 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.646065    5510 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 21:50:19.646259 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.646083    5510 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory"
Oct 30 21:50:19.646259 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:19.646063    5510 kubelet.go:2545] "Skipping pod synchronization" err="container runtime status check may not have completed yet"
Oct 30 21:50:19.646259 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.646232    5510 state_mem.go:94] "Updated default CPUSet" logger="CPUManager state checkpoint.CPUManager state memory" cpuSet=""
Oct 30 21:50:19.646259 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.646248    5510 state_mem.go:102] "Updated CPUSet assignments" logger="CPUManager state checkpoint.CPUManager state memory" assignments={}
Oct 30 21:50:19.646410 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.646266    5510 state_checkpoint.go:142] "restored state from checkpoint" logger="CPUManager state checkpoint"
Oct 30 21:50:19.646410 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.646271    5510 state_checkpoint.go:143] "defaultCPUSet" logger="CPUManager state checkpoint" defaultCpuSet=""
Oct 30 21:50:19.646410 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.646277    5510 policy_none.go:50] "Start"
Oct 30 21:50:19.646410 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.646285    5510 memory_manager.go:187] "Starting memorymanager" policy="None"
Oct 30 21:50:19.646410 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.646296    5510 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Oct 30 21:50:19.646562 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.646425    5510 state_mem.go:77] "Updated machine memory state" logger="Memory Manager state checkpoint"
Oct 30 21:50:19.646562 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.646432    5510 state_checkpoint.go:85] "State checkpoint: restored state from checkpoint" logger="Memory Manager state checkpoint"
Oct 30 21:50:19.646562 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.646437    5510 policy_none.go:44] "Start"
Oct 30 21:50:19.646960 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.646942    5510 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-210.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 21:50:19.647056 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.646969    5510 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-210.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 21:50:19.647056 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.646980    5510 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-210.us-west-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 21:50:19.651076 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.651052    5510 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"}
Oct 30 21:50:19.653669 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.653639    5510 manager.go:341] "Starting Device Plugin manager"
Oct 30 21:50:19.653754 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:19.653681    5510 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Oct 30 21:50:19.653754 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.653693    5510 server.go:87] "Starting device plugin registration server"
Oct 30 21:50:19.653889 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.653878    5510 eviction_manager.go:194] "Eviction manager: starting control loop"
Oct 30 21:50:19.653952 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.653891    5510 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Oct 30 21:50:19.654006 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.653990    5510 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Oct 30 21:50:19.654145 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.654102    5510 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Oct 30 21:50:19.654145 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.654146    5510 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts"
Oct 30 21:50:19.654265 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.654153    5510 plugin_manager.go:121] "Starting Kubelet Plugin Manager"
Oct 30 21:50:19.654312 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.654206    5510 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-10-30T21:50:19.654136468Z","UUID":"2ce60829-4f76-4ca6-a246-e3ea9b3b2b8c","Handler":null,"Name":"","Endpoint":""}
Oct 30 21:50:19.655441 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.655412    5510 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Oct 30 21:50:19.655441 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.655435    5510 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Oct 30 21:50:19.656584 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:19.656505    5510 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Oct 30 21:50:19.656584 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:19.656574    5510 eviction_manager.go:297] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-136-210.us-west-2.compute.internal\" not found"
Oct 30 21:50:19.704640 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.704614    5510 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 21:50:19.704770 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.704754    5510 reconstruct.go:205] "DevicePaths of reconstructed volumes updated"
Oct 30 21:50:19.704827 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.704785    5510 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 21:50:19.754107 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.754071    5510 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 21:50:19.754595 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.754567    5510 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 21:50:19.755182 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.755167    5510 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-210.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 21:50:19.755269 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.755195    5510 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-210.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 21:50:19.755269 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.755206    5510 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-210.us-west-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 21:50:19.755269 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.755248    5510 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-136-210.us-west-2.compute.internal"
Oct 30 21:50:19.797560 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.797541    5510 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 21:50:19.846530 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.846479    5510 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-210.us-west-2.compute.internal","kube-system/kube-apiserver-proxy-ip-10-0-136-210.us-west-2.compute.internal"]
Oct 30 21:50:19.847135 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.847089    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-210.us-west-2.compute.internal" event={"ID":"d91a9843014f769d901fa6aba545e397","Type":"ContainerStarted","Data":"8fcdfc2e2f635ddec2d7a89cb41f62bd819304fde0f88ea2b4f298ea9ccf1270"}
Oct 30 21:50:19.847257 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.847138    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-210.us-west-2.compute.internal" event={"ID":"d91a9843014f769d901fa6aba545e397","Type":"ContainerDied","Data":"3415a45fd4be33eac12c60f14895f6c19f67a6765533f5e8df07bc8b45eeaa3a"}
Oct 30 21:50:19.847257 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.847159    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-210.us-west-2.compute.internal" event={"ID":"d91a9843014f769d901fa6aba545e397","Type":"ContainerStarted","Data":"327a89302c3fd96103b398b276678511aec903499e1c71d72134532152be1296"}
Oct 30 21:50:19.847257 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.847168    5510 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-210.us-west-2.compute.internal"
Oct 30 21:50:19.847257 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.847175    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-210.us-west-2.compute.internal" event={"ID":"4492449458f4d650448405f5044e8d24","Type":"ContainerStarted","Data":"40c8a50d2211e2d2e13c659f3c218bbf16d947d371c5fed15f67fdc551f0f2b9"}
Oct 30 21:50:19.847257 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.847188    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-210.us-west-2.compute.internal" event={"ID":"4492449458f4d650448405f5044e8d24","Type":"ContainerStarted","Data":"c1581eb1a638205af61c045effda2bf3708d1f9f54df9b7a6c6c4d69436f7817"}
Oct 30 21:50:19.847399 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.847292    5510 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-210.us-west-2.compute.internal"
Oct 30 21:50:19.857520 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.857504    5510 kubelet_node_status.go:126] "Node was previously registered" node="ip-10-0-136-210.us-west-2.compute.internal"
Oct 30 21:50:19.857568 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.857560    5510 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-136-210.us-west-2.compute.internal"
Oct 30 21:50:19.907265 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.907191    5510 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 21:50:19.907265 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:19.907236    5510 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-136-210.us-west-2.compute.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-210.us-west-2.compute.internal"
Oct 30 21:50:19.907633 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:19.907617    5510 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 21:50:19.907711 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:19.907672    5510 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-136-210.us-west-2.compute.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-210.us-west-2.compute.internal"
Oct 30 21:50:20.005217 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.005180    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/d91a9843014f769d901fa6aba545e397-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-210.us-west-2.compute.internal\" (UID: \"d91a9843014f769d901fa6aba545e397\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-210.us-west-2.compute.internal"
Oct 30 21:50:20.005382 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.005228    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d91a9843014f769d901fa6aba545e397-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-210.us-west-2.compute.internal\" (UID: \"d91a9843014f769d901fa6aba545e397\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-210.us-west-2.compute.internal"
Oct 30 21:50:20.005382 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.005248    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/4492449458f4d650448405f5044e8d24-config\") pod \"kube-apiserver-proxy-ip-10-0-136-210.us-west-2.compute.internal\" (UID: \"4492449458f4d650448405f5044e8d24\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-210.us-west-2.compute.internal"
Oct 30 21:50:20.105936 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.105902    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/d91a9843014f769d901fa6aba545e397-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-210.us-west-2.compute.internal\" (UID: \"d91a9843014f769d901fa6aba545e397\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-210.us-west-2.compute.internal"
Oct 30 21:50:20.105936 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.105939    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d91a9843014f769d901fa6aba545e397-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-210.us-west-2.compute.internal\" (UID: \"d91a9843014f769d901fa6aba545e397\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-210.us-west-2.compute.internal"
Oct 30 21:50:20.106122 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.105964    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/4492449458f4d650448405f5044e8d24-config\") pod \"kube-apiserver-proxy-ip-10-0-136-210.us-west-2.compute.internal\" (UID: \"4492449458f4d650448405f5044e8d24\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-210.us-west-2.compute.internal"
Oct 30 21:50:20.106122 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.106005    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/d91a9843014f769d901fa6aba545e397-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-210.us-west-2.compute.internal\" (UID: \"d91a9843014f769d901fa6aba545e397\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-210.us-west-2.compute.internal"
Oct 30 21:50:20.106122 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.106019    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d91a9843014f769d901fa6aba545e397-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-210.us-west-2.compute.internal\" (UID: \"d91a9843014f769d901fa6aba545e397\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-210.us-west-2.compute.internal"
Oct 30 21:50:20.106122 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.106061    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/4492449458f4d650448405f5044e8d24-config\") pod \"kube-apiserver-proxy-ip-10-0-136-210.us-west-2.compute.internal\" (UID: \"4492449458f4d650448405f5044e8d24\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-210.us-west-2.compute.internal"
Oct 30 21:50:20.502145 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.502111    5510 apiserver.go:52] "Watching apiserver"
Oct 30 21:50:20.602158 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.602124    5510 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-210.us-west-2.compute.internal"
Oct 30 21:50:20.602339 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.602224    5510 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-210.us-west-2.compute.internal"
Oct 30 21:50:20.603086 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.603069    5510 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Oct 30 21:50:20.603620 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.603599    5510 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-77f4f76594-bfnw9","openshift-insights/insights-runtime-extractor-6gjbf","openshift-monitoring/node-exporter-mpsnt","kube-system/konnectivity-agent-5mw6n","kube-system/kube-apiserver-proxy-ip-10-0-136-210.us-west-2.compute.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-210.us-west-2.compute.internal","openshift-multus/network-metrics-daemon-vmrkz","openshift-dns/node-resolver-krtnr","openshift-monitoring/telemeter-client-bb95b88bf-g9f94","openshift-cluster-node-tuning-operator/tuned-ldvhf","openshift-console/console-6d5866d6cd-b6dpl","openshift-monitoring/thanos-querier-7c848fbc95-8jm99","openshift-dns/dns-default-w5ck5","openshift-monitoring/openshift-state-metrics-7bd7489957-h95jn","openshift-multus/multus-v8j9s","openshift-ingress-canary/ingress-canary-t8tbp","openshift-image-registry/node-ca-xkrx5","openshift-network-diagnostics/network-check-target-jmbng","openshift-network-operator/iptables-alerter-l9d98","openshift-ovn-kubernetes/ovnkube-node-wrxww","kube-system/global-pull-secret-syncer-68jrf","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cqxms","openshift-console/console-748df94584-88cjt","openshift-multus/multus-additional-cni-plugins-kggmc"]
Oct 30 21:50:20.655056 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.655015    5510 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Oct 30 21:50:20.655056 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.655033    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-nwhdt\""
Oct 30 21:50:20.655303 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.655082    5510 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Oct 30 21:50:20.655716 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.655699    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-dvh7z\""
Oct 30 21:50:20.661643 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.661630    5510 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Oct 30 21:50:20.665039 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.665026    5510 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 21:50:20.665087 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:20.665065    5510 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-136-210.us-west-2.compute.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-210.us-west-2.compute.internal"
Oct 30 21:50:20.666794 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.666766    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-hm4gw\""
Oct 30 21:50:20.667373 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.667361    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-9mttd\""
Oct 30 21:50:20.667700 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.667688    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Oct 30 21:50:20.667930 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.667903    5510 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Oct 30 21:50:20.668007 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.667975    5510 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 21:50:20.668007 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.668000    5510 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Oct 30 21:50:20.668108 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:20.668022    5510 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-136-210.us-west-2.compute.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-210.us-west-2.compute.internal"
Oct 30 21:50:20.668108 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.668039    5510 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Oct 30 21:50:20.668108 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.668055    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Oct 30 21:50:20.668108 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.668057    5510 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Oct 30 21:50:20.668108 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.668059    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Oct 30 21:50:20.668108 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.668092    5510 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Oct 30 21:50:20.668388 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.668204    5510 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Oct 30 21:50:20.668388 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.668236    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-6gwn9\""
Oct 30 21:50:20.668388 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.668245    5510 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Oct 30 21:50:20.668388 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.668254    5510 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Oct 30 21:50:20.668388 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.668266    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-gbc4r\""
Oct 30 21:50:20.668388 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.668337    5510 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Oct 30 21:50:20.668388 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.668341    5510 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Oct 30 21:50:20.668388 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.668358    5510 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Oct 30 21:50:20.668388 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.668386    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-7d2ft\""
Oct 30 21:50:20.668792 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.668378    5510 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Oct 30 21:50:20.668792 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.668359    5510 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Oct 30 21:50:20.668792 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.668440    5510 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Oct 30 21:50:20.668792 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.668441    5510 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Oct 30 21:50:20.668792 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.668458    5510 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Oct 30 21:50:20.668792 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.668443    5510 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Oct 30 21:50:20.668792 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.668457    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Oct 30 21:50:20.668792 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.668492    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-jbflf\""
Oct 30 21:50:20.668792 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.668536    5510 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Oct 30 21:50:20.668792 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.668588    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-92jf8\""
Oct 30 21:50:20.708712 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.708678    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d5e5ac6e-d37d-457d-9ca7-93065f5925ff-host-cni-bin\") pod \"ovnkube-node-wrxww\" (UID: \"d5e5ac6e-d37d-457d-9ca7-93065f5925ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrxww"
Oct 30 21:50:20.708799 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.708724    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e810bcd1-78da-409f-b86e-08c47031939e-kubelet-config\") pod \"global-pull-secret-syncer-68jrf\" (UID: \"e810bcd1-78da-409f-b86e-08c47031939e\") " pod="kube-system/global-pull-secret-syncer-68jrf"
Oct 30 21:50:20.708799 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.708759    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/8f2f95c4-21b2-477b-a2cb-7c8590ec340d-run\") pod \"tuned-ldvhf\" (UID: \"8f2f95c4-21b2-477b-a2cb-7c8590ec340d\") " pod="openshift-cluster-node-tuning-operator/tuned-ldvhf"
Oct 30 21:50:20.708799 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.708788    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8f2f95c4-21b2-477b-a2cb-7c8590ec340d-tmp\") pod \"tuned-ldvhf\" (UID: \"8f2f95c4-21b2-477b-a2cb-7c8590ec340d\") " pod="openshift-cluster-node-tuning-operator/tuned-ldvhf"
Oct 30 21:50:20.708906 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.708834    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/d5e5ac6e-d37d-457d-9ca7-93065f5925ff-ovnkube-script-lib\") pod \"ovnkube-node-wrxww\" (UID: \"d5e5ac6e-d37d-457d-9ca7-93065f5925ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrxww"
Oct 30 21:50:20.708906 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.708856    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-trusted-ca-bundle\") pod \"console-6d5866d6cd-b6dpl\" (UID: \"42d54b8a-4f48-46a7-8a34-1f0f29cd1602\") " pod="openshift-console/console-6d5866d6cd-b6dpl"
Oct 30 21:50:20.708906 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.708873    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d414339d-1a64-4a2a-ad15-805b71ebfd1b-node-exporter-tls\") pod \"node-exporter-mpsnt\" (UID: \"d414339d-1a64-4a2a-ad15-805b71ebfd1b\") " pod="openshift-monitoring/node-exporter-mpsnt"
Oct 30 21:50:20.708906 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.708891    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8f2f95c4-21b2-477b-a2cb-7c8590ec340d-tmp\") pod \"tuned-ldvhf\" (UID: \"8f2f95c4-21b2-477b-a2cb-7c8590ec340d\") " pod="openshift-cluster-node-tuning-operator/tuned-ldvhf"
Oct 30 21:50:20.709145 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.708894    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f69b6a2e-d33f-4cc8-8e0a-598b20948f38-agent-certs\") pod \"konnectivity-agent-5mw6n\" (UID: \"f69b6a2e-d33f-4cc8-8e0a-598b20948f38\") " pod="kube-system/konnectivity-agent-5mw6n"
Oct 30 21:50:20.709145 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.708957    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8f2f95c4-21b2-477b-a2cb-7c8590ec340d-var-lib-kubelet\") pod \"tuned-ldvhf\" (UID: \"8f2f95c4-21b2-477b-a2cb-7c8590ec340d\") " pod="openshift-cluster-node-tuning-operator/tuned-ldvhf"
Oct 30 21:50:20.709145 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:20.708973    5510 configmap.go:193] Couldn't get configMap openshift-console/trusted-ca-bundle: object "openshift-console"/"trusted-ca-bundle" not registered
Oct 30 21:50:20.709145 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.708984    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/d5e5ac6e-d37d-457d-9ca7-93065f5925ff-systemd-units\") pod \"ovnkube-node-wrxww\" (UID: \"d5e5ac6e-d37d-457d-9ca7-93065f5925ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrxww"
Oct 30 21:50:20.709145 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.709010    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/d5e5ac6e-d37d-457d-9ca7-93065f5925ff-env-overrides\") pod \"ovnkube-node-wrxww\" (UID: \"d5e5ac6e-d37d-457d-9ca7-93065f5925ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrxww"
Oct 30 21:50:20.709145 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:20.709058    5510 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-trusted-ca-bundle podName:42d54b8a-4f48-46a7-8a34-1f0f29cd1602 nodeName:}" failed. No retries permitted until 2026-10-30 21:50:21.209035499 +0000 UTC m=+1.769845504 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-trusted-ca-bundle") pod "console-6d5866d6cd-b6dpl" (UID: "42d54b8a-4f48-46a7-8a34-1f0f29cd1602") : object "openshift-console"/"trusted-ca-bundle" not registered
Oct 30 21:50:20.709145 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.709082    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/d414339d-1a64-4a2a-ad15-805b71ebfd1b-node-exporter-accelerators-collector-config\") pod \"node-exporter-mpsnt\" (UID: \"d414339d-1a64-4a2a-ad15-805b71ebfd1b\") " pod="openshift-monitoring/node-exporter-mpsnt"
Oct 30 21:50:20.709406 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.709174    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f69b6a2e-d33f-4cc8-8e0a-598b20948f38-agent-certs\") pod \"konnectivity-agent-5mw6n\" (UID: \"f69b6a2e-d33f-4cc8-8e0a-598b20948f38\") " pod="kube-system/konnectivity-agent-5mw6n"
Oct 30 21:50:20.709406 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.709178    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/d5e5ac6e-d37d-457d-9ca7-93065f5925ff-ovnkube-script-lib\") pod \"ovnkube-node-wrxww\" (UID: \"d5e5ac6e-d37d-457d-9ca7-93065f5925ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrxww"
Oct 30 21:50:20.709406 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.709198    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/8c88b730-5cf1-4434-950f-4a3aa8f739cd-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7c848fbc95-8jm99\" (UID: \"8c88b730-5cf1-4434-950f-4a3aa8f739cd\") " pod="openshift-monitoring/thanos-querier-7c848fbc95-8jm99"
Oct 30 21:50:20.709406 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.709221    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/d5e5ac6e-d37d-457d-9ca7-93065f5925ff-env-overrides\") pod \"ovnkube-node-wrxww\" (UID: \"d5e5ac6e-d37d-457d-9ca7-93065f5925ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrxww"
Oct 30 21:50:20.709406 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.709244    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/31a389bb-f36f-4e53-9654-5f4648dae38a-trusted-ca-bundle\") pod \"console-748df94584-88cjt\" (UID: \"31a389bb-f36f-4e53-9654-5f4648dae38a\") " pod="openshift-console/console-748df94584-88cjt"
Oct 30 21:50:20.709406 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.709318    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/db4b06a3-45e5-4d45-9d7a-407d117d9454-hosts-file\") pod \"node-resolver-krtnr\" (UID: \"db4b06a3-45e5-4d45-9d7a-407d117d9454\") " pod="openshift-dns/node-resolver-krtnr"
Oct 30 21:50:20.709406 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:20.709325    5510 configmap.go:193] Couldn't get configMap openshift-console/trusted-ca-bundle: object "openshift-console"/"trusted-ca-bundle" not registered
Oct 30 21:50:20.709406 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.709340    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/dc220f0e-8a21-4d7f-96b3-e7055c242f42-cni-binary-copy\") pod \"multus-additional-cni-plugins-kggmc\" (UID: \"dc220f0e-8a21-4d7f-96b3-e7055c242f42\") " pod="openshift-multus/multus-additional-cni-plugins-kggmc"
Oct 30 21:50:20.709406 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.709361    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9gzpz\" (UniqueName: \"kubernetes.io/projected/0e0d17a1-7742-42b0-9d9b-740bb945e454-kube-api-access-9gzpz\") pod \"telemeter-client-bb95b88bf-g9f94\" (UID: \"0e0d17a1-7742-42b0-9d9b-740bb945e454\") " pod="openshift-monitoring/telemeter-client-bb95b88bf-g9f94"
Oct 30 21:50:20.709406 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:20.709378    5510 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/31a389bb-f36f-4e53-9654-5f4648dae38a-trusted-ca-bundle podName:31a389bb-f36f-4e53-9654-5f4648dae38a nodeName:}" failed. No retries permitted until 2026-10-30 21:50:21.209361471 +0000 UTC m=+1.770171464 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/31a389bb-f36f-4e53-9654-5f4648dae38a-trusted-ca-bundle") pod "console-748df94584-88cjt" (UID: "31a389bb-f36f-4e53-9654-5f4648dae38a") : object "openshift-console"/"trusted-ca-bundle" not registered
Oct 30 21:50:20.709747 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.709411    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hrxwg\" (UniqueName: \"kubernetes.io/projected/04465eda-e6db-42d3-8d5f-c9be668038a9-kube-api-access-hrxwg\") pod \"node-ca-xkrx5\" (UID: \"04465eda-e6db-42d3-8d5f-c9be668038a9\") " pod="openshift-image-registry/node-ca-xkrx5"
Oct 30 21:50:20.709747 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.709444    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/d5e5ac6e-d37d-457d-9ca7-93065f5925ff-log-socket\") pod \"ovnkube-node-wrxww\" (UID: \"d5e5ac6e-d37d-457d-9ca7-93065f5925ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrxww"
Oct 30 21:50:20.709747 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.709472    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rq9p9\" (UniqueName: \"kubernetes.io/projected/d414339d-1a64-4a2a-ad15-805b71ebfd1b-kube-api-access-rq9p9\") pod \"node-exporter-mpsnt\" (UID: \"d414339d-1a64-4a2a-ad15-805b71ebfd1b\") " pod="openshift-monitoring/node-exporter-mpsnt"
Oct 30 21:50:20.709747 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.709497    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c8b13e55-b3c7-4656-9845-16749704265f-host-run-netns\") pod \"multus-v8j9s\" (UID: \"c8b13e55-b3c7-4656-9845-16749704265f\") " pod="openshift-multus/multus-v8j9s"
Oct 30 21:50:20.709747 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.709546    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/31a389bb-f36f-4e53-9654-5f4648dae38a-oauth-serving-cert\") pod \"console-748df94584-88cjt\" (UID: \"31a389bb-f36f-4e53-9654-5f4648dae38a\") " pod="openshift-console/console-748df94584-88cjt"
Oct 30 21:50:20.709747 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.709580    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/dc220f0e-8a21-4d7f-96b3-e7055c242f42-cni-binary-copy\") pod \"multus-additional-cni-plugins-kggmc\" (UID: \"dc220f0e-8a21-4d7f-96b3-e7055c242f42\") " pod="openshift-multus/multus-additional-cni-plugins-kggmc"
Oct 30 21:50:20.709747 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.709636    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d414339d-1a64-4a2a-ad15-805b71ebfd1b-metrics-client-ca\") pod \"node-exporter-mpsnt\" (UID: \"d414339d-1a64-4a2a-ad15-805b71ebfd1b\") " pod="openshift-monitoring/node-exporter-mpsnt"
Oct 30 21:50:20.709747 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:20.709688    5510 configmap.go:193] Couldn't get configMap openshift-console/oauth-serving-cert: object "openshift-console"/"oauth-serving-cert" not registered
Oct 30 21:50:20.709747 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:20.709738    5510 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/31a389bb-f36f-4e53-9654-5f4648dae38a-oauth-serving-cert podName:31a389bb-f36f-4e53-9654-5f4648dae38a nodeName:}" failed. No retries permitted until 2026-10-30 21:50:21.209727388 +0000 UTC m=+1.770537371 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "oauth-serving-cert" (UniqueName: "kubernetes.io/configmap/31a389bb-f36f-4e53-9654-5f4648dae38a-oauth-serving-cert") pod "console-748df94584-88cjt" (UID: "31a389bb-f36f-4e53-9654-5f4648dae38a") : object "openshift-console"/"oauth-serving-cert" not registered
Oct 30 21:50:20.710092 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.709753    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d7c355fa-2141-463e-8cbc-bae3c44a16f1-device-dir\") pod \"aws-ebs-csi-driver-node-cqxms\" (UID: \"d7c355fa-2141-463e-8cbc-bae3c44a16f1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cqxms"
Oct 30 21:50:20.710092 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.709773    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c8b13e55-b3c7-4656-9845-16749704265f-host-var-lib-kubelet\") pod \"multus-v8j9s\" (UID: \"c8b13e55-b3c7-4656-9845-16749704265f\") " pod="openshift-multus/multus-v8j9s"
Oct 30 21:50:20.710092 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.709787    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c8b13e55-b3c7-4656-9845-16749704265f-etc-kubernetes\") pod \"multus-v8j9s\" (UID: \"c8b13e55-b3c7-4656-9845-16749704265f\") " pod="openshift-multus/multus-v8j9s"
Oct 30 21:50:20.710092 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.709805    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e810bcd1-78da-409f-b86e-08c47031939e-original-pull-secret\") pod \"global-pull-secret-syncer-68jrf\" (UID: \"e810bcd1-78da-409f-b86e-08c47031939e\") " pod="kube-system/global-pull-secret-syncer-68jrf"
Oct 30 21:50:20.710092 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.709846    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/8f2f95c4-21b2-477b-a2cb-7c8590ec340d-etc-sysconfig\") pod \"tuned-ldvhf\" (UID: \"8f2f95c4-21b2-477b-a2cb-7c8590ec340d\") " pod="openshift-cluster-node-tuning-operator/tuned-ldvhf"
Oct 30 21:50:20.710092 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.709917    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/8f2f95c4-21b2-477b-a2cb-7c8590ec340d-etc-sysctl-conf\") pod \"tuned-ldvhf\" (UID: \"8f2f95c4-21b2-477b-a2cb-7c8590ec340d\") " pod="openshift-cluster-node-tuning-operator/tuned-ldvhf"
Oct 30 21:50:20.710092 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.709947    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8c88b730-5cf1-4434-950f-4a3aa8f739cd-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7c848fbc95-8jm99\" (UID: \"8c88b730-5cf1-4434-950f-4a3aa8f739cd\") " pod="openshift-monitoring/thanos-querier-7c848fbc95-8jm99"
Oct 30 21:50:20.710092 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.709969    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f69b6a2e-d33f-4cc8-8e0a-598b20948f38-konnectivity-ca\") pod \"konnectivity-agent-5mw6n\" (UID: \"f69b6a2e-d33f-4cc8-8e0a-598b20948f38\") " pod="kube-system/konnectivity-agent-5mw6n"
Oct 30 21:50:20.710092 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.710011    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/c8b13e55-b3c7-4656-9845-16749704265f-hostroot\") pod \"multus-v8j9s\" (UID: \"c8b13e55-b3c7-4656-9845-16749704265f\") " pod="openshift-multus/multus-v8j9s"
Oct 30 21:50:20.710092 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.710053    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e810bcd1-78da-409f-b86e-08c47031939e-original-pull-secret\") pod \"global-pull-secret-syncer-68jrf\" (UID: \"e810bcd1-78da-409f-b86e-08c47031939e\") " pod="kube-system/global-pull-secret-syncer-68jrf"
Oct 30 21:50:20.710092 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.710080    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8f2f95c4-21b2-477b-a2cb-7c8590ec340d-lib-modules\") pod \"tuned-ldvhf\" (UID: \"8f2f95c4-21b2-477b-a2cb-7c8590ec340d\") " pod="openshift-cluster-node-tuning-operator/tuned-ldvhf"
Oct 30 21:50:20.710428 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.710102    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f69b6a2e-d33f-4cc8-8e0a-598b20948f38-konnectivity-ca\") pod \"konnectivity-agent-5mw6n\" (UID: \"f69b6a2e-d33f-4cc8-8e0a-598b20948f38\") " pod="kube-system/konnectivity-agent-5mw6n"
Oct 30 21:50:20.710428 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.710101    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d5e5ac6e-d37d-457d-9ca7-93065f5925ff-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-wrxww\" (UID: \"d5e5ac6e-d37d-457d-9ca7-93065f5925ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrxww"
Oct 30 21:50:20.710428 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.710134    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8c88b730-5cf1-4434-950f-4a3aa8f739cd-secret-grpc-tls\") pod \"thanos-querier-7c848fbc95-8jm99\" (UID: \"8c88b730-5cf1-4434-950f-4a3aa8f739cd\") " pod="openshift-monitoring/thanos-querier-7c848fbc95-8jm99"
Oct 30 21:50:20.710428 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.710160    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t7ldj\" (UniqueName: \"kubernetes.io/projected/8f2f95c4-21b2-477b-a2cb-7c8590ec340d-kube-api-access-t7ldj\") pod \"tuned-ldvhf\" (UID: \"8f2f95c4-21b2-477b-a2cb-7c8590ec340d\") " pod="openshift-cluster-node-tuning-operator/tuned-ldvhf"
Oct 30 21:50:20.710428 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.710233    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d5e5ac6e-d37d-457d-9ca7-93065f5925ff-var-lib-openvswitch\") pod \"ovnkube-node-wrxww\" (UID: \"d5e5ac6e-d37d-457d-9ca7-93065f5925ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrxww"
Oct 30 21:50:20.710428 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.710252    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d5e5ac6e-d37d-457d-9ca7-93065f5925ff-etc-openvswitch\") pod \"ovnkube-node-wrxww\" (UID: \"d5e5ac6e-d37d-457d-9ca7-93065f5925ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrxww"
Oct 30 21:50:20.710428 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.710270    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-tmp\") pod \"console-6d5866d6cd-b6dpl\" (UID: \"42d54b8a-4f48-46a7-8a34-1f0f29cd1602\") " pod="openshift-console/console-6d5866d6cd-b6dpl"
Oct 30 21:50:20.710428 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.710293    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-td6rr\" (UniqueName: \"kubernetes.io/projected/0ab70415-9f1d-4988-a228-df6ed621f0d8-kube-api-access-td6rr\") pod \"iptables-alerter-l9d98\" (UID: \"0ab70415-9f1d-4988-a228-df6ed621f0d8\") " pod="openshift-network-operator/iptables-alerter-l9d98"
Oct 30 21:50:20.710428 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.710321    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d7c355fa-2141-463e-8cbc-bae3c44a16f1-etc-selinux\") pod \"aws-ebs-csi-driver-node-cqxms\" (UID: \"d7c355fa-2141-463e-8cbc-bae3c44a16f1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cqxms"
Oct 30 21:50:20.710428 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.710382    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7d6190d5-6574-4704-97cb-2241398548e3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6gjbf\" (UID: \"7d6190d5-6574-4704-97cb-2241398548e3\") " pod="openshift-insights/insights-runtime-extractor-6gjbf"
Oct 30 21:50:20.710428 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.710415    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-tmp\") pod \"console-6d5866d6cd-b6dpl\" (UID: \"42d54b8a-4f48-46a7-8a34-1f0f29cd1602\") " pod="openshift-console/console-6d5866d6cd-b6dpl"
Oct 30 21:50:20.710428 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.710417    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/0e0d17a1-7742-42b0-9d9b-740bb945e454-federate-client-tls\") pod \"telemeter-client-bb95b88bf-g9f94\" (UID: \"0e0d17a1-7742-42b0-9d9b-740bb945e454\") " pod="openshift-monitoring/telemeter-client-bb95b88bf-g9f94"
Oct 30 21:50:20.710795 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.710450    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/d414339d-1a64-4a2a-ad15-805b71ebfd1b-node-exporter-wtmp\") pod \"node-exporter-mpsnt\" (UID: \"d414339d-1a64-4a2a-ad15-805b71ebfd1b\") " pod="openshift-monitoring/node-exporter-mpsnt"
Oct 30 21:50:20.710795 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.710479    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/dc220f0e-8a21-4d7f-96b3-e7055c242f42-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-kggmc\" (UID: \"dc220f0e-8a21-4d7f-96b3-e7055c242f42\") " pod="openshift-multus/multus-additional-cni-plugins-kggmc"
Oct 30 21:50:20.710795 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.710505    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-25ggc\" (UniqueName: \"kubernetes.io/projected/2fe50918-ee11-4d17-b0fe-27c7898912c6-kube-api-access-25ggc\") pod \"dns-default-w5ck5\" (UID: \"2fe50918-ee11-4d17-b0fe-27c7898912c6\") " pod="openshift-dns/dns-default-w5ck5"
Oct 30 21:50:20.710795 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.710586    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d5e5ac6e-d37d-457d-9ca7-93065f5925ff-host-slash\") pod \"ovnkube-node-wrxww\" (UID: \"d5e5ac6e-d37d-457d-9ca7-93065f5925ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrxww"
Oct 30 21:50:20.710795 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.710624    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/d5e5ac6e-d37d-457d-9ca7-93065f5925ff-run-ovn\") pod \"ovnkube-node-wrxww\" (UID: \"d5e5ac6e-d37d-457d-9ca7-93065f5925ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrxww"
Oct 30 21:50:20.710795 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.710682    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/8f2f95c4-21b2-477b-a2cb-7c8590ec340d-etc-sysctl-d\") pod \"tuned-ldvhf\" (UID: \"8f2f95c4-21b2-477b-a2cb-7c8590ec340d\") " pod="openshift-cluster-node-tuning-operator/tuned-ldvhf"
Oct 30 21:50:20.710795 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.710704    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/d5e5ac6e-d37d-457d-9ca7-93065f5925ff-host-cni-netd\") pod \"ovnkube-node-wrxww\" (UID: \"d5e5ac6e-d37d-457d-9ca7-93065f5925ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrxww"
Oct 30 21:50:20.710795 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.710722    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z2p79\" (UniqueName: \"kubernetes.io/projected/d5e5ac6e-d37d-457d-9ca7-93065f5925ff-kube-api-access-z2p79\") pod \"ovnkube-node-wrxww\" (UID: \"d5e5ac6e-d37d-457d-9ca7-93065f5925ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrxww"
Oct 30 21:50:20.710795 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.710724    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/dc220f0e-8a21-4d7f-96b3-e7055c242f42-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-kggmc\" (UID: \"dc220f0e-8a21-4d7f-96b3-e7055c242f42\") " pod="openshift-multus/multus-additional-cni-plugins-kggmc"
Oct 30 21:50:20.710795 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.710773    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-console-serving-cert\") pod \"console-6d5866d6cd-b6dpl\" (UID: \"42d54b8a-4f48-46a7-8a34-1f0f29cd1602\") " pod="openshift-console/console-6d5866d6cd-b6dpl"
Oct 30 21:50:20.711080 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.710804    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-oauth-serving-cert\") pod \"console-6d5866d6cd-b6dpl\" (UID: \"42d54b8a-4f48-46a7-8a34-1f0f29cd1602\") " pod="openshift-console/console-6d5866d6cd-b6dpl"
Oct 30 21:50:20.711080 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.710829    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5c4fj\" (UniqueName: \"kubernetes.io/projected/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-kube-api-access-5c4fj\") pod \"console-6d5866d6cd-b6dpl\" (UID: \"42d54b8a-4f48-46a7-8a34-1f0f29cd1602\") " pod="openshift-console/console-6d5866d6cd-b6dpl"
Oct 30 21:50:20.711080 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:20.710857    5510 secret.go:189] Couldn't get secret openshift-console/console-serving-cert: object "openshift-console"/"console-serving-cert" not registered
Oct 30 21:50:20.711080 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:20.710866    5510 configmap.go:193] Couldn't get configMap openshift-console/oauth-serving-cert: object "openshift-console"/"oauth-serving-cert" not registered
Oct 30 21:50:20.711080 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:20.710893    5510 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-console-serving-cert podName:42d54b8a-4f48-46a7-8a34-1f0f29cd1602 nodeName:}" failed. No retries permitted until 2026-10-30 21:50:21.210883918 +0000 UTC m=+1.771693900 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "console-serving-cert" (UniqueName: "kubernetes.io/secret/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-console-serving-cert") pod "console-6d5866d6cd-b6dpl" (UID: "42d54b8a-4f48-46a7-8a34-1f0f29cd1602") : object "openshift-console"/"console-serving-cert" not registered
Oct 30 21:50:20.711080 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.710855    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/8c88b730-5cf1-4434-950f-4a3aa8f739cd-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7c848fbc95-8jm99\" (UID: \"8c88b730-5cf1-4434-950f-4a3aa8f739cd\") " pod="openshift-monitoring/thanos-querier-7c848fbc95-8jm99"
Oct 30 21:50:20.711080 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:20.710910    5510 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-oauth-serving-cert podName:42d54b8a-4f48-46a7-8a34-1f0f29cd1602 nodeName:}" failed. No retries permitted until 2026-10-30 21:50:21.210900536 +0000 UTC m=+1.771710558 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "oauth-serving-cert" (UniqueName: "kubernetes.io/configmap/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-oauth-serving-cert") pod "console-6d5866d6cd-b6dpl" (UID: "42d54b8a-4f48-46a7-8a34-1f0f29cd1602") : object "openshift-console"/"oauth-serving-cert" not registered
Oct 30 21:50:20.711080 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.710936    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ttgkm\" (UniqueName: \"kubernetes.io/projected/8c88b730-5cf1-4434-950f-4a3aa8f739cd-kube-api-access-ttgkm\") pod \"thanos-querier-7c848fbc95-8jm99\" (UID: \"8c88b730-5cf1-4434-950f-4a3aa8f739cd\") " pod="openshift-monitoring/thanos-querier-7c848fbc95-8jm99"
Oct 30 21:50:20.711080 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.710956    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8f2f95c4-21b2-477b-a2cb-7c8590ec340d-etc-kubernetes\") pod \"tuned-ldvhf\" (UID: \"8f2f95c4-21b2-477b-a2cb-7c8590ec340d\") " pod="openshift-cluster-node-tuning-operator/tuned-ldvhf"
Oct 30 21:50:20.711080 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.710972    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2fe50918-ee11-4d17-b0fe-27c7898912c6-metrics-tls\") pod \"dns-default-w5ck5\" (UID: \"2fe50918-ee11-4d17-b0fe-27c7898912c6\") " pod="openshift-dns/dns-default-w5ck5"
Oct 30 21:50:20.711080 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.711016    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d7c355fa-2141-463e-8cbc-bae3c44a16f1-socket-dir\") pod \"aws-ebs-csi-driver-node-cqxms\" (UID: \"d7c355fa-2141-463e-8cbc-bae3c44a16f1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cqxms"
Oct 30 21:50:20.711080 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.711050    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7d6190d5-6574-4704-97cb-2241398548e3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6gjbf\" (UID: \"7d6190d5-6574-4704-97cb-2241398548e3\") " pod="openshift-insights/insights-runtime-extractor-6gjbf"
Oct 30 21:50:20.711452 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.711101    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0e0d17a1-7742-42b0-9d9b-740bb945e454-serving-certs-ca-bundle\") pod \"telemeter-client-bb95b88bf-g9f94\" (UID: \"0e0d17a1-7742-42b0-9d9b-740bb945e454\") " pod="openshift-monitoring/telemeter-client-bb95b88bf-g9f94"
Oct 30 21:50:20.711452 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.711127    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/0e0d17a1-7742-42b0-9d9b-740bb945e454-secret-telemeter-client\") pod \"telemeter-client-bb95b88bf-g9f94\" (UID: \"0e0d17a1-7742-42b0-9d9b-740bb945e454\") " pod="openshift-monitoring/telemeter-client-bb95b88bf-g9f94"
Oct 30 21:50:20.711452 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.711152    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/31a389bb-f36f-4e53-9654-5f4648dae38a-console-oauth-config\") pod \"console-748df94584-88cjt\" (UID: \"31a389bb-f36f-4e53-9654-5f4648dae38a\") " pod="openshift-console/console-748df94584-88cjt"
Oct 30 21:50:20.711452 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.711169    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/04465eda-e6db-42d3-8d5f-c9be668038a9-host\") pod \"node-ca-xkrx5\" (UID: \"04465eda-e6db-42d3-8d5f-c9be668038a9\") " pod="openshift-image-registry/node-ca-xkrx5"
Oct 30 21:50:20.711452 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.711186    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2fe50918-ee11-4d17-b0fe-27c7898912c6-tmp-dir\") pod \"dns-default-w5ck5\" (UID: \"2fe50918-ee11-4d17-b0fe-27c7898912c6\") " pod="openshift-dns/dns-default-w5ck5"
Oct 30 21:50:20.711452 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.711246    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2fe50918-ee11-4d17-b0fe-27c7898912c6-tmp-dir\") pod \"dns-default-w5ck5\" (UID: \"2fe50918-ee11-4d17-b0fe-27c7898912c6\") " pod="openshift-dns/dns-default-w5ck5"
Oct 30 21:50:20.711452 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.711272    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8c88b730-5cf1-4434-950f-4a3aa8f739cd-tmp\") pod \"thanos-querier-7c848fbc95-8jm99\" (UID: \"8c88b730-5cf1-4434-950f-4a3aa8f739cd\") " pod="openshift-monitoring/thanos-querier-7c848fbc95-8jm99"
Oct 30 21:50:20.711452 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.711301    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b2929\" (UniqueName: \"kubernetes.io/projected/8883cb69-54c2-437c-a92e-3bc38547027f-kube-api-access-b2929\") pod \"monitoring-plugin-77f4f76594-bfnw9\" (UID: \"8883cb69-54c2-437c-a92e-3bc38547027f\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-bfnw9"
Oct 30 21:50:20.711452 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:20.711311    5510 secret.go:189] Couldn't get secret openshift-console/console-oauth-config: object "openshift-console"/"console-oauth-config" not registered
Oct 30 21:50:20.711452 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.711329    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3abd55eb-d1d8-4fdf-b82c-d35894b652ac-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-h95jn\" (UID: \"3abd55eb-d1d8-4fdf-b82c-d35894b652ac\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-h95jn"
Oct 30 21:50:20.711452 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:20.711352    5510 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/31a389bb-f36f-4e53-9654-5f4648dae38a-console-oauth-config podName:31a389bb-f36f-4e53-9654-5f4648dae38a nodeName:}" failed. No retries permitted until 2026-10-30 21:50:21.211338661 +0000 UTC m=+1.772148666 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "console-oauth-config" (UniqueName: "kubernetes.io/secret/31a389bb-f36f-4e53-9654-5f4648dae38a-console-oauth-config") pod "console-748df94584-88cjt" (UID: "31a389bb-f36f-4e53-9654-5f4648dae38a") : object "openshift-console"/"console-oauth-config" not registered
Oct 30 21:50:20.711452 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.711378    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8c88b730-5cf1-4434-950f-4a3aa8f739cd-tmp\") pod \"thanos-querier-7c848fbc95-8jm99\" (UID: \"8c88b730-5cf1-4434-950f-4a3aa8f739cd\") " pod="openshift-monitoring/thanos-querier-7c848fbc95-8jm99"
Oct 30 21:50:20.711452 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.711380    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f2qbm\" (UniqueName: \"kubernetes.io/projected/3abd55eb-d1d8-4fdf-b82c-d35894b652ac-kube-api-access-f2qbm\") pod \"openshift-state-metrics-7bd7489957-h95jn\" (UID: \"3abd55eb-d1d8-4fdf-b82c-d35894b652ac\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-h95jn"
Oct 30 21:50:20.711452 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.711421    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/d5e5ac6e-d37d-457d-9ca7-93065f5925ff-host-kubelet\") pod \"ovnkube-node-wrxww\" (UID: \"d5e5ac6e-d37d-457d-9ca7-93065f5925ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrxww"
Oct 30 21:50:20.711452 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.711447    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/c8b13e55-b3c7-4656-9845-16749704265f-host-run-k8s-cni-cncf-io\") pod \"multus-v8j9s\" (UID: \"c8b13e55-b3c7-4656-9845-16749704265f\") " pod="openshift-multus/multus-v8j9s"
Oct 30 21:50:20.711917 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.711474    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/0e0d17a1-7742-42b0-9d9b-740bb945e454-telemeter-client-tls\") pod \"telemeter-client-bb95b88bf-g9f94\" (UID: \"0e0d17a1-7742-42b0-9d9b-740bb945e454\") " pod="openshift-monitoring/telemeter-client-bb95b88bf-g9f94"
Oct 30 21:50:20.711917 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.711504    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0e0d17a1-7742-42b0-9d9b-740bb945e454-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-bb95b88bf-g9f94\" (UID: \"0e0d17a1-7742-42b0-9d9b-740bb945e454\") " pod="openshift-monitoring/telemeter-client-bb95b88bf-g9f94"
Oct 30 21:50:20.711917 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.711531    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/d5e5ac6e-d37d-457d-9ca7-93065f5925ff-run-systemd\") pod \"ovnkube-node-wrxww\" (UID: \"d5e5ac6e-d37d-457d-9ca7-93065f5925ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrxww"
Oct 30 21:50:20.711917 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.711672    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d5e5ac6e-d37d-457d-9ca7-93065f5925ff-host-run-ovn-kubernetes\") pod \"ovnkube-node-wrxww\" (UID: \"d5e5ac6e-d37d-457d-9ca7-93065f5925ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrxww"
Oct 30 21:50:20.711917 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.711693    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-console-config\") pod \"console-6d5866d6cd-b6dpl\" (UID: \"42d54b8a-4f48-46a7-8a34-1f0f29cd1602\") " pod="openshift-console/console-6d5866d6cd-b6dpl"
Oct 30 21:50:20.711917 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.711707    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/d414339d-1a64-4a2a-ad15-805b71ebfd1b-root\") pod \"node-exporter-mpsnt\" (UID: \"d414339d-1a64-4a2a-ad15-805b71ebfd1b\") " pod="openshift-monitoring/node-exporter-mpsnt"
Oct 30 21:50:20.711917 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.711723    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0ab70415-9f1d-4988-a228-df6ed621f0d8-host-slash\") pod \"iptables-alerter-l9d98\" (UID: \"0ab70415-9f1d-4988-a228-df6ed621f0d8\") " pod="openshift-network-operator/iptables-alerter-l9d98"
Oct 30 21:50:20.711917 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.711766    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c8b13e55-b3c7-4656-9845-16749704265f-multus-conf-dir\") pod \"multus-v8j9s\" (UID: \"c8b13e55-b3c7-4656-9845-16749704265f\") " pod="openshift-multus/multus-v8j9s"
Oct 30 21:50:20.711917 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:20.711778    5510 configmap.go:193] Couldn't get configMap openshift-console/console-config: object "openshift-console"/"console-config" not registered
Oct 30 21:50:20.711917 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.711789    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/c8b13e55-b3c7-4656-9845-16749704265f-multus-daemon-config\") pod \"multus-v8j9s\" (UID: \"c8b13e55-b3c7-4656-9845-16749704265f\") " pod="openshift-multus/multus-v8j9s"
Oct 30 21:50:20.711917 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.711813    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rb2wf\" (UniqueName: \"kubernetes.io/projected/31a389bb-f36f-4e53-9654-5f4648dae38a-kube-api-access-rb2wf\") pod \"console-748df94584-88cjt\" (UID: \"31a389bb-f36f-4e53-9654-5f4648dae38a\") " pod="openshift-console/console-748df94584-88cjt"
Oct 30 21:50:20.711917 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:20.711821    5510 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-console-config podName:42d54b8a-4f48-46a7-8a34-1f0f29cd1602 nodeName:}" failed. No retries permitted until 2026-10-30 21:50:21.21181201 +0000 UTC m=+1.772621993 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "console-config" (UniqueName: "kubernetes.io/configmap/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-console-config") pod "console-6d5866d6cd-b6dpl" (UID: "42d54b8a-4f48-46a7-8a34-1f0f29cd1602") : object "openshift-console"/"console-config" not registered
Oct 30 21:50:20.711917 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.711853    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8f2f95c4-21b2-477b-a2cb-7c8590ec340d-host\") pod \"tuned-ldvhf\" (UID: \"8f2f95c4-21b2-477b-a2cb-7c8590ec340d\") " pod="openshift-cluster-node-tuning-operator/tuned-ldvhf"
Oct 30 21:50:20.712395 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.711923    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8c88b730-5cf1-4434-950f-4a3aa8f739cd-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7c848fbc95-8jm99\" (UID: \"8c88b730-5cf1-4434-950f-4a3aa8f739cd\") " pod="openshift-monitoring/thanos-querier-7c848fbc95-8jm99"
Oct 30 21:50:20.712395 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.711995    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/7d6190d5-6574-4704-97cb-2241398548e3-data-volume\") pod \"insights-runtime-extractor-6gjbf\" (UID: \"7d6190d5-6574-4704-97cb-2241398548e3\") " pod="openshift-insights/insights-runtime-extractor-6gjbf"
Oct 30 21:50:20.712395 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.712027    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ggf4d\" (UniqueName: \"kubernetes.io/projected/7d6190d5-6574-4704-97cb-2241398548e3-kube-api-access-ggf4d\") pod \"insights-runtime-extractor-6gjbf\" (UID: \"7d6190d5-6574-4704-97cb-2241398548e3\") " pod="openshift-insights/insights-runtime-extractor-6gjbf"
Oct 30 21:50:20.712395 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.711998    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/c8b13e55-b3c7-4656-9845-16749704265f-multus-daemon-config\") pod \"multus-v8j9s\" (UID: \"c8b13e55-b3c7-4656-9845-16749704265f\") " pod="openshift-multus/multus-v8j9s"
Oct 30 21:50:20.712395 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.712053    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/7d6190d5-6574-4704-97cb-2241398548e3-data-volume\") pod \"insights-runtime-extractor-6gjbf\" (UID: \"7d6190d5-6574-4704-97cb-2241398548e3\") " pod="openshift-insights/insights-runtime-extractor-6gjbf"
Oct 30 21:50:20.712395 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.712054    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v4d5j\" (UniqueName: \"kubernetes.io/projected/c8b13e55-b3c7-4656-9845-16749704265f-kube-api-access-v4d5j\") pod \"multus-v8j9s\" (UID: \"c8b13e55-b3c7-4656-9845-16749704265f\") " pod="openshift-multus/multus-v8j9s"
Oct 30 21:50:20.712395 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.712088    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e810bcd1-78da-409f-b86e-08c47031939e-dbus\") pod \"global-pull-secret-syncer-68jrf\" (UID: \"e810bcd1-78da-409f-b86e-08c47031939e\") " pod="kube-system/global-pull-secret-syncer-68jrf"
Oct 30 21:50:20.712395 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.712117    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0e0d17a1-7742-42b0-9d9b-740bb945e454-telemeter-trusted-ca-bundle\") pod \"telemeter-client-bb95b88bf-g9f94\" (UID: \"0e0d17a1-7742-42b0-9d9b-740bb945e454\") " pod="openshift-monitoring/telemeter-client-bb95b88bf-g9f94"
Oct 30 21:50:20.712395 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.712142    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/31a389bb-f36f-4e53-9654-5f4648dae38a-console-config\") pod \"console-748df94584-88cjt\" (UID: \"31a389bb-f36f-4e53-9654-5f4648dae38a\") " pod="openshift-console/console-748df94584-88cjt"
Oct 30 21:50:20.712395 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.712166    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/d5e5ac6e-d37d-457d-9ca7-93065f5925ff-ovn-node-metrics-cert\") pod \"ovnkube-node-wrxww\" (UID: \"d5e5ac6e-d37d-457d-9ca7-93065f5925ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrxww"
Oct 30 21:50:20.712395 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.712184    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/52eb01f8-e86a-4458-8875-639394a87b30-metrics-certs\") pod \"network-metrics-daemon-vmrkz\" (UID: \"52eb01f8-e86a-4458-8875-639394a87b30\") " pod="openshift-multus/network-metrics-daemon-vmrkz"
Oct 30 21:50:20.712395 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:20.712199    5510 configmap.go:193] Couldn't get configMap openshift-console/console-config: object "openshift-console"/"console-config" not registered
Oct 30 21:50:20.712395 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.712208    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c8b13e55-b3c7-4656-9845-16749704265f-multus-cni-dir\") pod \"multus-v8j9s\" (UID: \"c8b13e55-b3c7-4656-9845-16749704265f\") " pod="openshift-multus/multus-v8j9s"
Oct 30 21:50:20.712395 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.712226    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d414339d-1a64-4a2a-ad15-805b71ebfd1b-sys\") pod \"node-exporter-mpsnt\" (UID: \"d414339d-1a64-4a2a-ad15-805b71ebfd1b\") " pod="openshift-monitoring/node-exporter-mpsnt"
Oct 30 21:50:20.712395 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.712241    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/8f2f95c4-21b2-477b-a2cb-7c8590ec340d-etc-systemd\") pod \"tuned-ldvhf\" (UID: \"8f2f95c4-21b2-477b-a2cb-7c8590ec340d\") " pod="openshift-cluster-node-tuning-operator/tuned-ldvhf"
Oct 30 21:50:20.712395 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.712278    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/2fe50918-ee11-4d17-b0fe-27c7898912c6-config-volume\") pod \"dns-default-w5ck5\" (UID: \"2fe50918-ee11-4d17-b0fe-27c7898912c6\") " pod="openshift-dns/dns-default-w5ck5"
Oct 30 21:50:20.712395 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.712315    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d7c355fa-2141-463e-8cbc-bae3c44a16f1-kubelet-dir\") pod \"aws-ebs-csi-driver-node-cqxms\" (UID: \"d7c355fa-2141-463e-8cbc-bae3c44a16f1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cqxms"
Oct 30 21:50:20.712395 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:20.712328    5510 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/31a389bb-f36f-4e53-9654-5f4648dae38a-console-config podName:31a389bb-f36f-4e53-9654-5f4648dae38a nodeName:}" failed. No retries permitted until 2026-10-30 21:50:21.212312409 +0000 UTC m=+1.773122411 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "console-config" (UniqueName: "kubernetes.io/configmap/31a389bb-f36f-4e53-9654-5f4648dae38a-console-config") pod "console-748df94584-88cjt" (UID: "31a389bb-f36f-4e53-9654-5f4648dae38a") : object "openshift-console"/"console-config" not registered
Oct 30 21:50:20.712395 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.712374    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vg89l\" (UniqueName: \"kubernetes.io/projected/d7c355fa-2141-463e-8cbc-bae3c44a16f1-kube-api-access-vg89l\") pod \"aws-ebs-csi-driver-node-cqxms\" (UID: \"d7c355fa-2141-463e-8cbc-bae3c44a16f1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cqxms"
Oct 30 21:50:20.712395 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.712407    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/d5e5ac6e-d37d-457d-9ca7-93065f5925ff-ovn-node-metrics-cert\") pod \"ovnkube-node-wrxww\" (UID: \"d5e5ac6e-d37d-457d-9ca7-93065f5925ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrxww"
Oct 30 21:50:20.713081 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.712415    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/3abd55eb-d1d8-4fdf-b82c-d35894b652ac-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-h95jn\" (UID: \"3abd55eb-d1d8-4fdf-b82c-d35894b652ac\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-h95jn"
Oct 30 21:50:20.713081 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.712476    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d8dc5e17-81b4-413a-a48f-6175d0ed3ec0-cert\") pod \"ingress-canary-t8tbp\" (UID: \"d8dc5e17-81b4-413a-a48f-6175d0ed3ec0\") " pod="openshift-ingress-canary/ingress-canary-t8tbp"
Oct 30 21:50:20.713081 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.712499    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fcjg4\" (UniqueName: \"kubernetes.io/projected/73c208eb-ffc0-46f2-8bd4-05ade5adb529-kube-api-access-fcjg4\") pod \"network-check-target-jmbng\" (UID: \"73c208eb-ffc0-46f2-8bd4-05ade5adb529\") " pod="openshift-network-diagnostics/network-check-target-jmbng"
Oct 30 21:50:20.713081 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.712538    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tq25h\" (UniqueName: \"kubernetes.io/projected/dc220f0e-8a21-4d7f-96b3-e7055c242f42-kube-api-access-tq25h\") pod \"multus-additional-cni-plugins-kggmc\" (UID: \"dc220f0e-8a21-4d7f-96b3-e7055c242f42\") " pod="openshift-multus/multus-additional-cni-plugins-kggmc"
Oct 30 21:50:20.713081 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.712586    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3abd55eb-d1d8-4fdf-b82c-d35894b652ac-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-h95jn\" (UID: \"3abd55eb-d1d8-4fdf-b82c-d35894b652ac\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-h95jn"
Oct 30 21:50:20.713081 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.712615    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0e0d17a1-7742-42b0-9d9b-740bb945e454-metrics-client-ca\") pod \"telemeter-client-bb95b88bf-g9f94\" (UID: \"0e0d17a1-7742-42b0-9d9b-740bb945e454\") " pod="openshift-monitoring/telemeter-client-bb95b88bf-g9f94"
Oct 30 21:50:20.713081 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.712647    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-console-oauth-config\") pod \"console-6d5866d6cd-b6dpl\" (UID: \"42d54b8a-4f48-46a7-8a34-1f0f29cd1602\") " pod="openshift-console/console-6d5866d6cd-b6dpl"
Oct 30 21:50:20.713081 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.712687    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-service-ca\") pod \"console-6d5866d6cd-b6dpl\" (UID: \"42d54b8a-4f48-46a7-8a34-1f0f29cd1602\") " pod="openshift-console/console-6d5866d6cd-b6dpl"
Oct 30 21:50:20.713081 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:20.712783    5510 secret.go:189] Couldn't get secret openshift-console/console-oauth-config: object "openshift-console"/"console-oauth-config" not registered
Oct 30 21:50:20.713081 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.712785    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8883cb69-54c2-437c-a92e-3bc38547027f-monitoring-plugin-cert\") pod \"monitoring-plugin-77f4f76594-bfnw9\" (UID: \"8883cb69-54c2-437c-a92e-3bc38547027f\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-bfnw9"
Oct 30 21:50:20.713081 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:20.712791    5510 configmap.go:193] Couldn't get configMap openshift-console/service-ca: object "openshift-console"/"service-ca" not registered
Oct 30 21:50:20.713081 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:20.712873    5510 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-console-oauth-config podName:42d54b8a-4f48-46a7-8a34-1f0f29cd1602 nodeName:}" failed. No retries permitted until 2026-10-30 21:50:21.21285986 +0000 UTC m=+1.773669857 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "console-oauth-config" (UniqueName: "kubernetes.io/secret/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-console-oauth-config") pod "console-6d5866d6cd-b6dpl" (UID: "42d54b8a-4f48-46a7-8a34-1f0f29cd1602") : object "openshift-console"/"console-oauth-config" not registered
Oct 30 21:50:20.713081 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:20.712891    5510 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-service-ca podName:42d54b8a-4f48-46a7-8a34-1f0f29cd1602 nodeName:}" failed. No retries permitted until 2026-10-30 21:50:21.212880561 +0000 UTC m=+1.773690544 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca" (UniqueName: "kubernetes.io/configmap/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-service-ca") pod "console-6d5866d6cd-b6dpl" (UID: "42d54b8a-4f48-46a7-8a34-1f0f29cd1602") : object "openshift-console"/"service-ca" not registered
Oct 30 21:50:20.713081 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.712906    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/0ab70415-9f1d-4988-a228-df6ed621f0d8-iptables-alerter-script\") pod \"iptables-alerter-l9d98\" (UID: \"0ab70415-9f1d-4988-a228-df6ed621f0d8\") " pod="openshift-network-operator/iptables-alerter-l9d98"
Oct 30 21:50:20.713081 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.712936    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/dc220f0e-8a21-4d7f-96b3-e7055c242f42-tuning-conf-dir\") pod \"multus-additional-cni-plugins-kggmc\" (UID: \"dc220f0e-8a21-4d7f-96b3-e7055c242f42\") " pod="openshift-multus/multus-additional-cni-plugins-kggmc"
Oct 30 21:50:20.713081 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.712954    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c8b13e55-b3c7-4656-9845-16749704265f-host-var-lib-cni-bin\") pod \"multus-v8j9s\" (UID: \"c8b13e55-b3c7-4656-9845-16749704265f\") " pod="openshift-multus/multus-v8j9s"
Oct 30 21:50:20.713081 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.712973    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/31a389bb-f36f-4e53-9654-5f4648dae38a-console-serving-cert\") pod \"console-748df94584-88cjt\" (UID: \"31a389bb-f36f-4e53-9654-5f4648dae38a\") " pod="openshift-console/console-748df94584-88cjt"
Oct 30 21:50:20.713081 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:20.713062    5510 secret.go:189] Couldn't get secret openshift-console/console-serving-cert: object "openshift-console"/"console-serving-cert" not registered
Oct 30 21:50:20.713081 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.713079    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/0ab70415-9f1d-4988-a228-df6ed621f0d8-iptables-alerter-script\") pod \"iptables-alerter-l9d98\" (UID: \"0ab70415-9f1d-4988-a228-df6ed621f0d8\") " pod="openshift-network-operator/iptables-alerter-l9d98"
Oct 30 21:50:20.713598 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.713083    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8f2f95c4-21b2-477b-a2cb-7c8590ec340d-sys\") pod \"tuned-ldvhf\" (UID: \"8f2f95c4-21b2-477b-a2cb-7c8590ec340d\") " pod="openshift-cluster-node-tuning-operator/tuned-ldvhf"
Oct 30 21:50:20.713598 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:20.713097    5510 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/31a389bb-f36f-4e53-9654-5f4648dae38a-console-serving-cert podName:31a389bb-f36f-4e53-9654-5f4648dae38a nodeName:}" failed. No retries permitted until 2026-10-30 21:50:21.213087924 +0000 UTC m=+1.773897921 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "console-serving-cert" (UniqueName: "kubernetes.io/secret/31a389bb-f36f-4e53-9654-5f4648dae38a-console-serving-cert") pod "console-748df94584-88cjt" (UID: "31a389bb-f36f-4e53-9654-5f4648dae38a") : object "openshift-console"/"console-serving-cert" not registered
Oct 30 21:50:20.713598 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.713122    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/8f2f95c4-21b2-477b-a2cb-7c8590ec340d-etc-modprobe-d\") pod \"tuned-ldvhf\" (UID: \"8f2f95c4-21b2-477b-a2cb-7c8590ec340d\") " pod="openshift-cluster-node-tuning-operator/tuned-ldvhf"
Oct 30 21:50:20.713598 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.713139    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dc220f0e-8a21-4d7f-96b3-e7055c242f42-system-cni-dir\") pod \"multus-additional-cni-plugins-kggmc\" (UID: \"dc220f0e-8a21-4d7f-96b3-e7055c242f42\") " pod="openshift-multus/multus-additional-cni-plugins-kggmc"
Oct 30 21:50:20.713598 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.713158    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c8b13e55-b3c7-4656-9845-16749704265f-cni-binary-copy\") pod \"multus-v8j9s\" (UID: \"c8b13e55-b3c7-4656-9845-16749704265f\") " pod="openshift-multus/multus-v8j9s"
Oct 30 21:50:20.713598 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.713186    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/c8b13e55-b3c7-4656-9845-16749704265f-host-run-multus-certs\") pod \"multus-v8j9s\" (UID: \"c8b13e55-b3c7-4656-9845-16749704265f\") " pod="openshift-multus/multus-v8j9s"
Oct 30 21:50:20.713598 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.713239    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d7c355fa-2141-463e-8cbc-bae3c44a16f1-sys-fs\") pod \"aws-ebs-csi-driver-node-cqxms\" (UID: \"d7c355fa-2141-463e-8cbc-bae3c44a16f1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cqxms"
Oct 30 21:50:20.713598 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.713265    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vh5d6\" (UniqueName: \"kubernetes.io/projected/d8dc5e17-81b4-413a-a48f-6175d0ed3ec0-kube-api-access-vh5d6\") pod \"ingress-canary-t8tbp\" (UID: \"d8dc5e17-81b4-413a-a48f-6175d0ed3ec0\") " pod="openshift-ingress-canary/ingress-canary-t8tbp"
Oct 30 21:50:20.713598 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.713282    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c8b13e55-b3c7-4656-9845-16749704265f-system-cni-dir\") pod \"multus-v8j9s\" (UID: \"c8b13e55-b3c7-4656-9845-16749704265f\") " pod="openshift-multus/multus-v8j9s"
Oct 30 21:50:20.713598 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.713281    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c8b13e55-b3c7-4656-9845-16749704265f-cni-binary-copy\") pod \"multus-v8j9s\" (UID: \"c8b13e55-b3c7-4656-9845-16749704265f\") " pod="openshift-multus/multus-v8j9s"
Oct 30 21:50:20.713598 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.713322    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/c8b13e55-b3c7-4656-9845-16749704265f-multus-socket-dir-parent\") pod \"multus-v8j9s\" (UID: \"c8b13e55-b3c7-4656-9845-16749704265f\") " pod="openshift-multus/multus-v8j9s"
Oct 30 21:50:20.713598 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.713412    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/8f2f95c4-21b2-477b-a2cb-7c8590ec340d-etc-tuned\") pod \"tuned-ldvhf\" (UID: \"8f2f95c4-21b2-477b-a2cb-7c8590ec340d\") " pod="openshift-cluster-node-tuning-operator/tuned-ldvhf"
Oct 30 21:50:20.713598 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.713445    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d5e5ac6e-d37d-457d-9ca7-93065f5925ff-host-run-netns\") pod \"ovnkube-node-wrxww\" (UID: \"d5e5ac6e-d37d-457d-9ca7-93065f5925ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrxww"
Oct 30 21:50:20.713598 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.713464    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/04465eda-e6db-42d3-8d5f-c9be668038a9-serviceca\") pod \"node-ca-xkrx5\" (UID: \"04465eda-e6db-42d3-8d5f-c9be668038a9\") " pod="openshift-image-registry/node-ca-xkrx5"
Oct 30 21:50:20.713598 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.713480    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d414339d-1a64-4a2a-ad15-805b71ebfd1b-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-mpsnt\" (UID: \"d414339d-1a64-4a2a-ad15-805b71ebfd1b\") " pod="openshift-monitoring/node-exporter-mpsnt"
Oct 30 21:50:20.713598 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.713483    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/8f2f95c4-21b2-477b-a2cb-7c8590ec340d-etc-tuned\") pod \"tuned-ldvhf\" (UID: \"8f2f95c4-21b2-477b-a2cb-7c8590ec340d\") " pod="openshift-cluster-node-tuning-operator/tuned-ldvhf"
Oct 30 21:50:20.713598 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.713517    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/8c88b730-5cf1-4434-950f-4a3aa8f739cd-secret-thanos-querier-tls\") pod \"thanos-querier-7c848fbc95-8jm99\" (UID: \"8c88b730-5cf1-4434-950f-4a3aa8f739cd\") " pod="openshift-monitoring/thanos-querier-7c848fbc95-8jm99"
Oct 30 21:50:20.713598 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.713537    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/7d6190d5-6574-4704-97cb-2241398548e3-crio-socket\") pod \"insights-runtime-extractor-6gjbf\" (UID: \"7d6190d5-6574-4704-97cb-2241398548e3\") " pod="openshift-insights/insights-runtime-extractor-6gjbf"
Oct 30 21:50:20.713598 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.713586    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/d5e5ac6e-d37d-457d-9ca7-93065f5925ff-ovnkube-config\") pod \"ovnkube-node-wrxww\" (UID: \"d5e5ac6e-d37d-457d-9ca7-93065f5925ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrxww"
Oct 30 21:50:20.714119 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.713621    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/dc220f0e-8a21-4d7f-96b3-e7055c242f42-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-kggmc\" (UID: \"dc220f0e-8a21-4d7f-96b3-e7055c242f42\") " pod="openshift-multus/multus-additional-cni-plugins-kggmc"
Oct 30 21:50:20.714119 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.713667    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c8b13e55-b3c7-4656-9845-16749704265f-cnibin\") pod \"multus-v8j9s\" (UID: \"c8b13e55-b3c7-4656-9845-16749704265f\") " pod="openshift-multus/multus-v8j9s"
Oct 30 21:50:20.714119 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.713711    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/d414339d-1a64-4a2a-ad15-805b71ebfd1b-node-exporter-textfile\") pod \"node-exporter-mpsnt\" (UID: \"d414339d-1a64-4a2a-ad15-805b71ebfd1b\") " pod="openshift-monitoring/node-exporter-mpsnt"
Oct 30 21:50:20.714119 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.713770    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/db4b06a3-45e5-4d45-9d7a-407d117d9454-tmp-dir\") pod \"node-resolver-krtnr\" (UID: \"db4b06a3-45e5-4d45-9d7a-407d117d9454\") " pod="openshift-dns/node-resolver-krtnr"
Oct 30 21:50:20.714119 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.713795    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/d414339d-1a64-4a2a-ad15-805b71ebfd1b-node-exporter-textfile\") pod \"node-exporter-mpsnt\" (UID: \"d414339d-1a64-4a2a-ad15-805b71ebfd1b\") " pod="openshift-monitoring/node-exporter-mpsnt"
Oct 30 21:50:20.714119 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.713804    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/c8b13e55-b3c7-4656-9845-16749704265f-host-var-lib-cni-multus\") pod \"multus-v8j9s\" (UID: \"c8b13e55-b3c7-4656-9845-16749704265f\") " pod="openshift-multus/multus-v8j9s"
Oct 30 21:50:20.714119 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.713806    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/dc220f0e-8a21-4d7f-96b3-e7055c242f42-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-kggmc\" (UID: \"dc220f0e-8a21-4d7f-96b3-e7055c242f42\") " pod="openshift-multus/multus-additional-cni-plugins-kggmc"
Oct 30 21:50:20.714119 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.713861    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d5e5ac6e-d37d-457d-9ca7-93065f5925ff-run-openvswitch\") pod \"ovnkube-node-wrxww\" (UID: \"d5e5ac6e-d37d-457d-9ca7-93065f5925ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrxww"
Oct 30 21:50:20.714119 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.713873    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/db4b06a3-45e5-4d45-9d7a-407d117d9454-tmp-dir\") pod \"node-resolver-krtnr\" (UID: \"db4b06a3-45e5-4d45-9d7a-407d117d9454\") " pod="openshift-dns/node-resolver-krtnr"
Oct 30 21:50:20.714119 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.713868    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/d5e5ac6e-d37d-457d-9ca7-93065f5925ff-ovnkube-config\") pod \"ovnkube-node-wrxww\" (UID: \"d5e5ac6e-d37d-457d-9ca7-93065f5925ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrxww"
Oct 30 21:50:20.714119 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.713910    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/d5e5ac6e-d37d-457d-9ca7-93065f5925ff-node-log\") pod \"ovnkube-node-wrxww\" (UID: \"d5e5ac6e-d37d-457d-9ca7-93065f5925ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrxww"
Oct 30 21:50:20.714119 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.713929    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8c88b730-5cf1-4434-950f-4a3aa8f739cd-metrics-client-ca\") pod \"thanos-querier-7c848fbc95-8jm99\" (UID: \"8c88b730-5cf1-4434-950f-4a3aa8f739cd\") " pod="openshift-monitoring/thanos-querier-7c848fbc95-8jm99"
Oct 30 21:50:20.714119 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.713960    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9krk9\" (UniqueName: \"kubernetes.io/projected/db4b06a3-45e5-4d45-9d7a-407d117d9454-kube-api-access-9krk9\") pod \"node-resolver-krtnr\" (UID: \"db4b06a3-45e5-4d45-9d7a-407d117d9454\") " pod="openshift-dns/node-resolver-krtnr"
Oct 30 21:50:20.714119 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.713975    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/dc220f0e-8a21-4d7f-96b3-e7055c242f42-cnibin\") pod \"multus-additional-cni-plugins-kggmc\" (UID: \"dc220f0e-8a21-4d7f-96b3-e7055c242f42\") " pod="openshift-multus/multus-additional-cni-plugins-kggmc"
Oct 30 21:50:20.714119 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.713995    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d7c355fa-2141-463e-8cbc-bae3c44a16f1-registration-dir\") pod \"aws-ebs-csi-driver-node-cqxms\" (UID: \"d7c355fa-2141-463e-8cbc-bae3c44a16f1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cqxms"
Oct 30 21:50:20.714119 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.714037    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/31a389bb-f36f-4e53-9654-5f4648dae38a-service-ca\") pod \"console-748df94584-88cjt\" (UID: \"31a389bb-f36f-4e53-9654-5f4648dae38a\") " pod="openshift-console/console-748df94584-88cjt"
Oct 30 21:50:20.714119 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.714076    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zxwjt\" (UniqueName: \"kubernetes.io/projected/52eb01f8-e86a-4458-8875-639394a87b30-kube-api-access-zxwjt\") pod \"network-metrics-daemon-vmrkz\" (UID: \"52eb01f8-e86a-4458-8875-639394a87b30\") " pod="openshift-multus/network-metrics-daemon-vmrkz"
Oct 30 21:50:20.714628 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:20.714151    5510 configmap.go:193] Couldn't get configMap openshift-console/service-ca: object "openshift-console"/"service-ca" not registered
Oct 30 21:50:20.714628 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.714157    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/31a389bb-f36f-4e53-9654-5f4648dae38a-tmp\") pod \"console-748df94584-88cjt\" (UID: \"31a389bb-f36f-4e53-9654-5f4648dae38a\") " pod="openshift-console/console-748df94584-88cjt"
Oct 30 21:50:20.714628 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.714233    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/31a389bb-f36f-4e53-9654-5f4648dae38a-tmp\") pod \"console-748df94584-88cjt\" (UID: \"31a389bb-f36f-4e53-9654-5f4648dae38a\") " pod="openshift-console/console-748df94584-88cjt"
Oct 30 21:50:20.714628 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:20.714243    5510 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/31a389bb-f36f-4e53-9654-5f4648dae38a-service-ca podName:31a389bb-f36f-4e53-9654-5f4648dae38a nodeName:}" failed. No retries permitted until 2026-10-30 21:50:21.214226184 +0000 UTC m=+1.775036190 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca" (UniqueName: "kubernetes.io/configmap/31a389bb-f36f-4e53-9654-5f4648dae38a-service-ca") pod "console-748df94584-88cjt" (UID: "31a389bb-f36f-4e53-9654-5f4648dae38a") : object "openshift-console"/"service-ca" not registered
Oct 30 21:50:20.714971 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.714956    5510 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Oct 30 21:50:20.755766 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.755687    5510 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Oct 30 21:50:20.755912 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.755836    5510 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Oct 30 21:50:20.756058 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.756040    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-bbh9g\""
Oct 30 21:50:20.756120 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.756041    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Oct 30 21:50:20.757036 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.756731    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Oct 30 21:50:20.757036 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.756778    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Oct 30 21:50:20.757212 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.757198    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-5ik3e6gooc4tm\""
Oct 30 21:50:20.757284 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.757262    5510 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Oct 30 21:50:20.757353 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.757263    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Oct 30 21:50:20.757520 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.757503    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-xkkts\""
Oct 30 21:50:20.757766 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.757749    5510 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Oct 30 21:50:20.757826 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.757796    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Oct 30 21:50:20.757958 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.757943    5510 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Oct 30 21:50:20.758120 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.758103    5510 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Oct 30 21:50:20.758186 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.758102    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Oct 30 21:50:20.758604 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.758586    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Oct 30 21:50:20.758733 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.758719    5510 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Oct 30 21:50:20.758845 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.758827    5510 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Oct 30 21:50:20.758938 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.758926    5510 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Oct 30 21:50:20.759019 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.759002    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Oct 30 21:50:20.759080 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.759018    5510 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Oct 30 21:50:20.759144 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.759133    5510 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Oct 30 21:50:20.759221 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.759209    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Oct 30 21:50:20.759277 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.759253    5510 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Oct 30 21:50:20.760574 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.759347    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Oct 30 21:50:20.760574 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.759386    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Oct 30 21:50:20.760574 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.759530    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-dhl5t\""
Oct 30 21:50:20.760574 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.759537    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Oct 30 21:50:20.760574 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.759560    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-pg2zk\""
Oct 30 21:50:20.760574 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.759668    5510 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Oct 30 21:50:20.760574 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.759809    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Oct 30 21:50:20.760574 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.759813    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/d414339d-1a64-4a2a-ad15-805b71ebfd1b-node-exporter-accelerators-collector-config\") pod \"node-exporter-mpsnt\" (UID: \"d414339d-1a64-4a2a-ad15-805b71ebfd1b\") " pod="openshift-monitoring/node-exporter-mpsnt"
Oct 30 21:50:20.760574 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.759868    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Oct 30 21:50:20.760574 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.759833    5510 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Oct 30 21:50:20.760574 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.759913    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Oct 30 21:50:20.760574 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.760107    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-cmhxq\""
Oct 30 21:50:20.760574 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.760253    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-9bjlv\""
Oct 30 21:50:20.760574 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.760380    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-8lg5r\""
Oct 30 21:50:20.760574 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.760415    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-9mgc8\""
Oct 30 21:50:20.760574 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.760452    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8c88b730-5cf1-4434-950f-4a3aa8f739cd-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7c848fbc95-8jm99\" (UID: \"8c88b730-5cf1-4434-950f-4a3aa8f739cd\") " pod="openshift-monitoring/thanos-querier-7c848fbc95-8jm99"
Oct 30 21:50:20.760574 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.760517    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8c88b730-5cf1-4434-950f-4a3aa8f739cd-secret-grpc-tls\") pod \"thanos-querier-7c848fbc95-8jm99\" (UID: \"8c88b730-5cf1-4434-950f-4a3aa8f739cd\") " pod="openshift-monitoring/thanos-querier-7c848fbc95-8jm99"
Oct 30 21:50:20.761914 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.760672    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-52nbt\""
Oct 30 21:50:20.761914 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.760740    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/0e0d17a1-7742-42b0-9d9b-740bb945e454-federate-client-tls\") pod \"telemeter-client-bb95b88bf-g9f94\" (UID: \"0e0d17a1-7742-42b0-9d9b-740bb945e454\") " pod="openshift-monitoring/telemeter-client-bb95b88bf-g9f94"
Oct 30 21:50:20.761914 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.760761    5510 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Oct 30 21:50:20.761914 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.760787    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Oct 30 21:50:20.761914 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.760898    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7d6190d5-6574-4704-97cb-2241398548e3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6gjbf\" (UID: \"7d6190d5-6574-4704-97cb-2241398548e3\") " pod="openshift-insights/insights-runtime-extractor-6gjbf"
Oct 30 21:50:20.761914 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.761096    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/8c88b730-5cf1-4434-950f-4a3aa8f739cd-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7c848fbc95-8jm99\" (UID: \"8c88b730-5cf1-4434-950f-4a3aa8f739cd\") " pod="openshift-monitoring/thanos-querier-7c848fbc95-8jm99"
Oct 30 21:50:20.761914 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.761326    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7d6190d5-6574-4704-97cb-2241398548e3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6gjbf\" (UID: \"7d6190d5-6574-4704-97cb-2241398548e3\") " pod="openshift-insights/insights-runtime-extractor-6gjbf"
Oct 30 21:50:20.761914 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.761370    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2fe50918-ee11-4d17-b0fe-27c7898912c6-metrics-tls\") pod \"dns-default-w5ck5\" (UID: \"2fe50918-ee11-4d17-b0fe-27c7898912c6\") " pod="openshift-dns/dns-default-w5ck5"
Oct 30 21:50:20.761914 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.761600    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/0e0d17a1-7742-42b0-9d9b-740bb945e454-secret-telemeter-client\") pod \"telemeter-client-bb95b88bf-g9f94\" (UID: \"0e0d17a1-7742-42b0-9d9b-740bb945e454\") " pod="openshift-monitoring/telemeter-client-bb95b88bf-g9f94"
Oct 30 21:50:20.761914 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.761670    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0e0d17a1-7742-42b0-9d9b-740bb945e454-serving-certs-ca-bundle\") pod \"telemeter-client-bb95b88bf-g9f94\" (UID: \"0e0d17a1-7742-42b0-9d9b-740bb945e454\") " pod="openshift-monitoring/telemeter-client-bb95b88bf-g9f94"
Oct 30 21:50:20.761914 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.761811    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Oct 30 21:50:20.761914 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.761856    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/0e0d17a1-7742-42b0-9d9b-740bb945e454-telemeter-client-tls\") pod \"telemeter-client-bb95b88bf-g9f94\" (UID: \"0e0d17a1-7742-42b0-9d9b-740bb945e454\") " pod="openshift-monitoring/telemeter-client-bb95b88bf-g9f94"
Oct 30 21:50:20.762427 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.761958    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0e0d17a1-7742-42b0-9d9b-740bb945e454-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-bb95b88bf-g9f94\" (UID: \"0e0d17a1-7742-42b0-9d9b-740bb945e454\") " pod="openshift-monitoring/telemeter-client-bb95b88bf-g9f94"
Oct 30 21:50:20.762427 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.762166    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8c88b730-5cf1-4434-950f-4a3aa8f739cd-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7c848fbc95-8jm99\" (UID: \"8c88b730-5cf1-4434-950f-4a3aa8f739cd\") " pod="openshift-monitoring/thanos-querier-7c848fbc95-8jm99"
Oct 30 21:50:20.762531 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.762500    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-brb56\""
Oct 30 21:50:20.762585 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.762563    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/2fe50918-ee11-4d17-b0fe-27c7898912c6-config-volume\") pod \"dns-default-w5ck5\" (UID: \"2fe50918-ee11-4d17-b0fe-27c7898912c6\") " pod="openshift-dns/dns-default-w5ck5"
Oct 30 21:50:20.762638 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.762613    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/52eb01f8-e86a-4458-8875-639394a87b30-metrics-certs\") pod \"network-metrics-daemon-vmrkz\" (UID: \"52eb01f8-e86a-4458-8875-639394a87b30\") " pod="openshift-multus/network-metrics-daemon-vmrkz"
Oct 30 21:50:20.763231 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.763202    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d8dc5e17-81b4-413a-a48f-6175d0ed3ec0-cert\") pod \"ingress-canary-t8tbp\" (UID: \"d8dc5e17-81b4-413a-a48f-6175d0ed3ec0\") " pod="openshift-ingress-canary/ingress-canary-t8tbp"
Oct 30 21:50:20.763296 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.763230    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/3abd55eb-d1d8-4fdf-b82c-d35894b652ac-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-h95jn\" (UID: \"3abd55eb-d1d8-4fdf-b82c-d35894b652ac\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-h95jn"
Oct 30 21:50:20.763296 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.763248    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Oct 30 21:50:20.763296 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.763257    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3abd55eb-d1d8-4fdf-b82c-d35894b652ac-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-h95jn\" (UID: \"3abd55eb-d1d8-4fdf-b82c-d35894b652ac\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-h95jn"
Oct 30 21:50:20.763296 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.763265    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8883cb69-54c2-437c-a92e-3bc38547027f-monitoring-plugin-cert\") pod \"monitoring-plugin-77f4f76594-bfnw9\" (UID: \"8883cb69-54c2-437c-a92e-3bc38547027f\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-bfnw9"
Oct 30 21:50:20.763561 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.763541    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-h2qxd\""
Oct 30 21:50:20.763650 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.763569    5510 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Oct 30 21:50:20.763650 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.763588    5510 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-28r105lbnph2c\""
Oct 30 21:50:20.763751 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.763705    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d414339d-1a64-4a2a-ad15-805b71ebfd1b-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-mpsnt\" (UID: \"d414339d-1a64-4a2a-ad15-805b71ebfd1b\") " pod="openshift-monitoring/node-exporter-mpsnt"
Oct 30 21:50:20.763817 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.763804    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/8c88b730-5cf1-4434-950f-4a3aa8f739cd-secret-thanos-querier-tls\") pod \"thanos-querier-7c848fbc95-8jm99\" (UID: \"8c88b730-5cf1-4434-950f-4a3aa8f739cd\") " pod="openshift-monitoring/thanos-querier-7c848fbc95-8jm99"
Oct 30 21:50:20.764149 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.764133    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/04465eda-e6db-42d3-8d5f-c9be668038a9-serviceca\") pod \"node-ca-xkrx5\" (UID: \"04465eda-e6db-42d3-8d5f-c9be668038a9\") " pod="openshift-image-registry/node-ca-xkrx5"
Oct 30 21:50:20.764297 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.764283    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8c88b730-5cf1-4434-950f-4a3aa8f739cd-metrics-client-ca\") pod \"thanos-querier-7c848fbc95-8jm99\" (UID: \"8c88b730-5cf1-4434-950f-4a3aa8f739cd\") " pod="openshift-monitoring/thanos-querier-7c848fbc95-8jm99"
Oct 30 21:50:20.769299 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.769280    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d414339d-1a64-4a2a-ad15-805b71ebfd1b-node-exporter-tls\") pod \"node-exporter-mpsnt\" (UID: \"d414339d-1a64-4a2a-ad15-805b71ebfd1b\") " pod="openshift-monitoring/node-exporter-mpsnt"
Oct 30 21:50:20.769417 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.769403    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/8c88b730-5cf1-4434-950f-4a3aa8f739cd-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7c848fbc95-8jm99\" (UID: \"8c88b730-5cf1-4434-950f-4a3aa8f739cd\") " pod="openshift-monitoring/thanos-querier-7c848fbc95-8jm99"
Oct 30 21:50:20.769934 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.769914    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d414339d-1a64-4a2a-ad15-805b71ebfd1b-metrics-client-ca\") pod \"node-exporter-mpsnt\" (UID: \"d414339d-1a64-4a2a-ad15-805b71ebfd1b\") " pod="openshift-monitoring/node-exporter-mpsnt"
Oct 30 21:50:20.770108 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.770074    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:20.770916 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.770895    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-25ggc\" (UniqueName: \"kubernetes.io/projected/2fe50918-ee11-4d17-b0fe-27c7898912c6-kube-api-access-25ggc\") pod \"dns-default-w5ck5\" (UID: \"2fe50918-ee11-4d17-b0fe-27c7898912c6\") " pod="openshift-dns/dns-default-w5ck5"
Oct 30 21:50:20.771034 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.770966    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rq9p9\" (UniqueName: \"kubernetes.io/projected/d414339d-1a64-4a2a-ad15-805b71ebfd1b-kube-api-access-rq9p9\") pod \"node-exporter-mpsnt\" (UID: \"d414339d-1a64-4a2a-ad15-805b71ebfd1b\") " pod="openshift-monitoring/node-exporter-mpsnt"
Oct 30 21:50:20.771731 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.771712    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3abd55eb-d1d8-4fdf-b82c-d35894b652ac-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-h95jn\" (UID: \"3abd55eb-d1d8-4fdf-b82c-d35894b652ac\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-h95jn"
Oct 30 21:50:20.772536 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.772519    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0e0d17a1-7742-42b0-9d9b-740bb945e454-telemeter-trusted-ca-bundle\") pod \"telemeter-client-bb95b88bf-g9f94\" (UID: \"0e0d17a1-7742-42b0-9d9b-740bb945e454\") " pod="openshift-monitoring/telemeter-client-bb95b88bf-g9f94"
Oct 30 21:50:20.773567 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.773552    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0e0d17a1-7742-42b0-9d9b-740bb945e454-metrics-client-ca\") pod \"telemeter-client-bb95b88bf-g9f94\" (UID: \"0e0d17a1-7742-42b0-9d9b-740bb945e454\") " pod="openshift-monitoring/telemeter-client-bb95b88bf-g9f94"
Oct 30 21:50:20.801392 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:20.801370    5510 projected.go:291] Couldn't get configMap openshift-console/kube-root-ca.crt: object "openshift-console"/"kube-root-ca.crt" not registered
Oct 30 21:50:20.801392 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:20.801394    5510 projected.go:291] Couldn't get configMap openshift-console/openshift-service-ca.crt: object "openshift-console"/"openshift-service-ca.crt" not registered
Oct 30 21:50:20.801543 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:20.801409    5510 projected.go:196] Error preparing data for projected volume kube-api-access-5c4fj for pod openshift-console/console-6d5866d6cd-b6dpl: [object "openshift-console"/"kube-root-ca.crt" not registered, object "openshift-console"/"openshift-service-ca.crt" not registered]
Oct 30 21:50:20.801543 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:20.801492    5510 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-kube-api-access-5c4fj podName:42d54b8a-4f48-46a7-8a34-1f0f29cd1602 nodeName:}" failed. No retries permitted until 2026-10-30 21:50:21.301473869 +0000 UTC m=+1.862283886 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-5c4fj" (UniqueName: "kubernetes.io/projected/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-kube-api-access-5c4fj") pod "console-6d5866d6cd-b6dpl" (UID: "42d54b8a-4f48-46a7-8a34-1f0f29cd1602") : [object "openshift-console"/"kube-root-ca.crt" not registered, object "openshift-console"/"openshift-service-ca.crt" not registered]
Oct 30 21:50:20.801717 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.801690    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-td6rr\" (UniqueName: \"kubernetes.io/projected/0ab70415-9f1d-4988-a228-df6ed621f0d8-kube-api-access-td6rr\") pod \"iptables-alerter-l9d98\" (UID: \"0ab70415-9f1d-4988-a228-df6ed621f0d8\") " pod="openshift-network-operator/iptables-alerter-l9d98"
Oct 30 21:50:20.801789 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.801773    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hrxwg\" (UniqueName: \"kubernetes.io/projected/04465eda-e6db-42d3-8d5f-c9be668038a9-kube-api-access-hrxwg\") pod \"node-ca-xkrx5\" (UID: \"04465eda-e6db-42d3-8d5f-c9be668038a9\") " pod="openshift-image-registry/node-ca-xkrx5"
Oct 30 21:50:20.801861 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.801815    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z2p79\" (UniqueName: \"kubernetes.io/projected/d5e5ac6e-d37d-457d-9ca7-93065f5925ff-kube-api-access-z2p79\") pod \"ovnkube-node-wrxww\" (UID: \"d5e5ac6e-d37d-457d-9ca7-93065f5925ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrxww"
Oct 30 21:50:20.801961 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.801940    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b2929\" (UniqueName: \"kubernetes.io/projected/8883cb69-54c2-437c-a92e-3bc38547027f-kube-api-access-b2929\") pod \"monitoring-plugin-77f4f76594-bfnw9\" (UID: \"8883cb69-54c2-437c-a92e-3bc38547027f\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-bfnw9"
Oct 30 21:50:20.802157 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:20.802140    5510 projected.go:291] Couldn't get configMap openshift-console/kube-root-ca.crt: object "openshift-console"/"kube-root-ca.crt" not registered
Oct 30 21:50:20.802212 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:20.802162    5510 projected.go:291] Couldn't get configMap openshift-console/openshift-service-ca.crt: object "openshift-console"/"openshift-service-ca.crt" not registered
Oct 30 21:50:20.802212 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:20.802177    5510 projected.go:196] Error preparing data for projected volume kube-api-access-rb2wf for pod openshift-console/console-748df94584-88cjt: [object "openshift-console"/"kube-root-ca.crt" not registered, object "openshift-console"/"openshift-service-ca.crt" not registered]
Oct 30 21:50:20.802212 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.802173    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t7ldj\" (UniqueName: \"kubernetes.io/projected/8f2f95c4-21b2-477b-a2cb-7c8590ec340d-kube-api-access-t7ldj\") pod \"tuned-ldvhf\" (UID: \"8f2f95c4-21b2-477b-a2cb-7c8590ec340d\") " pod="openshift-cluster-node-tuning-operator/tuned-ldvhf"
Oct 30 21:50:20.802343 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.802218    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fcjg4\" (UniqueName: \"kubernetes.io/projected/73c208eb-ffc0-46f2-8bd4-05ade5adb529-kube-api-access-fcjg4\") pod \"network-check-target-jmbng\" (UID: \"73c208eb-ffc0-46f2-8bd4-05ade5adb529\") " pod="openshift-network-diagnostics/network-check-target-jmbng"
Oct 30 21:50:20.802343 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:20.802228    5510 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/31a389bb-f36f-4e53-9654-5f4648dae38a-kube-api-access-rb2wf podName:31a389bb-f36f-4e53-9654-5f4648dae38a nodeName:}" failed. No retries permitted until 2026-10-30 21:50:21.30221288 +0000 UTC m=+1.863022864 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-rb2wf" (UniqueName: "kubernetes.io/projected/31a389bb-f36f-4e53-9654-5f4648dae38a-kube-api-access-rb2wf") pod "console-748df94584-88cjt" (UID: "31a389bb-f36f-4e53-9654-5f4648dae38a") : [object "openshift-console"/"kube-root-ca.crt" not registered, object "openshift-console"/"openshift-service-ca.crt" not registered]
Oct 30 21:50:20.802819 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.802465    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tq25h\" (UniqueName: \"kubernetes.io/projected/dc220f0e-8a21-4d7f-96b3-e7055c242f42-kube-api-access-tq25h\") pod \"multus-additional-cni-plugins-kggmc\" (UID: \"dc220f0e-8a21-4d7f-96b3-e7055c242f42\") " pod="openshift-multus/multus-additional-cni-plugins-kggmc"
Oct 30 21:50:20.802819 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.802468    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ggf4d\" (UniqueName: \"kubernetes.io/projected/7d6190d5-6574-4704-97cb-2241398548e3-kube-api-access-ggf4d\") pod \"insights-runtime-extractor-6gjbf\" (UID: \"7d6190d5-6574-4704-97cb-2241398548e3\") " pod="openshift-insights/insights-runtime-extractor-6gjbf"
Oct 30 21:50:20.802819 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.802579    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zxwjt\" (UniqueName: \"kubernetes.io/projected/52eb01f8-e86a-4458-8875-639394a87b30-kube-api-access-zxwjt\") pod \"network-metrics-daemon-vmrkz\" (UID: \"52eb01f8-e86a-4458-8875-639394a87b30\") " pod="openshift-multus/network-metrics-daemon-vmrkz"
Oct 30 21:50:20.802819 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.802767    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v4d5j\" (UniqueName: \"kubernetes.io/projected/c8b13e55-b3c7-4656-9845-16749704265f-kube-api-access-v4d5j\") pod \"multus-v8j9s\" (UID: \"c8b13e55-b3c7-4656-9845-16749704265f\") " pod="openshift-multus/multus-v8j9s"
Oct 30 21:50:20.803075 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.802824    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vg89l\" (UniqueName: \"kubernetes.io/projected/d7c355fa-2141-463e-8cbc-bae3c44a16f1-kube-api-access-vg89l\") pod \"aws-ebs-csi-driver-node-cqxms\" (UID: \"d7c355fa-2141-463e-8cbc-bae3c44a16f1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cqxms"
Oct 30 21:50:20.803075 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.802990    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f2qbm\" (UniqueName: \"kubernetes.io/projected/3abd55eb-d1d8-4fdf-b82c-d35894b652ac-kube-api-access-f2qbm\") pod \"openshift-state-metrics-7bd7489957-h95jn\" (UID: \"3abd55eb-d1d8-4fdf-b82c-d35894b652ac\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-h95jn"
Oct 30 21:50:20.803075 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.803015    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9krk9\" (UniqueName: \"kubernetes.io/projected/db4b06a3-45e5-4d45-9d7a-407d117d9454-kube-api-access-9krk9\") pod \"node-resolver-krtnr\" (UID: \"db4b06a3-45e5-4d45-9d7a-407d117d9454\") " pod="openshift-dns/node-resolver-krtnr"
Oct 30 21:50:20.803225 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.803188    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ttgkm\" (UniqueName: \"kubernetes.io/projected/8c88b730-5cf1-4434-950f-4a3aa8f739cd-kube-api-access-ttgkm\") pod \"thanos-querier-7c848fbc95-8jm99\" (UID: \"8c88b730-5cf1-4434-950f-4a3aa8f739cd\") " pod="openshift-monitoring/thanos-querier-7c848fbc95-8jm99"
Oct 30 21:50:20.803281 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.803273    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9gzpz\" (UniqueName: \"kubernetes.io/projected/0e0d17a1-7742-42b0-9d9b-740bb945e454-kube-api-access-9gzpz\") pod \"telemeter-client-bb95b88bf-g9f94\" (UID: \"0e0d17a1-7742-42b0-9d9b-740bb945e454\") " pod="openshift-monitoring/telemeter-client-bb95b88bf-g9f94"
Oct 30 21:50:20.803326 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.803279    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vh5d6\" (UniqueName: \"kubernetes.io/projected/d8dc5e17-81b4-413a-a48f-6175d0ed3ec0-kube-api-access-vh5d6\") pod \"ingress-canary-t8tbp\" (UID: \"d8dc5e17-81b4-413a-a48f-6175d0ed3ec0\") " pod="openshift-ingress-canary/ingress-canary-t8tbp"
Oct 30 21:50:20.814769 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.814746    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/8f2f95c4-21b2-477b-a2cb-7c8590ec340d-etc-sysconfig\") pod \"tuned-ldvhf\" (UID: \"8f2f95c4-21b2-477b-a2cb-7c8590ec340d\") " pod="openshift-cluster-node-tuning-operator/tuned-ldvhf"
Oct 30 21:50:20.814851 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.814771    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/8f2f95c4-21b2-477b-a2cb-7c8590ec340d-etc-sysctl-conf\") pod \"tuned-ldvhf\" (UID: \"8f2f95c4-21b2-477b-a2cb-7c8590ec340d\") " pod="openshift-cluster-node-tuning-operator/tuned-ldvhf"
Oct 30 21:50:20.814851 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.814795    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/c8b13e55-b3c7-4656-9845-16749704265f-hostroot\") pod \"multus-v8j9s\" (UID: \"c8b13e55-b3c7-4656-9845-16749704265f\") " pod="openshift-multus/multus-v8j9s"
Oct 30 21:50:20.814851 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.814821    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8f2f95c4-21b2-477b-a2cb-7c8590ec340d-lib-modules\") pod \"tuned-ldvhf\" (UID: \"8f2f95c4-21b2-477b-a2cb-7c8590ec340d\") " pod="openshift-cluster-node-tuning-operator/tuned-ldvhf"
Oct 30 21:50:20.814851 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.814841    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d5e5ac6e-d37d-457d-9ca7-93065f5925ff-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-wrxww\" (UID: \"d5e5ac6e-d37d-457d-9ca7-93065f5925ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrxww"
Oct 30 21:50:20.815041 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.814866    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d5e5ac6e-d37d-457d-9ca7-93065f5925ff-var-lib-openvswitch\") pod \"ovnkube-node-wrxww\" (UID: \"d5e5ac6e-d37d-457d-9ca7-93065f5925ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrxww"
Oct 30 21:50:20.815041 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.814875    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/c8b13e55-b3c7-4656-9845-16749704265f-hostroot\") pod \"multus-v8j9s\" (UID: \"c8b13e55-b3c7-4656-9845-16749704265f\") " pod="openshift-multus/multus-v8j9s"
Oct 30 21:50:20.815041 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.814890    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d5e5ac6e-d37d-457d-9ca7-93065f5925ff-etc-openvswitch\") pod \"ovnkube-node-wrxww\" (UID: \"d5e5ac6e-d37d-457d-9ca7-93065f5925ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrxww"
Oct 30 21:50:20.815041 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.814879    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/8f2f95c4-21b2-477b-a2cb-7c8590ec340d-etc-sysconfig\") pod \"tuned-ldvhf\" (UID: \"8f2f95c4-21b2-477b-a2cb-7c8590ec340d\") " pod="openshift-cluster-node-tuning-operator/tuned-ldvhf"
Oct 30 21:50:20.815041 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.814908    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d5e5ac6e-d37d-457d-9ca7-93065f5925ff-var-lib-openvswitch\") pod \"ovnkube-node-wrxww\" (UID: \"d5e5ac6e-d37d-457d-9ca7-93065f5925ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrxww"
Oct 30 21:50:20.815041 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.814914    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d7c355fa-2141-463e-8cbc-bae3c44a16f1-etc-selinux\") pod \"aws-ebs-csi-driver-node-cqxms\" (UID: \"d7c355fa-2141-463e-8cbc-bae3c44a16f1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cqxms"
Oct 30 21:50:20.815041 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.814936    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d5e5ac6e-d37d-457d-9ca7-93065f5925ff-etc-openvswitch\") pod \"ovnkube-node-wrxww\" (UID: \"d5e5ac6e-d37d-457d-9ca7-93065f5925ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrxww"
Oct 30 21:50:20.815041 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.814941    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/d414339d-1a64-4a2a-ad15-805b71ebfd1b-node-exporter-wtmp\") pod \"node-exporter-mpsnt\" (UID: \"d414339d-1a64-4a2a-ad15-805b71ebfd1b\") " pod="openshift-monitoring/node-exporter-mpsnt"
Oct 30 21:50:20.815041 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.814949    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8f2f95c4-21b2-477b-a2cb-7c8590ec340d-lib-modules\") pod \"tuned-ldvhf\" (UID: \"8f2f95c4-21b2-477b-a2cb-7c8590ec340d\") " pod="openshift-cluster-node-tuning-operator/tuned-ldvhf"
Oct 30 21:50:20.815041 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.814953    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/8f2f95c4-21b2-477b-a2cb-7c8590ec340d-etc-sysctl-conf\") pod \"tuned-ldvhf\" (UID: \"8f2f95c4-21b2-477b-a2cb-7c8590ec340d\") " pod="openshift-cluster-node-tuning-operator/tuned-ldvhf"
Oct 30 21:50:20.815041 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.814978    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/d414339d-1a64-4a2a-ad15-805b71ebfd1b-node-exporter-wtmp\") pod \"node-exporter-mpsnt\" (UID: \"d414339d-1a64-4a2a-ad15-805b71ebfd1b\") " pod="openshift-monitoring/node-exporter-mpsnt"
Oct 30 21:50:20.815041 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.814980    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d5e5ac6e-d37d-457d-9ca7-93065f5925ff-host-slash\") pod \"ovnkube-node-wrxww\" (UID: \"d5e5ac6e-d37d-457d-9ca7-93065f5925ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrxww"
Oct 30 21:50:20.815041 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.814999    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d5e5ac6e-d37d-457d-9ca7-93065f5925ff-host-slash\") pod \"ovnkube-node-wrxww\" (UID: \"d5e5ac6e-d37d-457d-9ca7-93065f5925ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrxww"
Oct 30 21:50:20.815041 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.814980    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d5e5ac6e-d37d-457d-9ca7-93065f5925ff-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-wrxww\" (UID: \"d5e5ac6e-d37d-457d-9ca7-93065f5925ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrxww"
Oct 30 21:50:20.815041 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.815010    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/d5e5ac6e-d37d-457d-9ca7-93065f5925ff-run-ovn\") pod \"ovnkube-node-wrxww\" (UID: \"d5e5ac6e-d37d-457d-9ca7-93065f5925ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrxww"
Oct 30 21:50:20.815041 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.815026    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/8f2f95c4-21b2-477b-a2cb-7c8590ec340d-etc-sysctl-d\") pod \"tuned-ldvhf\" (UID: \"8f2f95c4-21b2-477b-a2cb-7c8590ec340d\") " pod="openshift-cluster-node-tuning-operator/tuned-ldvhf"
Oct 30 21:50:20.815041 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.814959    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d7c355fa-2141-463e-8cbc-bae3c44a16f1-etc-selinux\") pod \"aws-ebs-csi-driver-node-cqxms\" (UID: \"d7c355fa-2141-463e-8cbc-bae3c44a16f1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cqxms"
Oct 30 21:50:20.815041 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.815041    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/d5e5ac6e-d37d-457d-9ca7-93065f5925ff-host-cni-netd\") pod \"ovnkube-node-wrxww\" (UID: \"d5e5ac6e-d37d-457d-9ca7-93065f5925ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrxww"
Oct 30 21:50:20.815557 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.815064    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/d5e5ac6e-d37d-457d-9ca7-93065f5925ff-run-ovn\") pod \"ovnkube-node-wrxww\" (UID: \"d5e5ac6e-d37d-457d-9ca7-93065f5925ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrxww"
Oct 30 21:50:20.815557 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.815088    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/d5e5ac6e-d37d-457d-9ca7-93065f5925ff-host-cni-netd\") pod \"ovnkube-node-wrxww\" (UID: \"d5e5ac6e-d37d-457d-9ca7-93065f5925ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrxww"
Oct 30 21:50:20.815557 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.815092    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8f2f95c4-21b2-477b-a2cb-7c8590ec340d-etc-kubernetes\") pod \"tuned-ldvhf\" (UID: \"8f2f95c4-21b2-477b-a2cb-7c8590ec340d\") " pod="openshift-cluster-node-tuning-operator/tuned-ldvhf"
Oct 30 21:50:20.815557 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.815107    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/8f2f95c4-21b2-477b-a2cb-7c8590ec340d-etc-sysctl-d\") pod \"tuned-ldvhf\" (UID: \"8f2f95c4-21b2-477b-a2cb-7c8590ec340d\") " pod="openshift-cluster-node-tuning-operator/tuned-ldvhf"
Oct 30 21:50:20.815557 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.815116    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d7c355fa-2141-463e-8cbc-bae3c44a16f1-socket-dir\") pod \"aws-ebs-csi-driver-node-cqxms\" (UID: \"d7c355fa-2141-463e-8cbc-bae3c44a16f1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cqxms"
Oct 30 21:50:20.815557 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.815126    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8f2f95c4-21b2-477b-a2cb-7c8590ec340d-etc-kubernetes\") pod \"tuned-ldvhf\" (UID: \"8f2f95c4-21b2-477b-a2cb-7c8590ec340d\") " pod="openshift-cluster-node-tuning-operator/tuned-ldvhf"
Oct 30 21:50:20.815557 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.815155    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d7c355fa-2141-463e-8cbc-bae3c44a16f1-socket-dir\") pod \"aws-ebs-csi-driver-node-cqxms\" (UID: \"d7c355fa-2141-463e-8cbc-bae3c44a16f1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cqxms"
Oct 30 21:50:20.815557 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.815161    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/04465eda-e6db-42d3-8d5f-c9be668038a9-host\") pod \"node-ca-xkrx5\" (UID: \"04465eda-e6db-42d3-8d5f-c9be668038a9\") " pod="openshift-image-registry/node-ca-xkrx5"
Oct 30 21:50:20.815557 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.815194    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/04465eda-e6db-42d3-8d5f-c9be668038a9-host\") pod \"node-ca-xkrx5\" (UID: \"04465eda-e6db-42d3-8d5f-c9be668038a9\") " pod="openshift-image-registry/node-ca-xkrx5"
Oct 30 21:50:20.815557 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.815198    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/d5e5ac6e-d37d-457d-9ca7-93065f5925ff-host-kubelet\") pod \"ovnkube-node-wrxww\" (UID: \"d5e5ac6e-d37d-457d-9ca7-93065f5925ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrxww"
Oct 30 21:50:20.815557 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.815220    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/d5e5ac6e-d37d-457d-9ca7-93065f5925ff-host-kubelet\") pod \"ovnkube-node-wrxww\" (UID: \"d5e5ac6e-d37d-457d-9ca7-93065f5925ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrxww"
Oct 30 21:50:20.815557 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.815222    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/c8b13e55-b3c7-4656-9845-16749704265f-host-run-k8s-cni-cncf-io\") pod \"multus-v8j9s\" (UID: \"c8b13e55-b3c7-4656-9845-16749704265f\") " pod="openshift-multus/multus-v8j9s"
Oct 30 21:50:20.815557 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.815264    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/d5e5ac6e-d37d-457d-9ca7-93065f5925ff-run-systemd\") pod \"ovnkube-node-wrxww\" (UID: \"d5e5ac6e-d37d-457d-9ca7-93065f5925ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrxww"
Oct 30 21:50:20.815557 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.815281    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d5e5ac6e-d37d-457d-9ca7-93065f5925ff-host-run-ovn-kubernetes\") pod \"ovnkube-node-wrxww\" (UID: \"d5e5ac6e-d37d-457d-9ca7-93065f5925ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrxww"
Oct 30 21:50:20.815557 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.815304    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/c8b13e55-b3c7-4656-9845-16749704265f-host-run-k8s-cni-cncf-io\") pod \"multus-v8j9s\" (UID: \"c8b13e55-b3c7-4656-9845-16749704265f\") " pod="openshift-multus/multus-v8j9s"
Oct 30 21:50:20.815557 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.815304    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/d414339d-1a64-4a2a-ad15-805b71ebfd1b-root\") pod \"node-exporter-mpsnt\" (UID: \"d414339d-1a64-4a2a-ad15-805b71ebfd1b\") " pod="openshift-monitoring/node-exporter-mpsnt"
Oct 30 21:50:20.815557 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.815317    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/d5e5ac6e-d37d-457d-9ca7-93065f5925ff-run-systemd\") pod \"ovnkube-node-wrxww\" (UID: \"d5e5ac6e-d37d-457d-9ca7-93065f5925ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrxww"
Oct 30 21:50:20.815557 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.815335    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0ab70415-9f1d-4988-a228-df6ed621f0d8-host-slash\") pod \"iptables-alerter-l9d98\" (UID: \"0ab70415-9f1d-4988-a228-df6ed621f0d8\") " pod="openshift-network-operator/iptables-alerter-l9d98"
Oct 30 21:50:20.815557 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.815337    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/d414339d-1a64-4a2a-ad15-805b71ebfd1b-root\") pod \"node-exporter-mpsnt\" (UID: \"d414339d-1a64-4a2a-ad15-805b71ebfd1b\") " pod="openshift-monitoring/node-exporter-mpsnt"
Oct 30 21:50:20.815557 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.815362    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c8b13e55-b3c7-4656-9845-16749704265f-multus-conf-dir\") pod \"multus-v8j9s\" (UID: \"c8b13e55-b3c7-4656-9845-16749704265f\") " pod="openshift-multus/multus-v8j9s"
Oct 30 21:50:20.815557 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.815366    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0ab70415-9f1d-4988-a228-df6ed621f0d8-host-slash\") pod \"iptables-alerter-l9d98\" (UID: \"0ab70415-9f1d-4988-a228-df6ed621f0d8\") " pod="openshift-network-operator/iptables-alerter-l9d98"
Oct 30 21:50:20.815557 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.815393    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8f2f95c4-21b2-477b-a2cb-7c8590ec340d-host\") pod \"tuned-ldvhf\" (UID: \"8f2f95c4-21b2-477b-a2cb-7c8590ec340d\") " pod="openshift-cluster-node-tuning-operator/tuned-ldvhf"
Oct 30 21:50:20.815557 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.815339    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d5e5ac6e-d37d-457d-9ca7-93065f5925ff-host-run-ovn-kubernetes\") pod \"ovnkube-node-wrxww\" (UID: \"d5e5ac6e-d37d-457d-9ca7-93065f5925ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrxww"
Oct 30 21:50:20.815557 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.815417    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c8b13e55-b3c7-4656-9845-16749704265f-multus-conf-dir\") pod \"multus-v8j9s\" (UID: \"c8b13e55-b3c7-4656-9845-16749704265f\") " pod="openshift-multus/multus-v8j9s"
Oct 30 21:50:20.815557 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.815420    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e810bcd1-78da-409f-b86e-08c47031939e-dbus\") pod \"global-pull-secret-syncer-68jrf\" (UID: \"e810bcd1-78da-409f-b86e-08c47031939e\") " pod="kube-system/global-pull-secret-syncer-68jrf"
Oct 30 21:50:20.815557 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.815454    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8f2f95c4-21b2-477b-a2cb-7c8590ec340d-host\") pod \"tuned-ldvhf\" (UID: \"8f2f95c4-21b2-477b-a2cb-7c8590ec340d\") " pod="openshift-cluster-node-tuning-operator/tuned-ldvhf"
Oct 30 21:50:20.815557 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.815458    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c8b13e55-b3c7-4656-9845-16749704265f-multus-cni-dir\") pod \"multus-v8j9s\" (UID: \"c8b13e55-b3c7-4656-9845-16749704265f\") " pod="openshift-multus/multus-v8j9s"
Oct 30 21:50:20.815557 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.815481    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d414339d-1a64-4a2a-ad15-805b71ebfd1b-sys\") pod \"node-exporter-mpsnt\" (UID: \"d414339d-1a64-4a2a-ad15-805b71ebfd1b\") " pod="openshift-monitoring/node-exporter-mpsnt"
Oct 30 21:50:20.815557 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.815496    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/8f2f95c4-21b2-477b-a2cb-7c8590ec340d-etc-systemd\") pod \"tuned-ldvhf\" (UID: \"8f2f95c4-21b2-477b-a2cb-7c8590ec340d\") " pod="openshift-cluster-node-tuning-operator/tuned-ldvhf"
Oct 30 21:50:20.815557 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.815517    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e810bcd1-78da-409f-b86e-08c47031939e-dbus\") pod \"global-pull-secret-syncer-68jrf\" (UID: \"e810bcd1-78da-409f-b86e-08c47031939e\") " pod="kube-system/global-pull-secret-syncer-68jrf"
Oct 30 21:50:20.815557 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.815529    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d414339d-1a64-4a2a-ad15-805b71ebfd1b-sys\") pod \"node-exporter-mpsnt\" (UID: \"d414339d-1a64-4a2a-ad15-805b71ebfd1b\") " pod="openshift-monitoring/node-exporter-mpsnt"
Oct 30 21:50:20.815557 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.815536    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d7c355fa-2141-463e-8cbc-bae3c44a16f1-kubelet-dir\") pod \"aws-ebs-csi-driver-node-cqxms\" (UID: \"d7c355fa-2141-463e-8cbc-bae3c44a16f1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cqxms"
Oct 30 21:50:20.815557 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.815563    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d7c355fa-2141-463e-8cbc-bae3c44a16f1-kubelet-dir\") pod \"aws-ebs-csi-driver-node-cqxms\" (UID: \"d7c355fa-2141-463e-8cbc-bae3c44a16f1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cqxms"
Oct 30 21:50:20.815557 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.815554    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c8b13e55-b3c7-4656-9845-16749704265f-multus-cni-dir\") pod \"multus-v8j9s\" (UID: \"c8b13e55-b3c7-4656-9845-16749704265f\") " pod="openshift-multus/multus-v8j9s"
Oct 30 21:50:20.815557 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.815582    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/dc220f0e-8a21-4d7f-96b3-e7055c242f42-tuning-conf-dir\") pod \"multus-additional-cni-plugins-kggmc\" (UID: \"dc220f0e-8a21-4d7f-96b3-e7055c242f42\") " pod="openshift-multus/multus-additional-cni-plugins-kggmc"
Oct 30 21:50:20.816748 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.815536    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/8f2f95c4-21b2-477b-a2cb-7c8590ec340d-etc-systemd\") pod \"tuned-ldvhf\" (UID: \"8f2f95c4-21b2-477b-a2cb-7c8590ec340d\") " pod="openshift-cluster-node-tuning-operator/tuned-ldvhf"
Oct 30 21:50:20.816748 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.815604    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c8b13e55-b3c7-4656-9845-16749704265f-host-var-lib-cni-bin\") pod \"multus-v8j9s\" (UID: \"c8b13e55-b3c7-4656-9845-16749704265f\") " pod="openshift-multus/multus-v8j9s"
Oct 30 21:50:20.816748 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.815630    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8f2f95c4-21b2-477b-a2cb-7c8590ec340d-sys\") pod \"tuned-ldvhf\" (UID: \"8f2f95c4-21b2-477b-a2cb-7c8590ec340d\") " pod="openshift-cluster-node-tuning-operator/tuned-ldvhf"
Oct 30 21:50:20.816748 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.815646    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/8f2f95c4-21b2-477b-a2cb-7c8590ec340d-etc-modprobe-d\") pod \"tuned-ldvhf\" (UID: \"8f2f95c4-21b2-477b-a2cb-7c8590ec340d\") " pod="openshift-cluster-node-tuning-operator/tuned-ldvhf"
Oct 30 21:50:20.816748 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.815630    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/dc220f0e-8a21-4d7f-96b3-e7055c242f42-tuning-conf-dir\") pod \"multus-additional-cni-plugins-kggmc\" (UID: \"dc220f0e-8a21-4d7f-96b3-e7055c242f42\") " pod="openshift-multus/multus-additional-cni-plugins-kggmc"
Oct 30 21:50:20.816748 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.815678    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dc220f0e-8a21-4d7f-96b3-e7055c242f42-system-cni-dir\") pod \"multus-additional-cni-plugins-kggmc\" (UID: \"dc220f0e-8a21-4d7f-96b3-e7055c242f42\") " pod="openshift-multus/multus-additional-cni-plugins-kggmc"
Oct 30 21:50:20.816748 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.815703    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/c8b13e55-b3c7-4656-9845-16749704265f-host-run-multus-certs\") pod \"multus-v8j9s\" (UID: \"c8b13e55-b3c7-4656-9845-16749704265f\") " pod="openshift-multus/multus-v8j9s"
Oct 30 21:50:20.816748 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.815704    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c8b13e55-b3c7-4656-9845-16749704265f-host-var-lib-cni-bin\") pod \"multus-v8j9s\" (UID: \"c8b13e55-b3c7-4656-9845-16749704265f\") " pod="openshift-multus/multus-v8j9s"
Oct 30 21:50:20.816748 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.815727    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d7c355fa-2141-463e-8cbc-bae3c44a16f1-sys-fs\") pod \"aws-ebs-csi-driver-node-cqxms\" (UID: \"d7c355fa-2141-463e-8cbc-bae3c44a16f1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cqxms"
Oct 30 21:50:20.816748 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.815703    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8f2f95c4-21b2-477b-a2cb-7c8590ec340d-sys\") pod \"tuned-ldvhf\" (UID: \"8f2f95c4-21b2-477b-a2cb-7c8590ec340d\") " pod="openshift-cluster-node-tuning-operator/tuned-ldvhf"
Oct 30 21:50:20.816748 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.815741    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/c8b13e55-b3c7-4656-9845-16749704265f-host-run-multus-certs\") pod \"multus-v8j9s\" (UID: \"c8b13e55-b3c7-4656-9845-16749704265f\") " pod="openshift-multus/multus-v8j9s"
Oct 30 21:50:20.816748 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.815734    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dc220f0e-8a21-4d7f-96b3-e7055c242f42-system-cni-dir\") pod \"multus-additional-cni-plugins-kggmc\" (UID: \"dc220f0e-8a21-4d7f-96b3-e7055c242f42\") " pod="openshift-multus/multus-additional-cni-plugins-kggmc"
Oct 30 21:50:20.816748 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.815757    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c8b13e55-b3c7-4656-9845-16749704265f-system-cni-dir\") pod \"multus-v8j9s\" (UID: \"c8b13e55-b3c7-4656-9845-16749704265f\") " pod="openshift-multus/multus-v8j9s"
Oct 30 21:50:20.816748 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.815773    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/c8b13e55-b3c7-4656-9845-16749704265f-multus-socket-dir-parent\") pod \"multus-v8j9s\" (UID: \"c8b13e55-b3c7-4656-9845-16749704265f\") " pod="openshift-multus/multus-v8j9s"
Oct 30 21:50:20.816748 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.815787    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d7c355fa-2141-463e-8cbc-bae3c44a16f1-sys-fs\") pod \"aws-ebs-csi-driver-node-cqxms\" (UID: \"d7c355fa-2141-463e-8cbc-bae3c44a16f1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cqxms"
Oct 30 21:50:20.816748 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.815788    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d5e5ac6e-d37d-457d-9ca7-93065f5925ff-host-run-netns\") pod \"ovnkube-node-wrxww\" (UID: \"d5e5ac6e-d37d-457d-9ca7-93065f5925ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrxww"
Oct 30 21:50:20.816748 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.815839    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/c8b13e55-b3c7-4656-9845-16749704265f-multus-socket-dir-parent\") pod \"multus-v8j9s\" (UID: \"c8b13e55-b3c7-4656-9845-16749704265f\") " pod="openshift-multus/multus-v8j9s"
Oct 30 21:50:20.816748 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.815784    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/8f2f95c4-21b2-477b-a2cb-7c8590ec340d-etc-modprobe-d\") pod \"tuned-ldvhf\" (UID: \"8f2f95c4-21b2-477b-a2cb-7c8590ec340d\") " pod="openshift-cluster-node-tuning-operator/tuned-ldvhf"
Oct 30 21:50:20.816748 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.815849    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/7d6190d5-6574-4704-97cb-2241398548e3-crio-socket\") pod \"insights-runtime-extractor-6gjbf\" (UID: \"7d6190d5-6574-4704-97cb-2241398548e3\") " pod="openshift-insights/insights-runtime-extractor-6gjbf"
Oct 30 21:50:20.816748 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.815841    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c8b13e55-b3c7-4656-9845-16749704265f-system-cni-dir\") pod \"multus-v8j9s\" (UID: \"c8b13e55-b3c7-4656-9845-16749704265f\") " pod="openshift-multus/multus-v8j9s"
Oct 30 21:50:20.816748 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.815864    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d5e5ac6e-d37d-457d-9ca7-93065f5925ff-host-run-netns\") pod \"ovnkube-node-wrxww\" (UID: \"d5e5ac6e-d37d-457d-9ca7-93065f5925ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrxww"
Oct 30 21:50:20.816748 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.815885    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c8b13e55-b3c7-4656-9845-16749704265f-cnibin\") pod \"multus-v8j9s\" (UID: \"c8b13e55-b3c7-4656-9845-16749704265f\") " pod="openshift-multus/multus-v8j9s"
Oct 30 21:50:20.816748 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.815890    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/7d6190d5-6574-4704-97cb-2241398548e3-crio-socket\") pod \"insights-runtime-extractor-6gjbf\" (UID: \"7d6190d5-6574-4704-97cb-2241398548e3\") " pod="openshift-insights/insights-runtime-extractor-6gjbf"
Oct 30 21:50:20.816748 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.815913    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/c8b13e55-b3c7-4656-9845-16749704265f-host-var-lib-cni-multus\") pod \"multus-v8j9s\" (UID: \"c8b13e55-b3c7-4656-9845-16749704265f\") " pod="openshift-multus/multus-v8j9s"
Oct 30 21:50:20.816748 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.815932    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d5e5ac6e-d37d-457d-9ca7-93065f5925ff-run-openvswitch\") pod \"ovnkube-node-wrxww\" (UID: \"d5e5ac6e-d37d-457d-9ca7-93065f5925ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrxww"
Oct 30 21:50:20.816748 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.815946    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/d5e5ac6e-d37d-457d-9ca7-93065f5925ff-node-log\") pod \"ovnkube-node-wrxww\" (UID: \"d5e5ac6e-d37d-457d-9ca7-93065f5925ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrxww"
Oct 30 21:50:20.816748 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.815965    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/d5e5ac6e-d37d-457d-9ca7-93065f5925ff-node-log\") pod \"ovnkube-node-wrxww\" (UID: \"d5e5ac6e-d37d-457d-9ca7-93065f5925ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrxww"
Oct 30 21:50:20.816748 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.815968    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/dc220f0e-8a21-4d7f-96b3-e7055c242f42-cnibin\") pod \"multus-additional-cni-plugins-kggmc\" (UID: \"dc220f0e-8a21-4d7f-96b3-e7055c242f42\") " pod="openshift-multus/multus-additional-cni-plugins-kggmc"
Oct 30 21:50:20.816748 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.815984    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d7c355fa-2141-463e-8cbc-bae3c44a16f1-registration-dir\") pod \"aws-ebs-csi-driver-node-cqxms\" (UID: \"d7c355fa-2141-463e-8cbc-bae3c44a16f1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cqxms"
Oct 30 21:50:20.816748 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.815982    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d5e5ac6e-d37d-457d-9ca7-93065f5925ff-run-openvswitch\") pod \"ovnkube-node-wrxww\" (UID: \"d5e5ac6e-d37d-457d-9ca7-93065f5925ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrxww"
Oct 30 21:50:20.816748 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.815988    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/dc220f0e-8a21-4d7f-96b3-e7055c242f42-cnibin\") pod \"multus-additional-cni-plugins-kggmc\" (UID: \"dc220f0e-8a21-4d7f-96b3-e7055c242f42\") " pod="openshift-multus/multus-additional-cni-plugins-kggmc"
Oct 30 21:50:20.816748 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.815947    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c8b13e55-b3c7-4656-9845-16749704265f-cnibin\") pod \"multus-v8j9s\" (UID: \"c8b13e55-b3c7-4656-9845-16749704265f\") " pod="openshift-multus/multus-v8j9s"
Oct 30 21:50:20.816748 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.815966    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/c8b13e55-b3c7-4656-9845-16749704265f-host-var-lib-cni-multus\") pod \"multus-v8j9s\" (UID: \"c8b13e55-b3c7-4656-9845-16749704265f\") " pod="openshift-multus/multus-v8j9s"
Oct 30 21:50:20.816748 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.816017    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d5e5ac6e-d37d-457d-9ca7-93065f5925ff-host-cni-bin\") pod \"ovnkube-node-wrxww\" (UID: \"d5e5ac6e-d37d-457d-9ca7-93065f5925ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrxww"
Oct 30 21:50:20.816748 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.816044    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d5e5ac6e-d37d-457d-9ca7-93065f5925ff-host-cni-bin\") pod \"ovnkube-node-wrxww\" (UID: \"d5e5ac6e-d37d-457d-9ca7-93065f5925ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrxww"
Oct 30 21:50:20.816748 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.816044    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e810bcd1-78da-409f-b86e-08c47031939e-kubelet-config\") pod \"global-pull-secret-syncer-68jrf\" (UID: \"e810bcd1-78da-409f-b86e-08c47031939e\") " pod="kube-system/global-pull-secret-syncer-68jrf"
Oct 30 21:50:20.816748 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.816066    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/8f2f95c4-21b2-477b-a2cb-7c8590ec340d-run\") pod \"tuned-ldvhf\" (UID: \"8f2f95c4-21b2-477b-a2cb-7c8590ec340d\") " pod="openshift-cluster-node-tuning-operator/tuned-ldvhf"
Oct 30 21:50:20.816748 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.816021    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d7c355fa-2141-463e-8cbc-bae3c44a16f1-registration-dir\") pod \"aws-ebs-csi-driver-node-cqxms\" (UID: \"d7c355fa-2141-463e-8cbc-bae3c44a16f1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cqxms"
Oct 30 21:50:20.816748 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.816087    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e810bcd1-78da-409f-b86e-08c47031939e-kubelet-config\") pod \"global-pull-secret-syncer-68jrf\" (UID: \"e810bcd1-78da-409f-b86e-08c47031939e\") " pod="kube-system/global-pull-secret-syncer-68jrf"
Oct 30 21:50:20.816748 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.816100    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8f2f95c4-21b2-477b-a2cb-7c8590ec340d-var-lib-kubelet\") pod \"tuned-ldvhf\" (UID: \"8f2f95c4-21b2-477b-a2cb-7c8590ec340d\") " pod="openshift-cluster-node-tuning-operator/tuned-ldvhf"
Oct 30 21:50:20.816748 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.816125    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/d5e5ac6e-d37d-457d-9ca7-93065f5925ff-systemd-units\") pod \"ovnkube-node-wrxww\" (UID: \"d5e5ac6e-d37d-457d-9ca7-93065f5925ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrxww"
Oct 30 21:50:20.816748 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.816126    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/8f2f95c4-21b2-477b-a2cb-7c8590ec340d-run\") pod \"tuned-ldvhf\" (UID: \"8f2f95c4-21b2-477b-a2cb-7c8590ec340d\") " pod="openshift-cluster-node-tuning-operator/tuned-ldvhf"
Oct 30 21:50:20.816748 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.816139    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8f2f95c4-21b2-477b-a2cb-7c8590ec340d-var-lib-kubelet\") pod \"tuned-ldvhf\" (UID: \"8f2f95c4-21b2-477b-a2cb-7c8590ec340d\") " pod="openshift-cluster-node-tuning-operator/tuned-ldvhf"
Oct 30 21:50:20.816748 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.816167    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/d5e5ac6e-d37d-457d-9ca7-93065f5925ff-systemd-units\") pod \"ovnkube-node-wrxww\" (UID: \"d5e5ac6e-d37d-457d-9ca7-93065f5925ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrxww"
Oct 30 21:50:20.816748 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.816191    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/db4b06a3-45e5-4d45-9d7a-407d117d9454-hosts-file\") pod \"node-resolver-krtnr\" (UID: \"db4b06a3-45e5-4d45-9d7a-407d117d9454\") " pod="openshift-dns/node-resolver-krtnr"
Oct 30 21:50:20.816748 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.816167    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/db4b06a3-45e5-4d45-9d7a-407d117d9454-hosts-file\") pod \"node-resolver-krtnr\" (UID: \"db4b06a3-45e5-4d45-9d7a-407d117d9454\") " pod="openshift-dns/node-resolver-krtnr"
Oct 30 21:50:20.816748 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.816230    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/d5e5ac6e-d37d-457d-9ca7-93065f5925ff-log-socket\") pod \"ovnkube-node-wrxww\" (UID: \"d5e5ac6e-d37d-457d-9ca7-93065f5925ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrxww"
Oct 30 21:50:20.816748 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.816249    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c8b13e55-b3c7-4656-9845-16749704265f-host-run-netns\") pod \"multus-v8j9s\" (UID: \"c8b13e55-b3c7-4656-9845-16749704265f\") " pod="openshift-multus/multus-v8j9s"
Oct 30 21:50:20.816748 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.816272    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d7c355fa-2141-463e-8cbc-bae3c44a16f1-device-dir\") pod \"aws-ebs-csi-driver-node-cqxms\" (UID: \"d7c355fa-2141-463e-8cbc-bae3c44a16f1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cqxms"
Oct 30 21:50:20.816748 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.816292    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c8b13e55-b3c7-4656-9845-16749704265f-host-var-lib-kubelet\") pod \"multus-v8j9s\" (UID: \"c8b13e55-b3c7-4656-9845-16749704265f\") " pod="openshift-multus/multus-v8j9s"
Oct 30 21:50:20.816748 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.816318    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c8b13e55-b3c7-4656-9845-16749704265f-etc-kubernetes\") pod \"multus-v8j9s\" (UID: \"c8b13e55-b3c7-4656-9845-16749704265f\") " pod="openshift-multus/multus-v8j9s"
Oct 30 21:50:20.816748 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.816319    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c8b13e55-b3c7-4656-9845-16749704265f-host-run-netns\") pod \"multus-v8j9s\" (UID: \"c8b13e55-b3c7-4656-9845-16749704265f\") " pod="openshift-multus/multus-v8j9s"
Oct 30 21:50:20.816748 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.816334    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d7c355fa-2141-463e-8cbc-bae3c44a16f1-device-dir\") pod \"aws-ebs-csi-driver-node-cqxms\" (UID: \"d7c355fa-2141-463e-8cbc-bae3c44a16f1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cqxms"
Oct 30 21:50:20.816748 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.816344    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c8b13e55-b3c7-4656-9845-16749704265f-host-var-lib-kubelet\") pod \"multus-v8j9s\" (UID: \"c8b13e55-b3c7-4656-9845-16749704265f\") " pod="openshift-multus/multus-v8j9s"
Oct 30 21:50:20.816748 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.816345    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/d5e5ac6e-d37d-457d-9ca7-93065f5925ff-log-socket\") pod \"ovnkube-node-wrxww\" (UID: \"d5e5ac6e-d37d-457d-9ca7-93065f5925ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-wrxww"
Oct 30 21:50:20.816748 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:20.816368    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c8b13e55-b3c7-4656-9845-16749704265f-etc-kubernetes\") pod \"multus-v8j9s\" (UID: \"c8b13e55-b3c7-4656-9845-16749704265f\") " pod="openshift-multus/multus-v8j9s"
Oct 30 21:50:21.033535 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:21.033498    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:21.219908 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:21.219875    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/31a389bb-f36f-4e53-9654-5f4648dae38a-service-ca\") pod \"console-748df94584-88cjt\" (UID: \"31a389bb-f36f-4e53-9654-5f4648dae38a\") " pod="openshift-console/console-748df94584-88cjt"
Oct 30 21:50:21.219908 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:21.219914    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-trusted-ca-bundle\") pod \"console-6d5866d6cd-b6dpl\" (UID: \"42d54b8a-4f48-46a7-8a34-1f0f29cd1602\") " pod="openshift-console/console-6d5866d6cd-b6dpl"
Oct 30 21:50:21.220125 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:21.219934    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/31a389bb-f36f-4e53-9654-5f4648dae38a-trusted-ca-bundle\") pod \"console-748df94584-88cjt\" (UID: \"31a389bb-f36f-4e53-9654-5f4648dae38a\") " pod="openshift-console/console-748df94584-88cjt"
Oct 30 21:50:21.220125 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:21.219989    5510 configmap.go:193] Couldn't get configMap openshift-console/trusted-ca-bundle: object "openshift-console"/"trusted-ca-bundle" not registered
Oct 30 21:50:21.220125 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:21.220000    5510 configmap.go:193] Couldn't get configMap openshift-console/trusted-ca-bundle: object "openshift-console"/"trusted-ca-bundle" not registered
Oct 30 21:50:21.220125 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:21.220032    5510 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/31a389bb-f36f-4e53-9654-5f4648dae38a-trusted-ca-bundle podName:31a389bb-f36f-4e53-9654-5f4648dae38a nodeName:}" failed. No retries permitted until 2026-10-30 21:50:22.220020939 +0000 UTC m=+2.780830922 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/31a389bb-f36f-4e53-9654-5f4648dae38a-trusted-ca-bundle") pod "console-748df94584-88cjt" (UID: "31a389bb-f36f-4e53-9654-5f4648dae38a") : object "openshift-console"/"trusted-ca-bundle" not registered
Oct 30 21:50:21.220125 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:21.220054    5510 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-trusted-ca-bundle podName:42d54b8a-4f48-46a7-8a34-1f0f29cd1602 nodeName:}" failed. No retries permitted until 2026-10-30 21:50:22.220042034 +0000 UTC m=+2.780852016 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-trusted-ca-bundle") pod "console-6d5866d6cd-b6dpl" (UID: "42d54b8a-4f48-46a7-8a34-1f0f29cd1602") : object "openshift-console"/"trusted-ca-bundle" not registered
Oct 30 21:50:21.220125 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:21.220074    5510 configmap.go:193] Couldn't get configMap openshift-console/service-ca: object "openshift-console"/"service-ca" not registered
Oct 30 21:50:21.220125 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:21.220100    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/31a389bb-f36f-4e53-9654-5f4648dae38a-oauth-serving-cert\") pod \"console-748df94584-88cjt\" (UID: \"31a389bb-f36f-4e53-9654-5f4648dae38a\") " pod="openshift-console/console-748df94584-88cjt"
Oct 30 21:50:21.220125 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:21.220114    5510 configmap.go:193] Couldn't get configMap openshift-console/oauth-serving-cert: object "openshift-console"/"oauth-serving-cert" not registered
Oct 30 21:50:21.220125 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:21.220121    5510 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/31a389bb-f36f-4e53-9654-5f4648dae38a-service-ca podName:31a389bb-f36f-4e53-9654-5f4648dae38a nodeName:}" failed. No retries permitted until 2026-10-30 21:50:22.22011306 +0000 UTC m=+2.780923044 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca" (UniqueName: "kubernetes.io/configmap/31a389bb-f36f-4e53-9654-5f4648dae38a-service-ca") pod "console-748df94584-88cjt" (UID: "31a389bb-f36f-4e53-9654-5f4648dae38a") : object "openshift-console"/"service-ca" not registered
Oct 30 21:50:21.220423 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:21.220138    5510 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/31a389bb-f36f-4e53-9654-5f4648dae38a-oauth-serving-cert podName:31a389bb-f36f-4e53-9654-5f4648dae38a nodeName:}" failed. No retries permitted until 2026-10-30 21:50:22.220132676 +0000 UTC m=+2.780942659 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "oauth-serving-cert" (UniqueName: "kubernetes.io/configmap/31a389bb-f36f-4e53-9654-5f4648dae38a-oauth-serving-cert") pod "console-748df94584-88cjt" (UID: "31a389bb-f36f-4e53-9654-5f4648dae38a") : object "openshift-console"/"oauth-serving-cert" not registered
Oct 30 21:50:21.220423 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:21.220173    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-console-serving-cert\") pod \"console-6d5866d6cd-b6dpl\" (UID: \"42d54b8a-4f48-46a7-8a34-1f0f29cd1602\") " pod="openshift-console/console-6d5866d6cd-b6dpl"
Oct 30 21:50:21.220423 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:21.220189    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-oauth-serving-cert\") pod \"console-6d5866d6cd-b6dpl\" (UID: \"42d54b8a-4f48-46a7-8a34-1f0f29cd1602\") " pod="openshift-console/console-6d5866d6cd-b6dpl"
Oct 30 21:50:21.220423 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:21.220218    5510 configmap.go:193] Couldn't get configMap openshift-console/oauth-serving-cert: object "openshift-console"/"oauth-serving-cert" not registered
Oct 30 21:50:21.220423 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:21.220239    5510 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-oauth-serving-cert podName:42d54b8a-4f48-46a7-8a34-1f0f29cd1602 nodeName:}" failed. No retries permitted until 2026-10-30 21:50:22.22023329 +0000 UTC m=+2.781043272 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "oauth-serving-cert" (UniqueName: "kubernetes.io/configmap/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-oauth-serving-cert") pod "console-6d5866d6cd-b6dpl" (UID: "42d54b8a-4f48-46a7-8a34-1f0f29cd1602") : object "openshift-console"/"oauth-serving-cert" not registered
Oct 30 21:50:21.220423 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:21.220244    5510 secret.go:189] Couldn't get secret openshift-console/console-serving-cert: object "openshift-console"/"console-serving-cert" not registered
Oct 30 21:50:21.220423 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:21.220267    5510 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-console-serving-cert podName:42d54b8a-4f48-46a7-8a34-1f0f29cd1602 nodeName:}" failed. No retries permitted until 2026-10-30 21:50:22.220259727 +0000 UTC m=+2.781069714 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "console-serving-cert" (UniqueName: "kubernetes.io/secret/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-console-serving-cert") pod "console-6d5866d6cd-b6dpl" (UID: "42d54b8a-4f48-46a7-8a34-1f0f29cd1602") : object "openshift-console"/"console-serving-cert" not registered
Oct 30 21:50:21.220423 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:21.220281    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/31a389bb-f36f-4e53-9654-5f4648dae38a-console-oauth-config\") pod \"console-748df94584-88cjt\" (UID: \"31a389bb-f36f-4e53-9654-5f4648dae38a\") " pod="openshift-console/console-748df94584-88cjt"
Oct 30 21:50:21.220423 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:21.220310    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-console-config\") pod \"console-6d5866d6cd-b6dpl\" (UID: \"42d54b8a-4f48-46a7-8a34-1f0f29cd1602\") " pod="openshift-console/console-6d5866d6cd-b6dpl"
Oct 30 21:50:21.220423 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:21.220341    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/31a389bb-f36f-4e53-9654-5f4648dae38a-console-config\") pod \"console-748df94584-88cjt\" (UID: \"31a389bb-f36f-4e53-9654-5f4648dae38a\") " pod="openshift-console/console-748df94584-88cjt"
Oct 30 21:50:21.220423 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:21.220365    5510 secret.go:189] Couldn't get secret openshift-console/console-oauth-config: object "openshift-console"/"console-oauth-config" not registered
Oct 30 21:50:21.220423 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:21.220388    5510 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/31a389bb-f36f-4e53-9654-5f4648dae38a-console-oauth-config podName:31a389bb-f36f-4e53-9654-5f4648dae38a nodeName:}" failed. No retries permitted until 2026-10-30 21:50:22.220381699 +0000 UTC m=+2.781191681 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "console-oauth-config" (UniqueName: "kubernetes.io/secret/31a389bb-f36f-4e53-9654-5f4648dae38a-console-oauth-config") pod "console-748df94584-88cjt" (UID: "31a389bb-f36f-4e53-9654-5f4648dae38a") : object "openshift-console"/"console-oauth-config" not registered
Oct 30 21:50:21.220423 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:21.220365    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-console-oauth-config\") pod \"console-6d5866d6cd-b6dpl\" (UID: \"42d54b8a-4f48-46a7-8a34-1f0f29cd1602\") " pod="openshift-console/console-6d5866d6cd-b6dpl"
Oct 30 21:50:21.220423 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:21.220397    5510 secret.go:189] Couldn't get secret openshift-console/console-oauth-config: object "openshift-console"/"console-oauth-config" not registered
Oct 30 21:50:21.220423 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:21.220407    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-service-ca\") pod \"console-6d5866d6cd-b6dpl\" (UID: \"42d54b8a-4f48-46a7-8a34-1f0f29cd1602\") " pod="openshift-console/console-6d5866d6cd-b6dpl"
Oct 30 21:50:21.220423 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:21.220424    5510 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-console-oauth-config podName:42d54b8a-4f48-46a7-8a34-1f0f29cd1602 nodeName:}" failed. No retries permitted until 2026-10-30 21:50:22.220414779 +0000 UTC m=+2.781224772 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "console-oauth-config" (UniqueName: "kubernetes.io/secret/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-console-oauth-config") pod "console-6d5866d6cd-b6dpl" (UID: "42d54b8a-4f48-46a7-8a34-1f0f29cd1602") : object "openshift-console"/"console-oauth-config" not registered
Oct 30 21:50:21.220423 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:21.220424    5510 configmap.go:193] Couldn't get configMap openshift-console/console-config: object "openshift-console"/"console-config" not registered
Oct 30 21:50:21.220990 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:21.220439    5510 configmap.go:193] Couldn't get configMap openshift-console/console-config: object "openshift-console"/"console-config" not registered
Oct 30 21:50:21.220990 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:21.220445    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/31a389bb-f36f-4e53-9654-5f4648dae38a-console-serving-cert\") pod \"console-748df94584-88cjt\" (UID: \"31a389bb-f36f-4e53-9654-5f4648dae38a\") " pod="openshift-console/console-748df94584-88cjt"
Oct 30 21:50:21.220990 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:21.220431    5510 configmap.go:193] Couldn't get configMap openshift-console/service-ca: object "openshift-console"/"service-ca" not registered
Oct 30 21:50:21.220990 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:21.220479    5510 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/31a389bb-f36f-4e53-9654-5f4648dae38a-console-config podName:31a389bb-f36f-4e53-9654-5f4648dae38a nodeName:}" failed. No retries permitted until 2026-10-30 21:50:22.220466746 +0000 UTC m=+2.781276746 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "console-config" (UniqueName: "kubernetes.io/configmap/31a389bb-f36f-4e53-9654-5f4648dae38a-console-config") pod "console-748df94584-88cjt" (UID: "31a389bb-f36f-4e53-9654-5f4648dae38a") : object "openshift-console"/"console-config" not registered
Oct 30 21:50:21.220990 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:21.220495    5510 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-console-config podName:42d54b8a-4f48-46a7-8a34-1f0f29cd1602 nodeName:}" failed. No retries permitted until 2026-10-30 21:50:22.220486835 +0000 UTC m=+2.781296822 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "console-config" (UniqueName: "kubernetes.io/configmap/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-console-config") pod "console-6d5866d6cd-b6dpl" (UID: "42d54b8a-4f48-46a7-8a34-1f0f29cd1602") : object "openshift-console"/"console-config" not registered
Oct 30 21:50:21.220990 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:21.220508    5510 secret.go:189] Couldn't get secret openshift-console/console-serving-cert: object "openshift-console"/"console-serving-cert" not registered
Oct 30 21:50:21.220990 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:21.220509    5510 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-service-ca podName:42d54b8a-4f48-46a7-8a34-1f0f29cd1602 nodeName:}" failed. No retries permitted until 2026-10-30 21:50:22.220504192 +0000 UTC m=+2.781314175 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca" (UniqueName: "kubernetes.io/configmap/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-service-ca") pod "console-6d5866d6cd-b6dpl" (UID: "42d54b8a-4f48-46a7-8a34-1f0f29cd1602") : object "openshift-console"/"service-ca" not registered
Oct 30 21:50:21.220990 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:21.220536    5510 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/31a389bb-f36f-4e53-9654-5f4648dae38a-console-serving-cert podName:31a389bb-f36f-4e53-9654-5f4648dae38a nodeName:}" failed. No retries permitted until 2026-10-30 21:50:22.220529968 +0000 UTC m=+2.781339956 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "console-serving-cert" (UniqueName: "kubernetes.io/secret/31a389bb-f36f-4e53-9654-5f4648dae38a-console-serving-cert") pod "console-748df94584-88cjt" (UID: "31a389bb-f36f-4e53-9654-5f4648dae38a") : object "openshift-console"/"console-serving-cert" not registered
Oct 30 21:50:21.321775 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:21.321694    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rb2wf\" (UniqueName: \"kubernetes.io/projected/31a389bb-f36f-4e53-9654-5f4648dae38a-kube-api-access-rb2wf\") pod \"console-748df94584-88cjt\" (UID: \"31a389bb-f36f-4e53-9654-5f4648dae38a\") " pod="openshift-console/console-748df94584-88cjt"
Oct 30 21:50:21.321917 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:21.321847    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5c4fj\" (UniqueName: \"kubernetes.io/projected/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-kube-api-access-5c4fj\") pod \"console-6d5866d6cd-b6dpl\" (UID: \"42d54b8a-4f48-46a7-8a34-1f0f29cd1602\") " pod="openshift-console/console-6d5866d6cd-b6dpl"
Oct 30 21:50:21.321917 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:21.321855    5510 projected.go:291] Couldn't get configMap openshift-console/kube-root-ca.crt: object "openshift-console"/"kube-root-ca.crt" not registered
Oct 30 21:50:21.321917 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:21.321878    5510 projected.go:291] Couldn't get configMap openshift-console/openshift-service-ca.crt: object "openshift-console"/"openshift-service-ca.crt" not registered
Oct 30 21:50:21.321917 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:21.321888    5510 projected.go:196] Error preparing data for projected volume kube-api-access-rb2wf for pod openshift-console/console-748df94584-88cjt: [object "openshift-console"/"kube-root-ca.crt" not registered, object "openshift-console"/"openshift-service-ca.crt" not registered]
Oct 30 21:50:21.322050 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:21.321955    5510 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/31a389bb-f36f-4e53-9654-5f4648dae38a-kube-api-access-rb2wf podName:31a389bb-f36f-4e53-9654-5f4648dae38a nodeName:}" failed. No retries permitted until 2026-10-30 21:50:22.321926161 +0000 UTC m=+2.882736148 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-rb2wf" (UniqueName: "kubernetes.io/projected/31a389bb-f36f-4e53-9654-5f4648dae38a-kube-api-access-rb2wf") pod "console-748df94584-88cjt" (UID: "31a389bb-f36f-4e53-9654-5f4648dae38a") : [object "openshift-console"/"kube-root-ca.crt" not registered, object "openshift-console"/"openshift-service-ca.crt" not registered]
Oct 30 21:50:21.322050 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:21.321975    5510 projected.go:291] Couldn't get configMap openshift-console/kube-root-ca.crt: object "openshift-console"/"kube-root-ca.crt" not registered
Oct 30 21:50:21.322050 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:21.321989    5510 projected.go:291] Couldn't get configMap openshift-console/openshift-service-ca.crt: object "openshift-console"/"openshift-service-ca.crt" not registered
Oct 30 21:50:21.322050 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:21.321997    5510 projected.go:196] Error preparing data for projected volume kube-api-access-5c4fj for pod openshift-console/console-6d5866d6cd-b6dpl: [object "openshift-console"/"kube-root-ca.crt" not registered, object "openshift-console"/"openshift-service-ca.crt" not registered]
Oct 30 21:50:21.322050 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:21.322035    5510 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-kube-api-access-5c4fj podName:42d54b8a-4f48-46a7-8a34-1f0f29cd1602 nodeName:}" failed. No retries permitted until 2026-10-30 21:50:22.322025139 +0000 UTC m=+2.882835121 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-5c4fj" (UniqueName: "kubernetes.io/projected/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-kube-api-access-5c4fj") pod "console-6d5866d6cd-b6dpl" (UID: "42d54b8a-4f48-46a7-8a34-1f0f29cd1602") : [object "openshift-console"/"kube-root-ca.crt" not registered, object "openshift-console"/"openshift-service-ca.crt" not registered]
Oct 30 21:50:21.446493 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:21.446466    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-w5ck5"
Oct 30 21:50:21.673018 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:21.672932    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-7c848fbc95-8jm99"
Oct 30 21:50:21.679299 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:21.679267    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-7c848fbc95-8jm99"
Oct 30 21:50:21.939275 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:21.939195    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-5mw6n"
Oct 30 21:50:22.230913 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:22.230832    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/31a389bb-f36f-4e53-9654-5f4648dae38a-console-oauth-config\") pod \"console-748df94584-88cjt\" (UID: \"31a389bb-f36f-4e53-9654-5f4648dae38a\") " pod="openshift-console/console-748df94584-88cjt"
Oct 30 21:50:22.230913 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:22.230872    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-console-config\") pod \"console-6d5866d6cd-b6dpl\" (UID: \"42d54b8a-4f48-46a7-8a34-1f0f29cd1602\") " pod="openshift-console/console-6d5866d6cd-b6dpl"
Oct 30 21:50:22.230913 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:22.230904    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/31a389bb-f36f-4e53-9654-5f4648dae38a-console-config\") pod \"console-748df94584-88cjt\" (UID: \"31a389bb-f36f-4e53-9654-5f4648dae38a\") " pod="openshift-console/console-748df94584-88cjt"
Oct 30 21:50:22.231150 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:22.230928    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-console-oauth-config\") pod \"console-6d5866d6cd-b6dpl\" (UID: \"42d54b8a-4f48-46a7-8a34-1f0f29cd1602\") " pod="openshift-console/console-6d5866d6cd-b6dpl"
Oct 30 21:50:22.231150 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:22.230943    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-service-ca\") pod \"console-6d5866d6cd-b6dpl\" (UID: \"42d54b8a-4f48-46a7-8a34-1f0f29cd1602\") " pod="openshift-console/console-6d5866d6cd-b6dpl"
Oct 30 21:50:22.231150 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:22.230952    5510 configmap.go:193] Couldn't get configMap openshift-console/console-config: object "openshift-console"/"console-config" not registered
Oct 30 21:50:22.231150 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:22.230974    5510 secret.go:189] Couldn't get secret openshift-console/console-oauth-config: object "openshift-console"/"console-oauth-config" not registered
Oct 30 21:50:22.231150 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:22.231000    5510 configmap.go:193] Couldn't get configMap openshift-console/console-config: object "openshift-console"/"console-config" not registered
Oct 30 21:50:22.231150 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:22.231015    5510 secret.go:189] Couldn't get secret openshift-console/console-oauth-config: object "openshift-console"/"console-oauth-config" not registered
Oct 30 21:50:22.231150 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:22.231016    5510 configmap.go:193] Couldn't get configMap openshift-console/service-ca: object "openshift-console"/"service-ca" not registered
Oct 30 21:50:22.231150 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:22.230961    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/31a389bb-f36f-4e53-9654-5f4648dae38a-console-serving-cert\") pod \"console-748df94584-88cjt\" (UID: \"31a389bb-f36f-4e53-9654-5f4648dae38a\") " pod="openshift-console/console-748df94584-88cjt"
Oct 30 21:50:22.231150 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:22.231058    5510 secret.go:189] Couldn't get secret openshift-console/console-serving-cert: object "openshift-console"/"console-serving-cert" not registered
Oct 30 21:50:22.231150 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:22.231025    5510 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-console-config podName:42d54b8a-4f48-46a7-8a34-1f0f29cd1602 nodeName:}" failed. No retries permitted until 2026-10-30 21:50:24.231004547 +0000 UTC m=+4.791814534 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "console-config" (UniqueName: "kubernetes.io/configmap/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-console-config") pod "console-6d5866d6cd-b6dpl" (UID: "42d54b8a-4f48-46a7-8a34-1f0f29cd1602") : object "openshift-console"/"console-config" not registered
Oct 30 21:50:22.231150 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:22.231082    5510 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/31a389bb-f36f-4e53-9654-5f4648dae38a-console-oauth-config podName:31a389bb-f36f-4e53-9654-5f4648dae38a nodeName:}" failed. No retries permitted until 2026-10-30 21:50:24.231073835 +0000 UTC m=+4.791883821 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "console-oauth-config" (UniqueName: "kubernetes.io/secret/31a389bb-f36f-4e53-9654-5f4648dae38a-console-oauth-config") pod "console-748df94584-88cjt" (UID: "31a389bb-f36f-4e53-9654-5f4648dae38a") : object "openshift-console"/"console-oauth-config" not registered
Oct 30 21:50:22.231150 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:22.231095    5510 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/31a389bb-f36f-4e53-9654-5f4648dae38a-console-config podName:31a389bb-f36f-4e53-9654-5f4648dae38a nodeName:}" failed. No retries permitted until 2026-10-30 21:50:24.231089202 +0000 UTC m=+4.791899186 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "console-config" (UniqueName: "kubernetes.io/configmap/31a389bb-f36f-4e53-9654-5f4648dae38a-console-config") pod "console-748df94584-88cjt" (UID: "31a389bb-f36f-4e53-9654-5f4648dae38a") : object "openshift-console"/"console-config" not registered
Oct 30 21:50:22.231150 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:22.231112    5510 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-console-oauth-config podName:42d54b8a-4f48-46a7-8a34-1f0f29cd1602 nodeName:}" failed. No retries permitted until 2026-10-30 21:50:24.231102994 +0000 UTC m=+4.791912991 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "console-oauth-config" (UniqueName: "kubernetes.io/secret/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-console-oauth-config") pod "console-6d5866d6cd-b6dpl" (UID: "42d54b8a-4f48-46a7-8a34-1f0f29cd1602") : object "openshift-console"/"console-oauth-config" not registered
Oct 30 21:50:22.231150 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:22.231131    5510 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-service-ca podName:42d54b8a-4f48-46a7-8a34-1f0f29cd1602 nodeName:}" failed. No retries permitted until 2026-10-30 21:50:24.231123387 +0000 UTC m=+4.791933376 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca" (UniqueName: "kubernetes.io/configmap/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-service-ca") pod "console-6d5866d6cd-b6dpl" (UID: "42d54b8a-4f48-46a7-8a34-1f0f29cd1602") : object "openshift-console"/"service-ca" not registered
Oct 30 21:50:22.231150 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:22.231151    5510 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/31a389bb-f36f-4e53-9654-5f4648dae38a-console-serving-cert podName:31a389bb-f36f-4e53-9654-5f4648dae38a nodeName:}" failed. No retries permitted until 2026-10-30 21:50:24.231143068 +0000 UTC m=+4.791953051 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "console-serving-cert" (UniqueName: "kubernetes.io/secret/31a389bb-f36f-4e53-9654-5f4648dae38a-console-serving-cert") pod "console-748df94584-88cjt" (UID: "31a389bb-f36f-4e53-9654-5f4648dae38a") : object "openshift-console"/"console-serving-cert" not registered
Oct 30 21:50:22.231701 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:22.231202    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/31a389bb-f36f-4e53-9654-5f4648dae38a-service-ca\") pod \"console-748df94584-88cjt\" (UID: \"31a389bb-f36f-4e53-9654-5f4648dae38a\") " pod="openshift-console/console-748df94584-88cjt"
Oct 30 21:50:22.231701 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:22.231227    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-trusted-ca-bundle\") pod \"console-6d5866d6cd-b6dpl\" (UID: \"42d54b8a-4f48-46a7-8a34-1f0f29cd1602\") " pod="openshift-console/console-6d5866d6cd-b6dpl"
Oct 30 21:50:22.231701 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:22.231250    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/31a389bb-f36f-4e53-9654-5f4648dae38a-trusted-ca-bundle\") pod \"console-748df94584-88cjt\" (UID: \"31a389bb-f36f-4e53-9654-5f4648dae38a\") " pod="openshift-console/console-748df94584-88cjt"
Oct 30 21:50:22.231701 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:22.231283    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/31a389bb-f36f-4e53-9654-5f4648dae38a-oauth-serving-cert\") pod \"console-748df94584-88cjt\" (UID: \"31a389bb-f36f-4e53-9654-5f4648dae38a\") " pod="openshift-console/console-748df94584-88cjt"
Oct 30 21:50:22.231701 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:22.231301    5510 configmap.go:193] Couldn't get configMap openshift-console/service-ca: object "openshift-console"/"service-ca" not registered
Oct 30 21:50:22.231701 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:22.231309    5510 configmap.go:193] Couldn't get configMap openshift-console/trusted-ca-bundle: object "openshift-console"/"trusted-ca-bundle" not registered
Oct 30 21:50:22.231701 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:22.231325    5510 configmap.go:193] Couldn't get configMap openshift-console/oauth-serving-cert: object "openshift-console"/"oauth-serving-cert" not registered
Oct 30 21:50:22.231701 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:22.231335    5510 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/31a389bb-f36f-4e53-9654-5f4648dae38a-service-ca podName:31a389bb-f36f-4e53-9654-5f4648dae38a nodeName:}" failed. No retries permitted until 2026-10-30 21:50:24.231326069 +0000 UTC m=+4.792136059 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca" (UniqueName: "kubernetes.io/configmap/31a389bb-f36f-4e53-9654-5f4648dae38a-service-ca") pod "console-748df94584-88cjt" (UID: "31a389bb-f36f-4e53-9654-5f4648dae38a") : object "openshift-console"/"service-ca" not registered
Oct 30 21:50:22.231701 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:22.231313    5510 configmap.go:193] Couldn't get configMap openshift-console/trusted-ca-bundle: object "openshift-console"/"trusted-ca-bundle" not registered
Oct 30 21:50:22.231701 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:22.231350    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-console-serving-cert\") pod \"console-6d5866d6cd-b6dpl\" (UID: \"42d54b8a-4f48-46a7-8a34-1f0f29cd1602\") " pod="openshift-console/console-6d5866d6cd-b6dpl"
Oct 30 21:50:22.231701 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:22.231356    5510 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/31a389bb-f36f-4e53-9654-5f4648dae38a-oauth-serving-cert podName:31a389bb-f36f-4e53-9654-5f4648dae38a nodeName:}" failed. No retries permitted until 2026-10-30 21:50:24.231345174 +0000 UTC m=+4.792155190 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "oauth-serving-cert" (UniqueName: "kubernetes.io/configmap/31a389bb-f36f-4e53-9654-5f4648dae38a-oauth-serving-cert") pod "console-748df94584-88cjt" (UID: "31a389bb-f36f-4e53-9654-5f4648dae38a") : object "openshift-console"/"oauth-serving-cert" not registered
Oct 30 21:50:22.231701 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:22.231381    5510 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-trusted-ca-bundle podName:42d54b8a-4f48-46a7-8a34-1f0f29cd1602 nodeName:}" failed. No retries permitted until 2026-10-30 21:50:24.231375535 +0000 UTC m=+4.792185522 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-trusted-ca-bundle") pod "console-6d5866d6cd-b6dpl" (UID: "42d54b8a-4f48-46a7-8a34-1f0f29cd1602") : object "openshift-console"/"trusted-ca-bundle" not registered
Oct 30 21:50:22.231701 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:22.231397    5510 secret.go:189] Couldn't get secret openshift-console/console-serving-cert: object "openshift-console"/"console-serving-cert" not registered
Oct 30 21:50:22.231701 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:22.231400    5510 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/31a389bb-f36f-4e53-9654-5f4648dae38a-trusted-ca-bundle podName:31a389bb-f36f-4e53-9654-5f4648dae38a nodeName:}" failed. No retries permitted until 2026-10-30 21:50:24.231391502 +0000 UTC m=+4.792201492 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/31a389bb-f36f-4e53-9654-5f4648dae38a-trusted-ca-bundle") pod "console-748df94584-88cjt" (UID: "31a389bb-f36f-4e53-9654-5f4648dae38a") : object "openshift-console"/"trusted-ca-bundle" not registered
Oct 30 21:50:22.231701 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:22.231432    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-oauth-serving-cert\") pod \"console-6d5866d6cd-b6dpl\" (UID: \"42d54b8a-4f48-46a7-8a34-1f0f29cd1602\") " pod="openshift-console/console-6d5866d6cd-b6dpl"
Oct 30 21:50:22.231701 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:22.231452    5510 configmap.go:193] Couldn't get configMap openshift-console/oauth-serving-cert: object "openshift-console"/"oauth-serving-cert" not registered
Oct 30 21:50:22.231701 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:22.231476    5510 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-oauth-serving-cert podName:42d54b8a-4f48-46a7-8a34-1f0f29cd1602 nodeName:}" failed. No retries permitted until 2026-10-30 21:50:24.23146558 +0000 UTC m=+4.792275567 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "oauth-serving-cert" (UniqueName: "kubernetes.io/configmap/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-oauth-serving-cert") pod "console-6d5866d6cd-b6dpl" (UID: "42d54b8a-4f48-46a7-8a34-1f0f29cd1602") : object "openshift-console"/"oauth-serving-cert" not registered
Oct 30 21:50:22.231701 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:22.231488    5510 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-console-serving-cert podName:42d54b8a-4f48-46a7-8a34-1f0f29cd1602 nodeName:}" failed. No retries permitted until 2026-10-30 21:50:24.2314823 +0000 UTC m=+4.792292283 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "console-serving-cert" (UniqueName: "kubernetes.io/secret/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-console-serving-cert") pod "console-6d5866d6cd-b6dpl" (UID: "42d54b8a-4f48-46a7-8a34-1f0f29cd1602") : object "openshift-console"/"console-serving-cert" not registered
Oct 30 21:50:22.332438 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:22.332403    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5c4fj\" (UniqueName: \"kubernetes.io/projected/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-kube-api-access-5c4fj\") pod \"console-6d5866d6cd-b6dpl\" (UID: \"42d54b8a-4f48-46a7-8a34-1f0f29cd1602\") " pod="openshift-console/console-6d5866d6cd-b6dpl"
Oct 30 21:50:22.332564 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:22.332461    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rb2wf\" (UniqueName: \"kubernetes.io/projected/31a389bb-f36f-4e53-9654-5f4648dae38a-kube-api-access-rb2wf\") pod \"console-748df94584-88cjt\" (UID: \"31a389bb-f36f-4e53-9654-5f4648dae38a\") " pod="openshift-console/console-748df94584-88cjt"
Oct 30 21:50:22.332615 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:22.332559    5510 projected.go:291] Couldn't get configMap openshift-console/kube-root-ca.crt: object "openshift-console"/"kube-root-ca.crt" not registered
Oct 30 21:50:22.332615 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:22.332579    5510 projected.go:291] Couldn't get configMap openshift-console/openshift-service-ca.crt: object "openshift-console"/"openshift-service-ca.crt" not registered
Oct 30 21:50:22.332615 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:22.332588    5510 projected.go:196] Error preparing data for projected volume kube-api-access-5c4fj for pod openshift-console/console-6d5866d6cd-b6dpl: [object "openshift-console"/"kube-root-ca.crt" not registered, object "openshift-console"/"openshift-service-ca.crt" not registered]
Oct 30 21:50:22.332615 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:22.332601    5510 projected.go:291] Couldn't get configMap openshift-console/kube-root-ca.crt: object "openshift-console"/"kube-root-ca.crt" not registered
Oct 30 21:50:22.332615 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:22.332615    5510 projected.go:291] Couldn't get configMap openshift-console/openshift-service-ca.crt: object "openshift-console"/"openshift-service-ca.crt" not registered
Oct 30 21:50:22.332773 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:22.332633    5510 projected.go:196] Error preparing data for projected volume kube-api-access-rb2wf for pod openshift-console/console-748df94584-88cjt: [object "openshift-console"/"kube-root-ca.crt" not registered, object "openshift-console"/"openshift-service-ca.crt" not registered]
Oct 30 21:50:22.332773 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:22.332643    5510 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-kube-api-access-5c4fj podName:42d54b8a-4f48-46a7-8a34-1f0f29cd1602 nodeName:}" failed. No retries permitted until 2026-10-30 21:50:24.332627971 +0000 UTC m=+4.893437957 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-5c4fj" (UniqueName: "kubernetes.io/projected/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-kube-api-access-5c4fj") pod "console-6d5866d6cd-b6dpl" (UID: "42d54b8a-4f48-46a7-8a34-1f0f29cd1602") : [object "openshift-console"/"kube-root-ca.crt" not registered, object "openshift-console"/"openshift-service-ca.crt" not registered]
Oct 30 21:50:22.332773 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:22.332692    5510 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/31a389bb-f36f-4e53-9654-5f4648dae38a-kube-api-access-rb2wf podName:31a389bb-f36f-4e53-9654-5f4648dae38a nodeName:}" failed. No retries permitted until 2026-10-30 21:50:24.33267935 +0000 UTC m=+4.893489334 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-rb2wf" (UniqueName: "kubernetes.io/projected/31a389bb-f36f-4e53-9654-5f4648dae38a-kube-api-access-rb2wf") pod "console-748df94584-88cjt" (UID: "31a389bb-f36f-4e53-9654-5f4648dae38a") : [object "openshift-console"/"kube-root-ca.crt" not registered, object "openshift-console"/"openshift-service-ca.crt" not registered]
Oct 30 21:50:22.381977 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:22.381951    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:22.715988 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:22.715944    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:23.425383 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:23.425357    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-wrxww"
Oct 30 21:50:24.257349 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:24.257309    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/31a389bb-f36f-4e53-9654-5f4648dae38a-console-config\") pod \"console-748df94584-88cjt\" (UID: \"31a389bb-f36f-4e53-9654-5f4648dae38a\") " pod="openshift-console/console-748df94584-88cjt"
Oct 30 21:50:24.257794 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:24.257361    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-console-oauth-config\") pod \"console-6d5866d6cd-b6dpl\" (UID: \"42d54b8a-4f48-46a7-8a34-1f0f29cd1602\") " pod="openshift-console/console-6d5866d6cd-b6dpl"
Oct 30 21:50:24.257794 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:24.257380    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-service-ca\") pod \"console-6d5866d6cd-b6dpl\" (UID: \"42d54b8a-4f48-46a7-8a34-1f0f29cd1602\") " pod="openshift-console/console-6d5866d6cd-b6dpl"
Oct 30 21:50:24.257794 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:24.257408    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/31a389bb-f36f-4e53-9654-5f4648dae38a-console-serving-cert\") pod \"console-748df94584-88cjt\" (UID: \"31a389bb-f36f-4e53-9654-5f4648dae38a\") " pod="openshift-console/console-748df94584-88cjt"
Oct 30 21:50:24.257794 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:24.257453    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/31a389bb-f36f-4e53-9654-5f4648dae38a-service-ca\") pod \"console-748df94584-88cjt\" (UID: \"31a389bb-f36f-4e53-9654-5f4648dae38a\") " pod="openshift-console/console-748df94584-88cjt"
Oct 30 21:50:24.257794 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:24.257472    5510 configmap.go:193] Couldn't get configMap openshift-console/console-config: object "openshift-console"/"console-config" not registered
Oct 30 21:50:24.257794 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:24.257518    5510 configmap.go:193] Couldn't get configMap openshift-console/service-ca: object "openshift-console"/"service-ca" not registered
Oct 30 21:50:24.257794 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:24.257522    5510 configmap.go:193] Couldn't get configMap openshift-console/trusted-ca-bundle: object "openshift-console"/"trusted-ca-bundle" not registered
Oct 30 21:50:24.257794 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:24.257532    5510 secret.go:189] Couldn't get secret openshift-console/console-serving-cert: object "openshift-console"/"console-serving-cert" not registered
Oct 30 21:50:24.257794 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:24.257483    5510 secret.go:189] Couldn't get secret openshift-console/console-oauth-config: object "openshift-console"/"console-oauth-config" not registered
Oct 30 21:50:24.257794 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:24.257576    5510 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/31a389bb-f36f-4e53-9654-5f4648dae38a-console-config podName:31a389bb-f36f-4e53-9654-5f4648dae38a nodeName:}" failed. No retries permitted until 2026-10-30 21:50:28.257554701 +0000 UTC m=+8.818364684 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "console-config" (UniqueName: "kubernetes.io/configmap/31a389bb-f36f-4e53-9654-5f4648dae38a-console-config") pod "console-748df94584-88cjt" (UID: "31a389bb-f36f-4e53-9654-5f4648dae38a") : object "openshift-console"/"console-config" not registered
Oct 30 21:50:24.257794 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:24.257476    5510 configmap.go:193] Couldn't get configMap openshift-console/service-ca: object "openshift-console"/"service-ca" not registered
Oct 30 21:50:24.257794 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:24.257599    5510 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/31a389bb-f36f-4e53-9654-5f4648dae38a-service-ca podName:31a389bb-f36f-4e53-9654-5f4648dae38a nodeName:}" failed. No retries permitted until 2026-10-30 21:50:28.257588793 +0000 UTC m=+8.818398781 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca" (UniqueName: "kubernetes.io/configmap/31a389bb-f36f-4e53-9654-5f4648dae38a-service-ca") pod "console-748df94584-88cjt" (UID: "31a389bb-f36f-4e53-9654-5f4648dae38a") : object "openshift-console"/"service-ca" not registered
Oct 30 21:50:24.257794 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:24.257483    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-trusted-ca-bundle\") pod \"console-6d5866d6cd-b6dpl\" (UID: \"42d54b8a-4f48-46a7-8a34-1f0f29cd1602\") " pod="openshift-console/console-6d5866d6cd-b6dpl"
Oct 30 21:50:24.257794 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:24.257618    5510 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-service-ca podName:42d54b8a-4f48-46a7-8a34-1f0f29cd1602 nodeName:}" failed. No retries permitted until 2026-10-30 21:50:28.257605923 +0000 UTC m=+8.818415907 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca" (UniqueName: "kubernetes.io/configmap/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-service-ca") pod "console-6d5866d6cd-b6dpl" (UID: "42d54b8a-4f48-46a7-8a34-1f0f29cd1602") : object "openshift-console"/"service-ca" not registered
Oct 30 21:50:24.257794 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:24.257636    5510 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-trusted-ca-bundle podName:42d54b8a-4f48-46a7-8a34-1f0f29cd1602 nodeName:}" failed. No retries permitted until 2026-10-30 21:50:28.257627917 +0000 UTC m=+8.818437909 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-trusted-ca-bundle") pod "console-6d5866d6cd-b6dpl" (UID: "42d54b8a-4f48-46a7-8a34-1f0f29cd1602") : object "openshift-console"/"trusted-ca-bundle" not registered
Oct 30 21:50:24.257794 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:24.257651    5510 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/31a389bb-f36f-4e53-9654-5f4648dae38a-console-serving-cert podName:31a389bb-f36f-4e53-9654-5f4648dae38a nodeName:}" failed. No retries permitted until 2026-10-30 21:50:28.257645763 +0000 UTC m=+8.818455750 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "console-serving-cert" (UniqueName: "kubernetes.io/secret/31a389bb-f36f-4e53-9654-5f4648dae38a-console-serving-cert") pod "console-748df94584-88cjt" (UID: "31a389bb-f36f-4e53-9654-5f4648dae38a") : object "openshift-console"/"console-serving-cert" not registered
Oct 30 21:50:24.257794 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:24.257680    5510 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-console-oauth-config podName:42d54b8a-4f48-46a7-8a34-1f0f29cd1602 nodeName:}" failed. No retries permitted until 2026-10-30 21:50:28.257674678 +0000 UTC m=+8.818484660 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "console-oauth-config" (UniqueName: "kubernetes.io/secret/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-console-oauth-config") pod "console-6d5866d6cd-b6dpl" (UID: "42d54b8a-4f48-46a7-8a34-1f0f29cd1602") : object "openshift-console"/"console-oauth-config" not registered
Oct 30 21:50:24.257794 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:24.257702    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/31a389bb-f36f-4e53-9654-5f4648dae38a-trusted-ca-bundle\") pod \"console-748df94584-88cjt\" (UID: \"31a389bb-f36f-4e53-9654-5f4648dae38a\") " pod="openshift-console/console-748df94584-88cjt"
Oct 30 21:50:24.257794 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:24.257724    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/31a389bb-f36f-4e53-9654-5f4648dae38a-oauth-serving-cert\") pod \"console-748df94584-88cjt\" (UID: \"31a389bb-f36f-4e53-9654-5f4648dae38a\") " pod="openshift-console/console-748df94584-88cjt"
Oct 30 21:50:24.257794 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:24.257760    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-console-serving-cert\") pod \"console-6d5866d6cd-b6dpl\" (UID: \"42d54b8a-4f48-46a7-8a34-1f0f29cd1602\") " pod="openshift-console/console-6d5866d6cd-b6dpl"
Oct 30 21:50:24.257794 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:24.257776    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-oauth-serving-cert\") pod \"console-6d5866d6cd-b6dpl\" (UID: \"42d54b8a-4f48-46a7-8a34-1f0f29cd1602\") " pod="openshift-console/console-6d5866d6cd-b6dpl"
Oct 30 21:50:24.257794 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:24.257805    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/31a389bb-f36f-4e53-9654-5f4648dae38a-console-oauth-config\") pod \"console-748df94584-88cjt\" (UID: \"31a389bb-f36f-4e53-9654-5f4648dae38a\") " pod="openshift-console/console-748df94584-88cjt"
Oct 30 21:50:24.258528 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:24.257828    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-console-config\") pod \"console-6d5866d6cd-b6dpl\" (UID: \"42d54b8a-4f48-46a7-8a34-1f0f29cd1602\") " pod="openshift-console/console-6d5866d6cd-b6dpl"
Oct 30 21:50:24.258528 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:24.257894    5510 configmap.go:193] Couldn't get configMap openshift-console/console-config: object "openshift-console"/"console-config" not registered
Oct 30 21:50:24.258528 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:24.257918    5510 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-console-config podName:42d54b8a-4f48-46a7-8a34-1f0f29cd1602 nodeName:}" failed. No retries permitted until 2026-10-30 21:50:28.25791022 +0000 UTC m=+8.818720203 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "console-config" (UniqueName: "kubernetes.io/configmap/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-console-config") pod "console-6d5866d6cd-b6dpl" (UID: "42d54b8a-4f48-46a7-8a34-1f0f29cd1602") : object "openshift-console"/"console-config" not registered
Oct 30 21:50:24.258528 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:24.257942    5510 configmap.go:193] Couldn't get configMap openshift-console/oauth-serving-cert: object "openshift-console"/"oauth-serving-cert" not registered
Oct 30 21:50:24.258528 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:24.257950    5510 secret.go:189] Couldn't get secret openshift-console/console-serving-cert: object "openshift-console"/"console-serving-cert" not registered
Oct 30 21:50:24.258528 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:24.257960    5510 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-oauth-serving-cert podName:42d54b8a-4f48-46a7-8a34-1f0f29cd1602 nodeName:}" failed. No retries permitted until 2026-10-30 21:50:28.257954433 +0000 UTC m=+8.818764417 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "oauth-serving-cert" (UniqueName: "kubernetes.io/configmap/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-oauth-serving-cert") pod "console-6d5866d6cd-b6dpl" (UID: "42d54b8a-4f48-46a7-8a34-1f0f29cd1602") : object "openshift-console"/"oauth-serving-cert" not registered
Oct 30 21:50:24.258528 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:24.257985    5510 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-console-serving-cert podName:42d54b8a-4f48-46a7-8a34-1f0f29cd1602 nodeName:}" failed. No retries permitted until 2026-10-30 21:50:28.257975349 +0000 UTC m=+8.818785337 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "console-serving-cert" (UniqueName: "kubernetes.io/secret/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-console-serving-cert") pod "console-6d5866d6cd-b6dpl" (UID: "42d54b8a-4f48-46a7-8a34-1f0f29cd1602") : object "openshift-console"/"console-serving-cert" not registered
Oct 30 21:50:24.258528 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:24.257996    5510 secret.go:189] Couldn't get secret openshift-console/console-oauth-config: object "openshift-console"/"console-oauth-config" not registered
Oct 30 21:50:24.258528 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:24.258016    5510 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/31a389bb-f36f-4e53-9654-5f4648dae38a-console-oauth-config podName:31a389bb-f36f-4e53-9654-5f4648dae38a nodeName:}" failed. No retries permitted until 2026-10-30 21:50:28.25801052 +0000 UTC m=+8.818820503 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "console-oauth-config" (UniqueName: "kubernetes.io/secret/31a389bb-f36f-4e53-9654-5f4648dae38a-console-oauth-config") pod "console-748df94584-88cjt" (UID: "31a389bb-f36f-4e53-9654-5f4648dae38a") : object "openshift-console"/"console-oauth-config" not registered
Oct 30 21:50:24.258528 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:24.258016    5510 configmap.go:193] Couldn't get configMap openshift-console/trusted-ca-bundle: object "openshift-console"/"trusted-ca-bundle" not registered
Oct 30 21:50:24.258528 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:24.258034    5510 configmap.go:193] Couldn't get configMap openshift-console/oauth-serving-cert: object "openshift-console"/"oauth-serving-cert" not registered
Oct 30 21:50:24.258528 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:24.258036    5510 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/31a389bb-f36f-4e53-9654-5f4648dae38a-trusted-ca-bundle podName:31a389bb-f36f-4e53-9654-5f4648dae38a nodeName:}" failed. No retries permitted until 2026-10-30 21:50:28.258030685 +0000 UTC m=+8.818840668 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/31a389bb-f36f-4e53-9654-5f4648dae38a-trusted-ca-bundle") pod "console-748df94584-88cjt" (UID: "31a389bb-f36f-4e53-9654-5f4648dae38a") : object "openshift-console"/"trusted-ca-bundle" not registered
Oct 30 21:50:24.258528 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:24.258064    5510 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/31a389bb-f36f-4e53-9654-5f4648dae38a-oauth-serving-cert podName:31a389bb-f36f-4e53-9654-5f4648dae38a nodeName:}" failed. No retries permitted until 2026-10-30 21:50:28.258057767 +0000 UTC m=+8.818867749 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "oauth-serving-cert" (UniqueName: "kubernetes.io/configmap/31a389bb-f36f-4e53-9654-5f4648dae38a-oauth-serving-cert") pod "console-748df94584-88cjt" (UID: "31a389bb-f36f-4e53-9654-5f4648dae38a") : object "openshift-console"/"oauth-serving-cert" not registered
Oct 30 21:50:24.286239 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:24.286206    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:24.358366 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:24.358323    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rb2wf\" (UniqueName: \"kubernetes.io/projected/31a389bb-f36f-4e53-9654-5f4648dae38a-kube-api-access-rb2wf\") pod \"console-748df94584-88cjt\" (UID: \"31a389bb-f36f-4e53-9654-5f4648dae38a\") " pod="openshift-console/console-748df94584-88cjt"
Oct 30 21:50:24.358522 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:24.358483    5510 projected.go:291] Couldn't get configMap openshift-console/kube-root-ca.crt: object "openshift-console"/"kube-root-ca.crt" not registered
Oct 30 21:50:24.358522 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:24.358506    5510 projected.go:291] Couldn't get configMap openshift-console/openshift-service-ca.crt: object "openshift-console"/"openshift-service-ca.crt" not registered
Oct 30 21:50:24.358522 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:24.358516    5510 projected.go:196] Error preparing data for projected volume kube-api-access-rb2wf for pod openshift-console/console-748df94584-88cjt: [object "openshift-console"/"kube-root-ca.crt" not registered, object "openshift-console"/"openshift-service-ca.crt" not registered]
Oct 30 21:50:24.358648 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:24.358573    5510 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/31a389bb-f36f-4e53-9654-5f4648dae38a-kube-api-access-rb2wf podName:31a389bb-f36f-4e53-9654-5f4648dae38a nodeName:}" failed. No retries permitted until 2026-10-30 21:50:28.358557026 +0000 UTC m=+8.919367012 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-rb2wf" (UniqueName: "kubernetes.io/projected/31a389bb-f36f-4e53-9654-5f4648dae38a-kube-api-access-rb2wf") pod "console-748df94584-88cjt" (UID: "31a389bb-f36f-4e53-9654-5f4648dae38a") : [object "openshift-console"/"kube-root-ca.crt" not registered, object "openshift-console"/"openshift-service-ca.crt" not registered]
Oct 30 21:50:24.358648 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:24.358628    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5c4fj\" (UniqueName: \"kubernetes.io/projected/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-kube-api-access-5c4fj\") pod \"console-6d5866d6cd-b6dpl\" (UID: \"42d54b8a-4f48-46a7-8a34-1f0f29cd1602\") " pod="openshift-console/console-6d5866d6cd-b6dpl"
Oct 30 21:50:24.358767 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:24.358728    5510 projected.go:291] Couldn't get configMap openshift-console/kube-root-ca.crt: object "openshift-console"/"kube-root-ca.crt" not registered
Oct 30 21:50:24.358767 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:24.358738    5510 projected.go:291] Couldn't get configMap openshift-console/openshift-service-ca.crt: object "openshift-console"/"openshift-service-ca.crt" not registered
Oct 30 21:50:24.358767 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:24.358745    5510 projected.go:196] Error preparing data for projected volume kube-api-access-5c4fj for pod openshift-console/console-6d5866d6cd-b6dpl: [object "openshift-console"/"kube-root-ca.crt" not registered, object "openshift-console"/"openshift-service-ca.crt" not registered]
Oct 30 21:50:24.358853 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:24.358773    5510 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-kube-api-access-5c4fj podName:42d54b8a-4f48-46a7-8a34-1f0f29cd1602 nodeName:}" failed. No retries permitted until 2026-10-30 21:50:28.35876627 +0000 UTC m=+8.919576258 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-5c4fj" (UniqueName: "kubernetes.io/projected/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-kube-api-access-5c4fj") pod "console-6d5866d6cd-b6dpl" (UID: "42d54b8a-4f48-46a7-8a34-1f0f29cd1602") : [object "openshift-console"/"kube-root-ca.crt" not registered, object "openshift-console"/"openshift-service-ca.crt" not registered]
Oct 30 21:50:24.511635 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:24.511555    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:25.311227 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:25.311186    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-jmbng"
Oct 30 21:50:26.298483 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:26.298453    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-wrxww"
Oct 30 21:50:26.298754 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:26.298736    5510 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 30 21:50:26.473019 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:26.472989    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-77f4f76594-bfnw9"
Oct 30 21:50:26.477484 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:26.477462    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-77f4f76594-bfnw9"
Oct 30 21:50:28.304805 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:28.304757    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-console-oauth-config\") pod \"console-6d5866d6cd-b6dpl\" (UID: \"42d54b8a-4f48-46a7-8a34-1f0f29cd1602\") " pod="openshift-console/console-6d5866d6cd-b6dpl"
Oct 30 21:50:28.304805 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:28.304807    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-service-ca\") pod \"console-6d5866d6cd-b6dpl\" (UID: \"42d54b8a-4f48-46a7-8a34-1f0f29cd1602\") " pod="openshift-console/console-6d5866d6cd-b6dpl"
Oct 30 21:50:28.305229 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:28.304837    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/31a389bb-f36f-4e53-9654-5f4648dae38a-console-serving-cert\") pod \"console-748df94584-88cjt\" (UID: \"31a389bb-f36f-4e53-9654-5f4648dae38a\") " pod="openshift-console/console-748df94584-88cjt"
Oct 30 21:50:28.305229 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:28.304883    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/31a389bb-f36f-4e53-9654-5f4648dae38a-service-ca\") pod \"console-748df94584-88cjt\" (UID: \"31a389bb-f36f-4e53-9654-5f4648dae38a\") " pod="openshift-console/console-748df94584-88cjt"
Oct 30 21:50:28.305229 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:28.304915    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-trusted-ca-bundle\") pod \"console-6d5866d6cd-b6dpl\" (UID: \"42d54b8a-4f48-46a7-8a34-1f0f29cd1602\") " pod="openshift-console/console-6d5866d6cd-b6dpl"
Oct 30 21:50:28.305229 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:28.304916    5510 secret.go:189] Couldn't get secret openshift-console/console-oauth-config: object "openshift-console"/"console-oauth-config" not registered
Oct 30 21:50:28.305229 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:28.304981    5510 configmap.go:193] Couldn't get configMap openshift-console/trusted-ca-bundle: object "openshift-console"/"trusted-ca-bundle" not registered
Oct 30 21:50:28.305229 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:28.304980    5510 secret.go:189] Couldn't get secret openshift-console/console-serving-cert: object "openshift-console"/"console-serving-cert" not registered
Oct 30 21:50:28.305229 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:28.304948    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/31a389bb-f36f-4e53-9654-5f4648dae38a-trusted-ca-bundle\") pod \"console-748df94584-88cjt\" (UID: \"31a389bb-f36f-4e53-9654-5f4648dae38a\") " pod="openshift-console/console-748df94584-88cjt"
Oct 30 21:50:28.305229 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:28.305000    5510 configmap.go:193] Couldn't get configMap openshift-console/trusted-ca-bundle: object "openshift-console"/"trusted-ca-bundle" not registered
Oct 30 21:50:28.305229 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:28.305038    5510 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-console-oauth-config podName:42d54b8a-4f48-46a7-8a34-1f0f29cd1602 nodeName:}" failed. No retries permitted until 2026-10-30 21:50:36.305019549 +0000 UTC m=+16.865829544 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "console-oauth-config" (UniqueName: "kubernetes.io/secret/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-console-oauth-config") pod "console-6d5866d6cd-b6dpl" (UID: "42d54b8a-4f48-46a7-8a34-1f0f29cd1602") : object "openshift-console"/"console-oauth-config" not registered
Oct 30 21:50:28.305229 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:28.305054    5510 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/31a389bb-f36f-4e53-9654-5f4648dae38a-trusted-ca-bundle podName:31a389bb-f36f-4e53-9654-5f4648dae38a nodeName:}" failed. No retries permitted until 2026-10-30 21:50:36.305048016 +0000 UTC m=+16.865857999 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/31a389bb-f36f-4e53-9654-5f4648dae38a-trusted-ca-bundle") pod "console-748df94584-88cjt" (UID: "31a389bb-f36f-4e53-9654-5f4648dae38a") : object "openshift-console"/"trusted-ca-bundle" not registered
Oct 30 21:50:28.305229 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:28.305079    5510 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-trusted-ca-bundle podName:42d54b8a-4f48-46a7-8a34-1f0f29cd1602 nodeName:}" failed. No retries permitted until 2026-10-30 21:50:36.305066994 +0000 UTC m=+16.865876992 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-trusted-ca-bundle") pod "console-6d5866d6cd-b6dpl" (UID: "42d54b8a-4f48-46a7-8a34-1f0f29cd1602") : object "openshift-console"/"trusted-ca-bundle" not registered
Oct 30 21:50:28.305229 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:28.305078    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/31a389bb-f36f-4e53-9654-5f4648dae38a-oauth-serving-cert\") pod \"console-748df94584-88cjt\" (UID: \"31a389bb-f36f-4e53-9654-5f4648dae38a\") " pod="openshift-console/console-748df94584-88cjt"
Oct 30 21:50:28.305229 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:28.304920    5510 configmap.go:193] Couldn't get configMap openshift-console/service-ca: object "openshift-console"/"service-ca" not registered
Oct 30 21:50:28.305229 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:28.305109    5510 configmap.go:193] Couldn't get configMap openshift-console/oauth-serving-cert: object "openshift-console"/"oauth-serving-cert" not registered
Oct 30 21:50:28.305229 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:28.305114    5510 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/31a389bb-f36f-4e53-9654-5f4648dae38a-console-serving-cert podName:31a389bb-f36f-4e53-9654-5f4648dae38a nodeName:}" failed. No retries permitted until 2026-10-30 21:50:36.305104276 +0000 UTC m=+16.865914263 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "console-serving-cert" (UniqueName: "kubernetes.io/secret/31a389bb-f36f-4e53-9654-5f4648dae38a-console-serving-cert") pod "console-748df94584-88cjt" (UID: "31a389bb-f36f-4e53-9654-5f4648dae38a") : object "openshift-console"/"console-serving-cert" not registered
Oct 30 21:50:28.305229 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:28.304955    5510 configmap.go:193] Couldn't get configMap openshift-console/service-ca: object "openshift-console"/"service-ca" not registered
Oct 30 21:50:28.305229 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:28.305130    5510 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/31a389bb-f36f-4e53-9654-5f4648dae38a-oauth-serving-cert podName:31a389bb-f36f-4e53-9654-5f4648dae38a nodeName:}" failed. No retries permitted until 2026-10-30 21:50:36.30512367 +0000 UTC m=+16.865933706 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "oauth-serving-cert" (UniqueName: "kubernetes.io/configmap/31a389bb-f36f-4e53-9654-5f4648dae38a-oauth-serving-cert") pod "console-748df94584-88cjt" (UID: "31a389bb-f36f-4e53-9654-5f4648dae38a") : object "openshift-console"/"oauth-serving-cert" not registered
Oct 30 21:50:28.305229 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:28.305146    5510 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-service-ca podName:42d54b8a-4f48-46a7-8a34-1f0f29cd1602 nodeName:}" failed. No retries permitted until 2026-10-30 21:50:36.305139261 +0000 UTC m=+16.865949243 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca" (UniqueName: "kubernetes.io/configmap/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-service-ca") pod "console-6d5866d6cd-b6dpl" (UID: "42d54b8a-4f48-46a7-8a34-1f0f29cd1602") : object "openshift-console"/"service-ca" not registered
Oct 30 21:50:28.305229 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:28.305165    5510 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/31a389bb-f36f-4e53-9654-5f4648dae38a-service-ca podName:31a389bb-f36f-4e53-9654-5f4648dae38a nodeName:}" failed. No retries permitted until 2026-10-30 21:50:36.305154112 +0000 UTC m=+16.865964100 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca" (UniqueName: "kubernetes.io/configmap/31a389bb-f36f-4e53-9654-5f4648dae38a-service-ca") pod "console-748df94584-88cjt" (UID: "31a389bb-f36f-4e53-9654-5f4648dae38a") : object "openshift-console"/"service-ca" not registered
Oct 30 21:50:28.305229 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:28.305207    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-console-serving-cert\") pod \"console-6d5866d6cd-b6dpl\" (UID: \"42d54b8a-4f48-46a7-8a34-1f0f29cd1602\") " pod="openshift-console/console-6d5866d6cd-b6dpl"
Oct 30 21:50:28.305826 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:28.305243    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-oauth-serving-cert\") pod \"console-6d5866d6cd-b6dpl\" (UID: \"42d54b8a-4f48-46a7-8a34-1f0f29cd1602\") " pod="openshift-console/console-6d5866d6cd-b6dpl"
Oct 30 21:50:28.305826 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:28.305250    5510 secret.go:189] Couldn't get secret openshift-console/console-serving-cert: object "openshift-console"/"console-serving-cert" not registered
Oct 30 21:50:28.305826 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:28.305271    5510 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-console-serving-cert podName:42d54b8a-4f48-46a7-8a34-1f0f29cd1602 nodeName:}" failed. No retries permitted until 2026-10-30 21:50:36.305265347 +0000 UTC m=+16.866075387 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "console-serving-cert" (UniqueName: "kubernetes.io/secret/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-console-serving-cert") pod "console-6d5866d6cd-b6dpl" (UID: "42d54b8a-4f48-46a7-8a34-1f0f29cd1602") : object "openshift-console"/"console-serving-cert" not registered
Oct 30 21:50:28.305826 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:28.305293    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/31a389bb-f36f-4e53-9654-5f4648dae38a-console-oauth-config\") pod \"console-748df94584-88cjt\" (UID: \"31a389bb-f36f-4e53-9654-5f4648dae38a\") " pod="openshift-console/console-748df94584-88cjt"
Oct 30 21:50:28.305826 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:28.305301    5510 configmap.go:193] Couldn't get configMap openshift-console/oauth-serving-cert: object "openshift-console"/"oauth-serving-cert" not registered
Oct 30 21:50:28.305826 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:28.305319    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-console-config\") pod \"console-6d5866d6cd-b6dpl\" (UID: \"42d54b8a-4f48-46a7-8a34-1f0f29cd1602\") " pod="openshift-console/console-6d5866d6cd-b6dpl"
Oct 30 21:50:28.305826 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:28.305338    5510 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-oauth-serving-cert podName:42d54b8a-4f48-46a7-8a34-1f0f29cd1602 nodeName:}" failed. No retries permitted until 2026-10-30 21:50:36.305325434 +0000 UTC m=+16.866135431 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "oauth-serving-cert" (UniqueName: "kubernetes.io/configmap/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-oauth-serving-cert") pod "console-6d5866d6cd-b6dpl" (UID: "42d54b8a-4f48-46a7-8a34-1f0f29cd1602") : object "openshift-console"/"oauth-serving-cert" not registered
Oct 30 21:50:28.305826 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:28.305343    5510 configmap.go:193] Couldn't get configMap openshift-console/console-config: object "openshift-console"/"console-config" not registered
Oct 30 21:50:28.305826 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:28.305363    5510 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-console-config podName:42d54b8a-4f48-46a7-8a34-1f0f29cd1602 nodeName:}" failed. No retries permitted until 2026-10-30 21:50:36.305357486 +0000 UTC m=+16.866167515 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "console-config" (UniqueName: "kubernetes.io/configmap/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-console-config") pod "console-6d5866d6cd-b6dpl" (UID: "42d54b8a-4f48-46a7-8a34-1f0f29cd1602") : object "openshift-console"/"console-config" not registered
Oct 30 21:50:28.305826 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:28.305388    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/31a389bb-f36f-4e53-9654-5f4648dae38a-console-config\") pod \"console-748df94584-88cjt\" (UID: \"31a389bb-f36f-4e53-9654-5f4648dae38a\") " pod="openshift-console/console-748df94584-88cjt"
Oct 30 21:50:28.305826 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:28.305413    5510 secret.go:189] Couldn't get secret openshift-console/console-oauth-config: object "openshift-console"/"console-oauth-config" not registered
Oct 30 21:50:28.305826 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:28.305433    5510 configmap.go:193] Couldn't get configMap openshift-console/console-config: object "openshift-console"/"console-config" not registered
Oct 30 21:50:28.305826 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:28.305447    5510 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/31a389bb-f36f-4e53-9654-5f4648dae38a-console-oauth-config podName:31a389bb-f36f-4e53-9654-5f4648dae38a nodeName:}" failed. No retries permitted until 2026-10-30 21:50:36.305436887 +0000 UTC m=+16.866246905 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "console-oauth-config" (UniqueName: "kubernetes.io/secret/31a389bb-f36f-4e53-9654-5f4648dae38a-console-oauth-config") pod "console-748df94584-88cjt" (UID: "31a389bb-f36f-4e53-9654-5f4648dae38a") : object "openshift-console"/"console-oauth-config" not registered
Oct 30 21:50:28.305826 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:28.305463    5510 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/31a389bb-f36f-4e53-9654-5f4648dae38a-console-config podName:31a389bb-f36f-4e53-9654-5f4648dae38a nodeName:}" failed. No retries permitted until 2026-10-30 21:50:36.305454698 +0000 UTC m=+16.866264683 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "console-config" (UniqueName: "kubernetes.io/configmap/31a389bb-f36f-4e53-9654-5f4648dae38a-console-config") pod "console-748df94584-88cjt" (UID: "31a389bb-f36f-4e53-9654-5f4648dae38a") : object "openshift-console"/"console-config" not registered
Oct 30 21:50:28.406250 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:28.406203    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5c4fj\" (UniqueName: \"kubernetes.io/projected/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-kube-api-access-5c4fj\") pod \"console-6d5866d6cd-b6dpl\" (UID: \"42d54b8a-4f48-46a7-8a34-1f0f29cd1602\") " pod="openshift-console/console-6d5866d6cd-b6dpl"
Oct 30 21:50:28.406402 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:28.406272    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rb2wf\" (UniqueName: \"kubernetes.io/projected/31a389bb-f36f-4e53-9654-5f4648dae38a-kube-api-access-rb2wf\") pod \"console-748df94584-88cjt\" (UID: \"31a389bb-f36f-4e53-9654-5f4648dae38a\") " pod="openshift-console/console-748df94584-88cjt"
Oct 30 21:50:28.406402 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:28.406361    5510 projected.go:291] Couldn't get configMap openshift-console/kube-root-ca.crt: object "openshift-console"/"kube-root-ca.crt" not registered
Oct 30 21:50:28.406402 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:28.406380    5510 projected.go:291] Couldn't get configMap openshift-console/openshift-service-ca.crt: object "openshift-console"/"openshift-service-ca.crt" not registered
Oct 30 21:50:28.406402 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:28.406391    5510 projected.go:196] Error preparing data for projected volume kube-api-access-5c4fj for pod openshift-console/console-6d5866d6cd-b6dpl: [object "openshift-console"/"kube-root-ca.crt" not registered, object "openshift-console"/"openshift-service-ca.crt" not registered]
Oct 30 21:50:28.406543 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:28.406361    5510 projected.go:291] Couldn't get configMap openshift-console/kube-root-ca.crt: object "openshift-console"/"kube-root-ca.crt" not registered
Oct 30 21:50:28.406543 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:28.406435    5510 projected.go:291] Couldn't get configMap openshift-console/openshift-service-ca.crt: object "openshift-console"/"openshift-service-ca.crt" not registered
Oct 30 21:50:28.406543 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:28.406446    5510 projected.go:196] Error preparing data for projected volume kube-api-access-rb2wf for pod openshift-console/console-748df94584-88cjt: [object "openshift-console"/"kube-root-ca.crt" not registered, object "openshift-console"/"openshift-service-ca.crt" not registered]
Oct 30 21:50:28.406543 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:28.406437    5510 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-kube-api-access-5c4fj podName:42d54b8a-4f48-46a7-8a34-1f0f29cd1602 nodeName:}" failed. No retries permitted until 2026-10-30 21:50:36.40642459 +0000 UTC m=+16.967234578 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-5c4fj" (UniqueName: "kubernetes.io/projected/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-kube-api-access-5c4fj") pod "console-6d5866d6cd-b6dpl" (UID: "42d54b8a-4f48-46a7-8a34-1f0f29cd1602") : [object "openshift-console"/"kube-root-ca.crt" not registered, object "openshift-console"/"openshift-service-ca.crt" not registered]
Oct 30 21:50:28.406543 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:28.406515    5510 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/31a389bb-f36f-4e53-9654-5f4648dae38a-kube-api-access-rb2wf podName:31a389bb-f36f-4e53-9654-5f4648dae38a nodeName:}" failed. No retries permitted until 2026-10-30 21:50:36.406499363 +0000 UTC m=+16.967309353 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-rb2wf" (UniqueName: "kubernetes.io/projected/31a389bb-f36f-4e53-9654-5f4648dae38a-kube-api-access-rb2wf") pod "console-748df94584-88cjt" (UID: "31a389bb-f36f-4e53-9654-5f4648dae38a") : [object "openshift-console"/"kube-root-ca.crt" not registered, object "openshift-console"/"openshift-service-ca.crt" not registered]
Oct 30 21:50:28.831888 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:28.831848    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:29.022602 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:29.022558    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:33.256574 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:33.256484    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:33.494336 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:33.494274    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:34.337456 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:34.337418    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:34.518555 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:34.518523    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:36.388077 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:36.388043    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-console-serving-cert\") pod \"console-6d5866d6cd-b6dpl\" (UID: \"42d54b8a-4f48-46a7-8a34-1f0f29cd1602\") " pod="openshift-console/console-6d5866d6cd-b6dpl"
Oct 30 21:50:36.388077 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:36.388077    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-oauth-serving-cert\") pod \"console-6d5866d6cd-b6dpl\" (UID: \"42d54b8a-4f48-46a7-8a34-1f0f29cd1602\") " pod="openshift-console/console-6d5866d6cd-b6dpl"
Oct 30 21:50:36.388532 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:36.388107    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/31a389bb-f36f-4e53-9654-5f4648dae38a-console-oauth-config\") pod \"console-748df94584-88cjt\" (UID: \"31a389bb-f36f-4e53-9654-5f4648dae38a\") " pod="openshift-console/console-748df94584-88cjt"
Oct 30 21:50:36.388532 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:36.388129    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-console-config\") pod \"console-6d5866d6cd-b6dpl\" (UID: \"42d54b8a-4f48-46a7-8a34-1f0f29cd1602\") " pod="openshift-console/console-6d5866d6cd-b6dpl"
Oct 30 21:50:36.388532 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:36.388165    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/31a389bb-f36f-4e53-9654-5f4648dae38a-console-config\") pod \"console-748df94584-88cjt\" (UID: \"31a389bb-f36f-4e53-9654-5f4648dae38a\") " pod="openshift-console/console-748df94584-88cjt"
Oct 30 21:50:36.388532 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:36.388178    5510 configmap.go:193] Couldn't get configMap openshift-console/oauth-serving-cert: object "openshift-console"/"oauth-serving-cert" not registered
Oct 30 21:50:36.388532 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:36.388196    5510 secret.go:189] Couldn't get secret openshift-console/console-serving-cert: object "openshift-console"/"console-serving-cert" not registered
Oct 30 21:50:36.388532 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:36.388221    5510 secret.go:189] Couldn't get secret openshift-console/console-oauth-config: object "openshift-console"/"console-oauth-config" not registered
Oct 30 21:50:36.388532 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:36.388190    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-console-oauth-config\") pod \"console-6d5866d6cd-b6dpl\" (UID: \"42d54b8a-4f48-46a7-8a34-1f0f29cd1602\") " pod="openshift-console/console-6d5866d6cd-b6dpl"
Oct 30 21:50:36.388532 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:36.388245    5510 secret.go:189] Couldn't get secret openshift-console/console-oauth-config: object "openshift-console"/"console-oauth-config" not registered
Oct 30 21:50:36.388532 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:36.388265    5510 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-oauth-serving-cert podName:42d54b8a-4f48-46a7-8a34-1f0f29cd1602 nodeName:}" failed. No retries permitted until 2026-10-30 21:50:52.3882444 +0000 UTC m=+32.949054403 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "oauth-serving-cert" (UniqueName: "kubernetes.io/configmap/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-oauth-serving-cert") pod "console-6d5866d6cd-b6dpl" (UID: "42d54b8a-4f48-46a7-8a34-1f0f29cd1602") : object "openshift-console"/"oauth-serving-cert" not registered
Oct 30 21:50:36.388532 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:36.388277    5510 configmap.go:193] Couldn't get configMap openshift-console/console-config: object "openshift-console"/"console-config" not registered
Oct 30 21:50:36.388532 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:36.388283    5510 configmap.go:193] Couldn't get configMap openshift-console/console-config: object "openshift-console"/"console-config" not registered
Oct 30 21:50:36.388532 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:36.388349    5510 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-console-serving-cert podName:42d54b8a-4f48-46a7-8a34-1f0f29cd1602 nodeName:}" failed. No retries permitted until 2026-10-30 21:50:52.388333453 +0000 UTC m=+32.949143437 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "console-serving-cert" (UniqueName: "kubernetes.io/secret/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-console-serving-cert") pod "console-6d5866d6cd-b6dpl" (UID: "42d54b8a-4f48-46a7-8a34-1f0f29cd1602") : object "openshift-console"/"console-serving-cert" not registered
Oct 30 21:50:36.388532 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:36.388366    5510 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-console-oauth-config podName:42d54b8a-4f48-46a7-8a34-1f0f29cd1602 nodeName:}" failed. No retries permitted until 2026-10-30 21:50:52.388359138 +0000 UTC m=+32.949169121 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "console-oauth-config" (UniqueName: "kubernetes.io/secret/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-console-oauth-config") pod "console-6d5866d6cd-b6dpl" (UID: "42d54b8a-4f48-46a7-8a34-1f0f29cd1602") : object "openshift-console"/"console-oauth-config" not registered
Oct 30 21:50:36.388532 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:36.388380    5510 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/31a389bb-f36f-4e53-9654-5f4648dae38a-console-oauth-config podName:31a389bb-f36f-4e53-9654-5f4648dae38a nodeName:}" failed. No retries permitted until 2026-10-30 21:50:52.38837225 +0000 UTC m=+32.949182232 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "console-oauth-config" (UniqueName: "kubernetes.io/secret/31a389bb-f36f-4e53-9654-5f4648dae38a-console-oauth-config") pod "console-748df94584-88cjt" (UID: "31a389bb-f36f-4e53-9654-5f4648dae38a") : object "openshift-console"/"console-oauth-config" not registered
Oct 30 21:50:36.388532 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:36.388395    5510 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/31a389bb-f36f-4e53-9654-5f4648dae38a-console-config podName:31a389bb-f36f-4e53-9654-5f4648dae38a nodeName:}" failed. No retries permitted until 2026-10-30 21:50:52.388388702 +0000 UTC m=+32.949198685 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "console-config" (UniqueName: "kubernetes.io/configmap/31a389bb-f36f-4e53-9654-5f4648dae38a-console-config") pod "console-748df94584-88cjt" (UID: "31a389bb-f36f-4e53-9654-5f4648dae38a") : object "openshift-console"/"console-config" not registered
Oct 30 21:50:36.388532 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:36.388414    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-service-ca\") pod \"console-6d5866d6cd-b6dpl\" (UID: \"42d54b8a-4f48-46a7-8a34-1f0f29cd1602\") " pod="openshift-console/console-6d5866d6cd-b6dpl"
Oct 30 21:50:36.388532 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:36.388432    5510 configmap.go:193] Couldn't get configMap openshift-console/service-ca: object "openshift-console"/"service-ca" not registered
Oct 30 21:50:36.388532 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:36.388438    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/31a389bb-f36f-4e53-9654-5f4648dae38a-console-serving-cert\") pod \"console-748df94584-88cjt\" (UID: \"31a389bb-f36f-4e53-9654-5f4648dae38a\") " pod="openshift-console/console-748df94584-88cjt"
Oct 30 21:50:36.388532 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:36.388463    5510 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-service-ca podName:42d54b8a-4f48-46a7-8a34-1f0f29cd1602 nodeName:}" failed. No retries permitted until 2026-10-30 21:50:52.388449202 +0000 UTC m=+32.949259211 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca" (UniqueName: "kubernetes.io/configmap/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-service-ca") pod "console-6d5866d6cd-b6dpl" (UID: "42d54b8a-4f48-46a7-8a34-1f0f29cd1602") : object "openshift-console"/"service-ca" not registered
Oct 30 21:50:36.388532 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:36.388504    5510 secret.go:189] Couldn't get secret openshift-console/console-serving-cert: object "openshift-console"/"console-serving-cert" not registered
Oct 30 21:50:36.388532 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:36.388504    5510 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-console-config podName:42d54b8a-4f48-46a7-8a34-1f0f29cd1602 nodeName:}" failed. No retries permitted until 2026-10-30 21:50:52.388486426 +0000 UTC m=+32.949296420 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "console-config" (UniqueName: "kubernetes.io/configmap/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-console-config") pod "console-6d5866d6cd-b6dpl" (UID: "42d54b8a-4f48-46a7-8a34-1f0f29cd1602") : object "openshift-console"/"console-config" not registered
Oct 30 21:50:36.389188 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:36.388549    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/31a389bb-f36f-4e53-9654-5f4648dae38a-service-ca\") pod \"console-748df94584-88cjt\" (UID: \"31a389bb-f36f-4e53-9654-5f4648dae38a\") " pod="openshift-console/console-748df94584-88cjt"
Oct 30 21:50:36.389188 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:36.388565    5510 configmap.go:193] Couldn't get configMap openshift-console/service-ca: object "openshift-console"/"service-ca" not registered
Oct 30 21:50:36.389188 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:36.388592    5510 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/31a389bb-f36f-4e53-9654-5f4648dae38a-console-serving-cert podName:31a389bb-f36f-4e53-9654-5f4648dae38a nodeName:}" failed. No retries permitted until 2026-10-30 21:50:52.388584861 +0000 UTC m=+32.949394847 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "console-serving-cert" (UniqueName: "kubernetes.io/secret/31a389bb-f36f-4e53-9654-5f4648dae38a-console-serving-cert") pod "console-748df94584-88cjt" (UID: "31a389bb-f36f-4e53-9654-5f4648dae38a") : object "openshift-console"/"console-serving-cert" not registered
Oct 30 21:50:36.389188 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:36.388591    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-trusted-ca-bundle\") pod \"console-6d5866d6cd-b6dpl\" (UID: \"42d54b8a-4f48-46a7-8a34-1f0f29cd1602\") " pod="openshift-console/console-6d5866d6cd-b6dpl"
Oct 30 21:50:36.389188 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:36.388612    5510 configmap.go:193] Couldn't get configMap openshift-console/trusted-ca-bundle: object "openshift-console"/"trusted-ca-bundle" not registered
Oct 30 21:50:36.389188 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:36.388626    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/31a389bb-f36f-4e53-9654-5f4648dae38a-trusted-ca-bundle\") pod \"console-748df94584-88cjt\" (UID: \"31a389bb-f36f-4e53-9654-5f4648dae38a\") " pod="openshift-console/console-748df94584-88cjt"
Oct 30 21:50:36.389188 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:36.388632    5510 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-trusted-ca-bundle podName:42d54b8a-4f48-46a7-8a34-1f0f29cd1602 nodeName:}" failed. No retries permitted until 2026-10-30 21:50:52.388625288 +0000 UTC m=+32.949435275 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-trusted-ca-bundle") pod "console-6d5866d6cd-b6dpl" (UID: "42d54b8a-4f48-46a7-8a34-1f0f29cd1602") : object "openshift-console"/"trusted-ca-bundle" not registered
Oct 30 21:50:36.389188 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:36.388671    5510 configmap.go:193] Couldn't get configMap openshift-console/trusted-ca-bundle: object "openshift-console"/"trusted-ca-bundle" not registered
Oct 30 21:50:36.389188 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:36.388675    5510 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/31a389bb-f36f-4e53-9654-5f4648dae38a-service-ca podName:31a389bb-f36f-4e53-9654-5f4648dae38a nodeName:}" failed. No retries permitted until 2026-10-30 21:50:52.388645664 +0000 UTC m=+32.949455659 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca" (UniqueName: "kubernetes.io/configmap/31a389bb-f36f-4e53-9654-5f4648dae38a-service-ca") pod "console-748df94584-88cjt" (UID: "31a389bb-f36f-4e53-9654-5f4648dae38a") : object "openshift-console"/"service-ca" not registered
Oct 30 21:50:36.389188 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:36.388696    5510 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/31a389bb-f36f-4e53-9654-5f4648dae38a-trusted-ca-bundle podName:31a389bb-f36f-4e53-9654-5f4648dae38a nodeName:}" failed. No retries permitted until 2026-10-30 21:50:52.38869071 +0000 UTC m=+32.949500693 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/31a389bb-f36f-4e53-9654-5f4648dae38a-trusted-ca-bundle") pod "console-748df94584-88cjt" (UID: "31a389bb-f36f-4e53-9654-5f4648dae38a") : object "openshift-console"/"trusted-ca-bundle" not registered
Oct 30 21:50:36.389188 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:36.388718    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/31a389bb-f36f-4e53-9654-5f4648dae38a-oauth-serving-cert\") pod \"console-748df94584-88cjt\" (UID: \"31a389bb-f36f-4e53-9654-5f4648dae38a\") " pod="openshift-console/console-748df94584-88cjt"
Oct 30 21:50:36.389188 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:36.388795    5510 configmap.go:193] Couldn't get configMap openshift-console/oauth-serving-cert: object "openshift-console"/"oauth-serving-cert" not registered
Oct 30 21:50:36.389188 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:36.388837    5510 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/31a389bb-f36f-4e53-9654-5f4648dae38a-oauth-serving-cert podName:31a389bb-f36f-4e53-9654-5f4648dae38a nodeName:}" failed. No retries permitted until 2026-10-30 21:50:52.388826446 +0000 UTC m=+32.949636448 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "oauth-serving-cert" (UniqueName: "kubernetes.io/configmap/31a389bb-f36f-4e53-9654-5f4648dae38a-oauth-serving-cert") pod "console-748df94584-88cjt" (UID: "31a389bb-f36f-4e53-9654-5f4648dae38a") : object "openshift-console"/"oauth-serving-cert" not registered
Oct 30 21:50:36.489448 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:36.489409    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5c4fj\" (UniqueName: \"kubernetes.io/projected/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-kube-api-access-5c4fj\") pod \"console-6d5866d6cd-b6dpl\" (UID: \"42d54b8a-4f48-46a7-8a34-1f0f29cd1602\") " pod="openshift-console/console-6d5866d6cd-b6dpl"
Oct 30 21:50:36.489620 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:36.489464    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rb2wf\" (UniqueName: \"kubernetes.io/projected/31a389bb-f36f-4e53-9654-5f4648dae38a-kube-api-access-rb2wf\") pod \"console-748df94584-88cjt\" (UID: \"31a389bb-f36f-4e53-9654-5f4648dae38a\") " pod="openshift-console/console-748df94584-88cjt"
Oct 30 21:50:36.489620 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:36.489578    5510 projected.go:291] Couldn't get configMap openshift-console/kube-root-ca.crt: object "openshift-console"/"kube-root-ca.crt" not registered
Oct 30 21:50:36.489620 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:36.489599    5510 projected.go:291] Couldn't get configMap openshift-console/openshift-service-ca.crt: object "openshift-console"/"openshift-service-ca.crt" not registered
Oct 30 21:50:36.489620 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:36.489611    5510 projected.go:196] Error preparing data for projected volume kube-api-access-5c4fj for pod openshift-console/console-6d5866d6cd-b6dpl: [object "openshift-console"/"kube-root-ca.crt" not registered, object "openshift-console"/"openshift-service-ca.crt" not registered]
Oct 30 21:50:36.489792 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:36.489579    5510 projected.go:291] Couldn't get configMap openshift-console/kube-root-ca.crt: object "openshift-console"/"kube-root-ca.crt" not registered
Oct 30 21:50:36.489792 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:36.489676    5510 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-kube-api-access-5c4fj podName:42d54b8a-4f48-46a7-8a34-1f0f29cd1602 nodeName:}" failed. No retries permitted until 2026-10-30 21:50:52.489643803 +0000 UTC m=+33.050453786 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-5c4fj" (UniqueName: "kubernetes.io/projected/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-kube-api-access-5c4fj") pod "console-6d5866d6cd-b6dpl" (UID: "42d54b8a-4f48-46a7-8a34-1f0f29cd1602") : [object "openshift-console"/"kube-root-ca.crt" not registered, object "openshift-console"/"openshift-service-ca.crt" not registered]
Oct 30 21:50:36.489792 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:36.489680    5510 projected.go:291] Couldn't get configMap openshift-console/openshift-service-ca.crt: object "openshift-console"/"openshift-service-ca.crt" not registered
Oct 30 21:50:36.489792 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:36.489693    5510 projected.go:196] Error preparing data for projected volume kube-api-access-rb2wf for pod openshift-console/console-748df94584-88cjt: [object "openshift-console"/"kube-root-ca.crt" not registered, object "openshift-console"/"openshift-service-ca.crt" not registered]
Oct 30 21:50:36.489792 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:36.489717    5510 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/31a389bb-f36f-4e53-9654-5f4648dae38a-kube-api-access-rb2wf podName:31a389bb-f36f-4e53-9654-5f4648dae38a nodeName:}" failed. No retries permitted until 2026-10-30 21:50:52.489709587 +0000 UTC m=+33.050519569 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-rb2wf" (UniqueName: "kubernetes.io/projected/31a389bb-f36f-4e53-9654-5f4648dae38a-kube-api-access-rb2wf") pod "console-748df94584-88cjt" (UID: "31a389bb-f36f-4e53-9654-5f4648dae38a") : [object "openshift-console"/"kube-root-ca.crt" not registered, object "openshift-console"/"openshift-service-ca.crt" not registered]
Oct 30 21:50:37.513155 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:37.513125    5510 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-b787f5d44-6nhxn"]
Oct 30 21:50:37.541040 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:37.541008    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-b787f5d44-6nhxn"
Oct 30 21:50:37.591038 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:37.591005    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Oct 30 21:50:37.591038 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:37.591013    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-jlg2v\""
Oct 30 21:50:37.637563 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:37.637537    5510 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Oct 30 21:50:37.637563 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:37.637535    5510 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Oct 30 21:50:37.637797 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:37.637565    5510 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Oct 30 21:50:37.637797 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:37.637625    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Oct 30 21:50:37.637797 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:37.637638    5510 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Oct 30 21:50:37.684542 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:37.684514    5510 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Oct 30 21:50:37.702936 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:37.702903    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/82940f4e-4f15-40d9-9ed2-3abc559e0864-console-serving-cert\") pod \"console-b787f5d44-6nhxn\" (UID: \"82940f4e-4f15-40d9-9ed2-3abc559e0864\") " pod="openshift-console/console-b787f5d44-6nhxn"
Oct 30 21:50:37.702936 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:37.702934    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l8qxp\" (UniqueName: \"kubernetes.io/projected/82940f4e-4f15-40d9-9ed2-3abc559e0864-kube-api-access-l8qxp\") pod \"console-b787f5d44-6nhxn\" (UID: \"82940f4e-4f15-40d9-9ed2-3abc559e0864\") " pod="openshift-console/console-b787f5d44-6nhxn"
Oct 30 21:50:37.703125 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:37.703054    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/82940f4e-4f15-40d9-9ed2-3abc559e0864-tmp\") pod \"console-b787f5d44-6nhxn\" (UID: \"82940f4e-4f15-40d9-9ed2-3abc559e0864\") " pod="openshift-console/console-b787f5d44-6nhxn"
Oct 30 21:50:37.703159 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:37.703150    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/82940f4e-4f15-40d9-9ed2-3abc559e0864-console-oauth-config\") pod \"console-b787f5d44-6nhxn\" (UID: \"82940f4e-4f15-40d9-9ed2-3abc559e0864\") " pod="openshift-console/console-b787f5d44-6nhxn"
Oct 30 21:50:37.703197 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:37.703177    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/82940f4e-4f15-40d9-9ed2-3abc559e0864-console-config\") pod \"console-b787f5d44-6nhxn\" (UID: \"82940f4e-4f15-40d9-9ed2-3abc559e0864\") " pod="openshift-console/console-b787f5d44-6nhxn"
Oct 30 21:50:37.703229 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:37.703214    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/82940f4e-4f15-40d9-9ed2-3abc559e0864-trusted-ca-bundle\") pod \"console-b787f5d44-6nhxn\" (UID: \"82940f4e-4f15-40d9-9ed2-3abc559e0864\") " pod="openshift-console/console-b787f5d44-6nhxn"
Oct 30 21:50:37.703262 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:37.703237    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/82940f4e-4f15-40d9-9ed2-3abc559e0864-oauth-serving-cert\") pod \"console-b787f5d44-6nhxn\" (UID: \"82940f4e-4f15-40d9-9ed2-3abc559e0864\") " pod="openshift-console/console-b787f5d44-6nhxn"
Oct 30 21:50:37.703320 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:37.703308    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/82940f4e-4f15-40d9-9ed2-3abc559e0864-service-ca\") pod \"console-b787f5d44-6nhxn\" (UID: \"82940f4e-4f15-40d9-9ed2-3abc559e0864\") " pod="openshift-console/console-b787f5d44-6nhxn"
Oct 30 21:50:37.732702 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:37.732649    5510 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Oct 30 21:50:37.733469 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:37.733450    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-b787f5d44-6nhxn"]
Oct 30 21:50:37.804347 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:37.804317    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/82940f4e-4f15-40d9-9ed2-3abc559e0864-oauth-serving-cert\") pod \"console-b787f5d44-6nhxn\" (UID: \"82940f4e-4f15-40d9-9ed2-3abc559e0864\") " pod="openshift-console/console-b787f5d44-6nhxn"
Oct 30 21:50:37.804421 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:37.804363    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/82940f4e-4f15-40d9-9ed2-3abc559e0864-service-ca\") pod \"console-b787f5d44-6nhxn\" (UID: \"82940f4e-4f15-40d9-9ed2-3abc559e0864\") " pod="openshift-console/console-b787f5d44-6nhxn"
Oct 30 21:50:37.804601 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:37.804581    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/82940f4e-4f15-40d9-9ed2-3abc559e0864-console-serving-cert\") pod \"console-b787f5d44-6nhxn\" (UID: \"82940f4e-4f15-40d9-9ed2-3abc559e0864\") " pod="openshift-console/console-b787f5d44-6nhxn"
Oct 30 21:50:37.804635 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:37.804611    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l8qxp\" (UniqueName: \"kubernetes.io/projected/82940f4e-4f15-40d9-9ed2-3abc559e0864-kube-api-access-l8qxp\") pod \"console-b787f5d44-6nhxn\" (UID: \"82940f4e-4f15-40d9-9ed2-3abc559e0864\") " pod="openshift-console/console-b787f5d44-6nhxn"
Oct 30 21:50:37.804716 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:37.804701    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/82940f4e-4f15-40d9-9ed2-3abc559e0864-tmp\") pod \"console-b787f5d44-6nhxn\" (UID: \"82940f4e-4f15-40d9-9ed2-3abc559e0864\") " pod="openshift-console/console-b787f5d44-6nhxn"
Oct 30 21:50:37.804892 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:37.804866    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/82940f4e-4f15-40d9-9ed2-3abc559e0864-console-oauth-config\") pod \"console-b787f5d44-6nhxn\" (UID: \"82940f4e-4f15-40d9-9ed2-3abc559e0864\") " pod="openshift-console/console-b787f5d44-6nhxn"
Oct 30 21:50:37.804993 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:37.804912    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/82940f4e-4f15-40d9-9ed2-3abc559e0864-console-config\") pod \"console-b787f5d44-6nhxn\" (UID: \"82940f4e-4f15-40d9-9ed2-3abc559e0864\") " pod="openshift-console/console-b787f5d44-6nhxn"
Oct 30 21:50:37.805044 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:37.805005    5510 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Oct 30 21:50:37.805109 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:37.805085    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/82940f4e-4f15-40d9-9ed2-3abc559e0864-service-ca\") pod \"console-b787f5d44-6nhxn\" (UID: \"82940f4e-4f15-40d9-9ed2-3abc559e0864\") " pod="openshift-console/console-b787f5d44-6nhxn"
Oct 30 21:50:37.805109 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:37.805090    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/82940f4e-4f15-40d9-9ed2-3abc559e0864-tmp\") pod \"console-b787f5d44-6nhxn\" (UID: \"82940f4e-4f15-40d9-9ed2-3abc559e0864\") " pod="openshift-console/console-b787f5d44-6nhxn"
Oct 30 21:50:37.805251 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:37.805138    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/82940f4e-4f15-40d9-9ed2-3abc559e0864-oauth-serving-cert\") pod \"console-b787f5d44-6nhxn\" (UID: \"82940f4e-4f15-40d9-9ed2-3abc559e0864\") " pod="openshift-console/console-b787f5d44-6nhxn"
Oct 30 21:50:37.805251 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:37.805146    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/82940f4e-4f15-40d9-9ed2-3abc559e0864-trusted-ca-bundle\") pod \"console-b787f5d44-6nhxn\" (UID: \"82940f4e-4f15-40d9-9ed2-3abc559e0864\") " pod="openshift-console/console-b787f5d44-6nhxn"
Oct 30 21:50:37.805546 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:37.805522    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/82940f4e-4f15-40d9-9ed2-3abc559e0864-console-config\") pod \"console-b787f5d44-6nhxn\" (UID: \"82940f4e-4f15-40d9-9ed2-3abc559e0864\") " pod="openshift-console/console-b787f5d44-6nhxn"
Oct 30 21:50:37.805922 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:37.805903    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/82940f4e-4f15-40d9-9ed2-3abc559e0864-trusted-ca-bundle\") pod \"console-b787f5d44-6nhxn\" (UID: \"82940f4e-4f15-40d9-9ed2-3abc559e0864\") " pod="openshift-console/console-b787f5d44-6nhxn"
Oct 30 21:50:37.807138 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:37.807118    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/82940f4e-4f15-40d9-9ed2-3abc559e0864-console-serving-cert\") pod \"console-b787f5d44-6nhxn\" (UID: \"82940f4e-4f15-40d9-9ed2-3abc559e0864\") " pod="openshift-console/console-b787f5d44-6nhxn"
Oct 30 21:50:37.807227 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:37.807192    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/82940f4e-4f15-40d9-9ed2-3abc559e0864-console-oauth-config\") pod \"console-b787f5d44-6nhxn\" (UID: \"82940f4e-4f15-40d9-9ed2-3abc559e0864\") " pod="openshift-console/console-b787f5d44-6nhxn"
Oct 30 21:50:37.863214 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:37.863179    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l8qxp\" (UniqueName: \"kubernetes.io/projected/82940f4e-4f15-40d9-9ed2-3abc559e0864-kube-api-access-l8qxp\") pod \"console-b787f5d44-6nhxn\" (UID: \"82940f4e-4f15-40d9-9ed2-3abc559e0864\") " pod="openshift-console/console-b787f5d44-6nhxn"
Oct 30 21:50:37.867091 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:37.867052    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-b787f5d44-6nhxn"
Oct 30 21:50:37.867333 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:37.867309    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:38.120525 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:38.120448    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:38.230860 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:38.230829    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-b787f5d44-6nhxn"]
Oct 30 21:50:38.234857 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:38.234825    5510 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod82940f4e_4f15_40d9_9ed2_3abc559e0864.slice/crio-09b6fa4290a3f26fa5b1c99005749fee54c5fbfb412f894d240e67b8fb190cb8 WatchSource:0}: Error finding container 09b6fa4290a3f26fa5b1c99005749fee54c5fbfb412f894d240e67b8fb190cb8: Status 404 returned error can't find the container with id 09b6fa4290a3f26fa5b1c99005749fee54c5fbfb412f894d240e67b8fb190cb8
Oct 30 21:50:38.420860 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:38.420787    5510 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6gjbf" containerName="extractor"
Oct 30 21:50:38.675917 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:38.675840    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-b787f5d44-6nhxn" event={"ID":"82940f4e-4f15-40d9-9ed2-3abc559e0864","Type":"ContainerStarted","Data":"b7f206f3a48e5ea441dfacb62ddc16914cdf191394248b984cb534809f5ee190"}
Oct 30 21:50:38.675917 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:38.675886    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-b787f5d44-6nhxn" event={"ID":"82940f4e-4f15-40d9-9ed2-3abc559e0864","Type":"ContainerStarted","Data":"09b6fa4290a3f26fa5b1c99005749fee54c5fbfb412f894d240e67b8fb190cb8"}
Oct 30 21:50:38.788764 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:38.788709    5510 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/console-b787f5d44-6nhxn" podStartSLOduration=1.788694982 podStartE2EDuration="1.788694982s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:50:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:50:38.788528129 +0000 UTC m=+19.349338133" watchObservedRunningTime="2026-10-30 21:50:38.788694982 +0000 UTC m=+19.349504983"
Oct 30 21:50:39.249434 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:39.249395    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:39.438274 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:39.438238    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:40.969472 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:40.969439    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-wrxww"
Oct 30 21:50:42.910408 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:42.910368    5510 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Oct 30 21:50:42.913120 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:42.913101    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:50:42.960245 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:42.960213    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c59dcc75-2e4e-4ece-ae39-2e6c5f693942-config-out\") pod \"alertmanager-main-0\" (UID: \"c59dcc75-2e4e-4ece-ae39-2e6c5f693942\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:50:42.960245 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:42.960245    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/c59dcc75-2e4e-4ece-ae39-2e6c5f693942-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"c59dcc75-2e4e-4ece-ae39-2e6c5f693942\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:50:42.960435 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:42.960264    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/c59dcc75-2e4e-4ece-ae39-2e6c5f693942-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"c59dcc75-2e4e-4ece-ae39-2e6c5f693942\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:50:42.960435 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:42.960385    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/c59dcc75-2e4e-4ece-ae39-2e6c5f693942-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"c59dcc75-2e4e-4ece-ae39-2e6c5f693942\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:50:42.960499 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:42.960466    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/c59dcc75-2e4e-4ece-ae39-2e6c5f693942-config-volume\") pod \"alertmanager-main-0\" (UID: \"c59dcc75-2e4e-4ece-ae39-2e6c5f693942\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:50:42.960499 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:42.960487    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/c59dcc75-2e4e-4ece-ae39-2e6c5f693942-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"c59dcc75-2e4e-4ece-ae39-2e6c5f693942\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:50:42.960555 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:42.960511    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c59dcc75-2e4e-4ece-ae39-2e6c5f693942-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"c59dcc75-2e4e-4ece-ae39-2e6c5f693942\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:50:42.960555 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:42.960528    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c59dcc75-2e4e-4ece-ae39-2e6c5f693942-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"c59dcc75-2e4e-4ece-ae39-2e6c5f693942\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:50:42.960615 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:42.960601    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c59dcc75-2e4e-4ece-ae39-2e6c5f693942-web-config\") pod \"alertmanager-main-0\" (UID: \"c59dcc75-2e4e-4ece-ae39-2e6c5f693942\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:50:42.960727 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:42.960714    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gpxw2\" (UniqueName: \"kubernetes.io/projected/c59dcc75-2e4e-4ece-ae39-2e6c5f693942-kube-api-access-gpxw2\") pod \"alertmanager-main-0\" (UID: \"c59dcc75-2e4e-4ece-ae39-2e6c5f693942\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:50:42.960767 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:42.960747    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c59dcc75-2e4e-4ece-ae39-2e6c5f693942-tls-assets\") pod \"alertmanager-main-0\" (UID: \"c59dcc75-2e4e-4ece-ae39-2e6c5f693942\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:50:42.960800 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:42.960772    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c59dcc75-2e4e-4ece-ae39-2e6c5f693942-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"c59dcc75-2e4e-4ece-ae39-2e6c5f693942\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:50:42.960836 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:42.960806    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c59dcc75-2e4e-4ece-ae39-2e6c5f693942-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"c59dcc75-2e4e-4ece-ae39-2e6c5f693942\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:50:42.972620 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:42.972578    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Oct 30 21:50:42.972787 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:42.972635    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Oct 30 21:50:43.018980 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:43.018957    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Oct 30 21:50:43.018980 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:43.018956    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Oct 30 21:50:43.019134 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:43.018957    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Oct 30 21:50:43.019134 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:43.018961    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Oct 30 21:50:43.019134 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:43.019018    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Oct 30 21:50:43.019296 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:43.019284    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-lxh58\""
Oct 30 21:50:43.019342 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:43.019307    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Oct 30 21:50:43.020308 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:43.020290    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Oct 30 21:50:43.061445 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:43.061421    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/c59dcc75-2e4e-4ece-ae39-2e6c5f693942-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"c59dcc75-2e4e-4ece-ae39-2e6c5f693942\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:50:43.061560 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:43.061476    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/c59dcc75-2e4e-4ece-ae39-2e6c5f693942-config-volume\") pod \"alertmanager-main-0\" (UID: \"c59dcc75-2e4e-4ece-ae39-2e6c5f693942\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:50:43.061620 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:43.061590    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/c59dcc75-2e4e-4ece-ae39-2e6c5f693942-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"c59dcc75-2e4e-4ece-ae39-2e6c5f693942\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:50:43.061702 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:43.061627    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c59dcc75-2e4e-4ece-ae39-2e6c5f693942-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"c59dcc75-2e4e-4ece-ae39-2e6c5f693942\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:50:43.061702 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:43.061649    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c59dcc75-2e4e-4ece-ae39-2e6c5f693942-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"c59dcc75-2e4e-4ece-ae39-2e6c5f693942\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:50:43.061702 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:43.061692    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c59dcc75-2e4e-4ece-ae39-2e6c5f693942-web-config\") pod \"alertmanager-main-0\" (UID: \"c59dcc75-2e4e-4ece-ae39-2e6c5f693942\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:50:43.061864 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:43.061735    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gpxw2\" (UniqueName: \"kubernetes.io/projected/c59dcc75-2e4e-4ece-ae39-2e6c5f693942-kube-api-access-gpxw2\") pod \"alertmanager-main-0\" (UID: \"c59dcc75-2e4e-4ece-ae39-2e6c5f693942\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:50:43.061864 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:43.061779    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c59dcc75-2e4e-4ece-ae39-2e6c5f693942-tls-assets\") pod \"alertmanager-main-0\" (UID: \"c59dcc75-2e4e-4ece-ae39-2e6c5f693942\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:50:43.061864 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:43.061823    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c59dcc75-2e4e-4ece-ae39-2e6c5f693942-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"c59dcc75-2e4e-4ece-ae39-2e6c5f693942\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:50:43.062007 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:43.061874    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c59dcc75-2e4e-4ece-ae39-2e6c5f693942-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"c59dcc75-2e4e-4ece-ae39-2e6c5f693942\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:50:43.062007 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:43.061930    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/c59dcc75-2e4e-4ece-ae39-2e6c5f693942-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"c59dcc75-2e4e-4ece-ae39-2e6c5f693942\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:50:43.062117 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:43.062003    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c59dcc75-2e4e-4ece-ae39-2e6c5f693942-config-out\") pod \"alertmanager-main-0\" (UID: \"c59dcc75-2e4e-4ece-ae39-2e6c5f693942\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:50:43.062117 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:43.062035    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/c59dcc75-2e4e-4ece-ae39-2e6c5f693942-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"c59dcc75-2e4e-4ece-ae39-2e6c5f693942\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:50:43.062117 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:43.062067    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/c59dcc75-2e4e-4ece-ae39-2e6c5f693942-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"c59dcc75-2e4e-4ece-ae39-2e6c5f693942\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:50:43.063102 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:43.063029    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c59dcc75-2e4e-4ece-ae39-2e6c5f693942-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"c59dcc75-2e4e-4ece-ae39-2e6c5f693942\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:50:43.064728 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:43.064678    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/c59dcc75-2e4e-4ece-ae39-2e6c5f693942-config-volume\") pod \"alertmanager-main-0\" (UID: \"c59dcc75-2e4e-4ece-ae39-2e6c5f693942\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:50:43.064865 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:43.064842    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/c59dcc75-2e4e-4ece-ae39-2e6c5f693942-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"c59dcc75-2e4e-4ece-ae39-2e6c5f693942\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:50:43.064939 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:43.064852    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c59dcc75-2e4e-4ece-ae39-2e6c5f693942-web-config\") pod \"alertmanager-main-0\" (UID: \"c59dcc75-2e4e-4ece-ae39-2e6c5f693942\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:50:43.064997 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:43.064944    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c59dcc75-2e4e-4ece-ae39-2e6c5f693942-tls-assets\") pod \"alertmanager-main-0\" (UID: \"c59dcc75-2e4e-4ece-ae39-2e6c5f693942\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:50:43.064997 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:43.064945    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c59dcc75-2e4e-4ece-ae39-2e6c5f693942-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"c59dcc75-2e4e-4ece-ae39-2e6c5f693942\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:50:43.065477 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:43.065452    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c59dcc75-2e4e-4ece-ae39-2e6c5f693942-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"c59dcc75-2e4e-4ece-ae39-2e6c5f693942\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:50:43.065863 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:43.065840    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/c59dcc75-2e4e-4ece-ae39-2e6c5f693942-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"c59dcc75-2e4e-4ece-ae39-2e6c5f693942\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:50:43.066228 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:43.066205    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c59dcc75-2e4e-4ece-ae39-2e6c5f693942-config-out\") pod \"alertmanager-main-0\" (UID: \"c59dcc75-2e4e-4ece-ae39-2e6c5f693942\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:50:43.066874 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:43.066858    5510 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Oct 30 21:50:43.066941 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:43.066873    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/c59dcc75-2e4e-4ece-ae39-2e6c5f693942-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"c59dcc75-2e4e-4ece-ae39-2e6c5f693942\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:50:43.072510 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:43.072486    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c59dcc75-2e4e-4ece-ae39-2e6c5f693942-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"c59dcc75-2e4e-4ece-ae39-2e6c5f693942\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:50:43.120601 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:43.120573    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gpxw2\" (UniqueName: \"kubernetes.io/projected/c59dcc75-2e4e-4ece-ae39-2e6c5f693942-kube-api-access-gpxw2\") pod \"alertmanager-main-0\" (UID: \"c59dcc75-2e4e-4ece-ae39-2e6c5f693942\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:50:43.229788 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:43.229703    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:50:43.652337 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:43.652305    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Oct 30 21:50:43.655709 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:50:43.655677    5510 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc59dcc75_2e4e_4ece_ae39_2e6c5f693942.slice/crio-a7fc82b3bb8664f4243ae91df36b1fe9832458cc3b99405461eb8876af96f74d WatchSource:0}: Error finding container a7fc82b3bb8664f4243ae91df36b1fe9832458cc3b99405461eb8876af96f74d: Status 404 returned error can't find the container with id a7fc82b3bb8664f4243ae91df36b1fe9832458cc3b99405461eb8876af96f74d
Oct 30 21:50:43.693279 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:43.693249    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c59dcc75-2e4e-4ece-ae39-2e6c5f693942","Type":"ContainerStarted","Data":"a7fc82b3bb8664f4243ae91df36b1fe9832458cc3b99405461eb8876af96f74d"}
Oct 30 21:50:44.698106 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:44.698064    5510 generic.go:356] "Generic (PLEG): container finished" podID="c59dcc75-2e4e-4ece-ae39-2e6c5f693942" containerID="240e4cbe17956be22789e77281fd3c93b4938063d93a45e7877a3756931ce598" exitCode=0
Oct 30 21:50:44.698575 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:44.698148    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c59dcc75-2e4e-4ece-ae39-2e6c5f693942","Type":"ContainerDied","Data":"240e4cbe17956be22789e77281fd3c93b4938063d93a45e7877a3756931ce598"}
Oct 30 21:50:44.699381 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:44.699364    5510 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 21:50:45.631111 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:45.631073    5510 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-console/console-6d5866d6cd-b6dpl" podUID="42d54b8a-4f48-46a7-8a34-1f0f29cd1602" containerName="console" containerID="cri-o://c8252358442af04b3522aa680915b0a81265fed2df35832d0ac243fc1ce490c0" gracePeriod=15
Oct 30 21:50:45.633210 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:45.633173    5510 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-console/console-748df94584-88cjt" podUID="31a389bb-f36f-4e53-9654-5f4648dae38a" containerName="console" containerID="cri-o://be8d014ac774c705b67487af5cf43bac9c0460dc8d8be9cde6f1501fd7fdd9b5" gracePeriod=15
Oct 30 21:50:45.937473 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:45.937447    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6d5866d6cd-b6dpl"
Oct 30 21:50:45.941152 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:45.941130    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-748df94584-88cjt"
Oct 30 21:50:45.991961 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:45.991926    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/31a389bb-f36f-4e53-9654-5f4648dae38a-tmp\" (UniqueName: \"kubernetes.io/empty-dir/31a389bb-f36f-4e53-9654-5f4648dae38a-tmp\") pod \"31a389bb-f36f-4e53-9654-5f4648dae38a\" (UID: \"31a389bb-f36f-4e53-9654-5f4648dae38a\") "
Oct 30 21:50:45.992143 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:45.991983    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/31a389bb-f36f-4e53-9654-5f4648dae38a-console-serving-cert\" (UniqueName: \"kubernetes.io/secret/31a389bb-f36f-4e53-9654-5f4648dae38a-console-serving-cert\") pod \"31a389bb-f36f-4e53-9654-5f4648dae38a\" (UID: \"31a389bb-f36f-4e53-9654-5f4648dae38a\") "
Oct 30 21:50:45.992143 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:45.992058    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/31a389bb-f36f-4e53-9654-5f4648dae38a-oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/31a389bb-f36f-4e53-9654-5f4648dae38a-oauth-serving-cert\") pod \"31a389bb-f36f-4e53-9654-5f4648dae38a\" (UID: \"31a389bb-f36f-4e53-9654-5f4648dae38a\") "
Oct 30 21:50:45.992143 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:45.992097    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-tmp\" (UniqueName: \"kubernetes.io/empty-dir/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-tmp\") pod \"42d54b8a-4f48-46a7-8a34-1f0f29cd1602\" (UID: \"42d54b8a-4f48-46a7-8a34-1f0f29cd1602\") "
Oct 30 21:50:45.992324 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:45.992179    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-console-serving-cert\" (UniqueName: \"kubernetes.io/secret/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-console-serving-cert\") pod \"42d54b8a-4f48-46a7-8a34-1f0f29cd1602\" (UID: \"42d54b8a-4f48-46a7-8a34-1f0f29cd1602\") "
Oct 30 21:50:45.992324 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:45.992217    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-kube-api-access-5c4fj\" (UniqueName: \"kubernetes.io/projected/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-kube-api-access-5c4fj\") pod \"42d54b8a-4f48-46a7-8a34-1f0f29cd1602\" (UID: \"42d54b8a-4f48-46a7-8a34-1f0f29cd1602\") "
Oct 30 21:50:45.992324 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:45.992293    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-oauth-serving-cert\") pod \"42d54b8a-4f48-46a7-8a34-1f0f29cd1602\" (UID: \"42d54b8a-4f48-46a7-8a34-1f0f29cd1602\") "
Oct 30 21:50:45.992481 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:45.992327    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/31a389bb-f36f-4e53-9654-5f4648dae38a-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/31a389bb-f36f-4e53-9654-5f4648dae38a-trusted-ca-bundle\") pod \"31a389bb-f36f-4e53-9654-5f4648dae38a\" (UID: \"31a389bb-f36f-4e53-9654-5f4648dae38a\") "
Oct 30 21:50:45.992481 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:45.992383    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/31a389bb-f36f-4e53-9654-5f4648dae38a-service-ca\" (UniqueName: \"kubernetes.io/configmap/31a389bb-f36f-4e53-9654-5f4648dae38a-service-ca\") pod \"31a389bb-f36f-4e53-9654-5f4648dae38a\" (UID: \"31a389bb-f36f-4e53-9654-5f4648dae38a\") "
Oct 30 21:50:45.992481 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:45.992442    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/31a389bb-f36f-4e53-9654-5f4648dae38a-console-oauth-config\" (UniqueName: \"kubernetes.io/secret/31a389bb-f36f-4e53-9654-5f4648dae38a-console-oauth-config\") pod \"31a389bb-f36f-4e53-9654-5f4648dae38a\" (UID: \"31a389bb-f36f-4e53-9654-5f4648dae38a\") "
Oct 30 21:50:45.992481 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:45.992479    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-console-config\" (UniqueName: \"kubernetes.io/configmap/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-console-config\") pod \"42d54b8a-4f48-46a7-8a34-1f0f29cd1602\" (UID: \"42d54b8a-4f48-46a7-8a34-1f0f29cd1602\") "
Oct 30 21:50:45.992708 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:45.992498    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/31a389bb-f36f-4e53-9654-5f4648dae38a-oauth-serving-cert" pod "31a389bb-f36f-4e53-9654-5f4648dae38a" (UID: "31a389bb-f36f-4e53-9654-5f4648dae38a"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:50:45.992708 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:45.992534    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-trusted-ca-bundle\") pod \"42d54b8a-4f48-46a7-8a34-1f0f29cd1602\" (UID: \"42d54b8a-4f48-46a7-8a34-1f0f29cd1602\") "
Oct 30 21:50:45.992708 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:45.992590    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-console-oauth-config\" (UniqueName: \"kubernetes.io/secret/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-console-oauth-config\") pod \"42d54b8a-4f48-46a7-8a34-1f0f29cd1602\" (UID: \"42d54b8a-4f48-46a7-8a34-1f0f29cd1602\") "
Oct 30 21:50:45.992873 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:45.992720    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/31a389bb-f36f-4e53-9654-5f4648dae38a-tmp" pod "31a389bb-f36f-4e53-9654-5f4648dae38a" (UID: "31a389bb-f36f-4e53-9654-5f4648dae38a"). InnerVolumeSpecName "tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:50:45.992873 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:45.992800    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/31a389bb-f36f-4e53-9654-5f4648dae38a-trusted-ca-bundle" pod "31a389bb-f36f-4e53-9654-5f4648dae38a" (UID: "31a389bb-f36f-4e53-9654-5f4648dae38a"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:50:45.992979 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:45.992922    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-oauth-serving-cert" pod "42d54b8a-4f48-46a7-8a34-1f0f29cd1602" (UID: "42d54b8a-4f48-46a7-8a34-1f0f29cd1602"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:50:45.992979 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:45.992806    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/31a389bb-f36f-4e53-9654-5f4648dae38a-console-config\" (UniqueName: \"kubernetes.io/configmap/31a389bb-f36f-4e53-9654-5f4648dae38a-console-config\") pod \"31a389bb-f36f-4e53-9654-5f4648dae38a\" (UID: \"31a389bb-f36f-4e53-9654-5f4648dae38a\") "
Oct 30 21:50:45.992979 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:45.992964    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-service-ca\" (UniqueName: \"kubernetes.io/configmap/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-service-ca\") pod \"42d54b8a-4f48-46a7-8a34-1f0f29cd1602\" (UID: \"42d54b8a-4f48-46a7-8a34-1f0f29cd1602\") "
Oct 30 21:50:45.993137 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:45.993011    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/31a389bb-f36f-4e53-9654-5f4648dae38a-kube-api-access-rb2wf\" (UniqueName: \"kubernetes.io/projected/31a389bb-f36f-4e53-9654-5f4648dae38a-kube-api-access-rb2wf\") pod \"31a389bb-f36f-4e53-9654-5f4648dae38a\" (UID: \"31a389bb-f36f-4e53-9654-5f4648dae38a\") "
Oct 30 21:50:45.993137 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:45.993017    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-tmp" pod "42d54b8a-4f48-46a7-8a34-1f0f29cd1602" (UID: "42d54b8a-4f48-46a7-8a34-1f0f29cd1602"). InnerVolumeSpecName "tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:50:45.993349 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:45.993315    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-service-ca" pod "42d54b8a-4f48-46a7-8a34-1f0f29cd1602" (UID: "42d54b8a-4f48-46a7-8a34-1f0f29cd1602"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:50:45.993505 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:45.993484    5510 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-oauth-serving-cert\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:50:45.993570 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:45.993514    5510 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/31a389bb-f36f-4e53-9654-5f4648dae38a-trusted-ca-bundle\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:50:45.993570 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:45.993530    5510 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-service-ca\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:50:45.993570 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:45.993544    5510 reconciler_common.go:299] "Volume detached for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/31a389bb-f36f-4e53-9654-5f4648dae38a-tmp\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:50:45.993570 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:45.993559    5510 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/31a389bb-f36f-4e53-9654-5f4648dae38a-oauth-serving-cert\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:50:45.993793 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:45.993574    5510 reconciler_common.go:299] "Volume detached for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-tmp\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:50:45.993793 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:45.993672    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-trusted-ca-bundle" pod "42d54b8a-4f48-46a7-8a34-1f0f29cd1602" (UID: "42d54b8a-4f48-46a7-8a34-1f0f29cd1602"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:50:45.993793 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:45.993711    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-console-config" pod "42d54b8a-4f48-46a7-8a34-1f0f29cd1602" (UID: "42d54b8a-4f48-46a7-8a34-1f0f29cd1602"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:50:45.994250 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:45.994225    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/31a389bb-f36f-4e53-9654-5f4648dae38a-service-ca" pod "31a389bb-f36f-4e53-9654-5f4648dae38a" (UID: "31a389bb-f36f-4e53-9654-5f4648dae38a"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:50:45.994548 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:45.994515    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/31a389bb-f36f-4e53-9654-5f4648dae38a-console-config" pod "31a389bb-f36f-4e53-9654-5f4648dae38a" (UID: "31a389bb-f36f-4e53-9654-5f4648dae38a"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:50:45.996389 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:45.996362    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-console-serving-cert" pod "42d54b8a-4f48-46a7-8a34-1f0f29cd1602" (UID: "42d54b8a-4f48-46a7-8a34-1f0f29cd1602"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:50:45.996456 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:45.996386    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/31a389bb-f36f-4e53-9654-5f4648dae38a-kube-api-access-rb2wf" pod "31a389bb-f36f-4e53-9654-5f4648dae38a" (UID: "31a389bb-f36f-4e53-9654-5f4648dae38a"). InnerVolumeSpecName "kube-api-access-rb2wf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:50:45.996566 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:45.996521    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-console-oauth-config" pod "42d54b8a-4f48-46a7-8a34-1f0f29cd1602" (UID: "42d54b8a-4f48-46a7-8a34-1f0f29cd1602"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:50:45.996642 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:45.996617    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-kube-api-access-5c4fj" pod "42d54b8a-4f48-46a7-8a34-1f0f29cd1602" (UID: "42d54b8a-4f48-46a7-8a34-1f0f29cd1602"). InnerVolumeSpecName "kube-api-access-5c4fj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:50:45.996785 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:45.996698    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/31a389bb-f36f-4e53-9654-5f4648dae38a-console-oauth-config" pod "31a389bb-f36f-4e53-9654-5f4648dae38a" (UID: "31a389bb-f36f-4e53-9654-5f4648dae38a"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:50:45.997064 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:45.997043    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/31a389bb-f36f-4e53-9654-5f4648dae38a-console-serving-cert" pod "31a389bb-f36f-4e53-9654-5f4648dae38a" (UID: "31a389bb-f36f-4e53-9654-5f4648dae38a"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:50:46.094076 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:46.094033    5510 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-console-serving-cert\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:50:46.094076 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:46.094074    5510 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5c4fj\" (UniqueName: \"kubernetes.io/projected/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-kube-api-access-5c4fj\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:50:46.094328 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:46.094090    5510 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/31a389bb-f36f-4e53-9654-5f4648dae38a-service-ca\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:50:46.094328 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:46.094105    5510 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/31a389bb-f36f-4e53-9654-5f4648dae38a-console-oauth-config\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:50:46.094328 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:46.094122    5510 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-console-config\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:50:46.094328 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:46.094137    5510 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-trusted-ca-bundle\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:50:46.094328 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:46.094151    5510 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/42d54b8a-4f48-46a7-8a34-1f0f29cd1602-console-oauth-config\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:50:46.094328 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:46.094164    5510 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/31a389bb-f36f-4e53-9654-5f4648dae38a-console-config\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:50:46.094328 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:46.094175    5510 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rb2wf\" (UniqueName: \"kubernetes.io/projected/31a389bb-f36f-4e53-9654-5f4648dae38a-kube-api-access-rb2wf\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:50:46.094328 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:46.094185    5510 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/31a389bb-f36f-4e53-9654-5f4648dae38a-console-serving-cert\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:50:46.706958 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:46.706928    5510 generic.go:356] "Generic (PLEG): container finished" podID="42d54b8a-4f48-46a7-8a34-1f0f29cd1602" containerID="c8252358442af04b3522aa680915b0a81265fed2df35832d0ac243fc1ce490c0" exitCode=0
Oct 30 21:50:46.707102 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:46.706978    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6d5866d6cd-b6dpl"
Oct 30 21:50:46.707102 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:46.707012    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6d5866d6cd-b6dpl" event={"ID":"42d54b8a-4f48-46a7-8a34-1f0f29cd1602","Type":"ContainerDied","Data":"c8252358442af04b3522aa680915b0a81265fed2df35832d0ac243fc1ce490c0"}
Oct 30 21:50:46.707102 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:46.707061    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6d5866d6cd-b6dpl" event={"ID":"42d54b8a-4f48-46a7-8a34-1f0f29cd1602","Type":"ContainerDied","Data":"fbffb78323609cf8dd21b10efbe35374ed8184fc5b3e616fe9f4b87fd69b0936"}
Oct 30 21:50:46.707102 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:46.707086    5510 scope.go:122] "RemoveContainer" containerID="c8252358442af04b3522aa680915b0a81265fed2df35832d0ac243fc1ce490c0"
Oct 30 21:50:46.708498 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:46.708476    5510 generic.go:356] "Generic (PLEG): container finished" podID="31a389bb-f36f-4e53-9654-5f4648dae38a" containerID="be8d014ac774c705b67487af5cf43bac9c0460dc8d8be9cde6f1501fd7fdd9b5" exitCode=0
Oct 30 21:50:46.708603 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:46.708518    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-748df94584-88cjt"
Oct 30 21:50:46.708603 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:46.708548    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-748df94584-88cjt" event={"ID":"31a389bb-f36f-4e53-9654-5f4648dae38a","Type":"ContainerDied","Data":"be8d014ac774c705b67487af5cf43bac9c0460dc8d8be9cde6f1501fd7fdd9b5"}
Oct 30 21:50:46.708603 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:46.708582    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-748df94584-88cjt" event={"ID":"31a389bb-f36f-4e53-9654-5f4648dae38a","Type":"ContainerDied","Data":"d188c9559e229f137adf7ecb5417f5f9310e2d711a052c0aece44396a292cae9"}
Oct 30 21:50:46.720842 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:46.720815    5510 scope.go:122] "RemoveContainer" containerID="c8252358442af04b3522aa680915b0a81265fed2df35832d0ac243fc1ce490c0"
Oct 30 21:50:46.721184 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:46.721160    5510 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c8252358442af04b3522aa680915b0a81265fed2df35832d0ac243fc1ce490c0\": container with ID starting with c8252358442af04b3522aa680915b0a81265fed2df35832d0ac243fc1ce490c0 not found: ID does not exist" containerID="c8252358442af04b3522aa680915b0a81265fed2df35832d0ac243fc1ce490c0"
Oct 30 21:50:46.721251 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:46.721197    5510 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c8252358442af04b3522aa680915b0a81265fed2df35832d0ac243fc1ce490c0"} err="failed to get container status \"c8252358442af04b3522aa680915b0a81265fed2df35832d0ac243fc1ce490c0\": rpc error: code = NotFound desc = could not find container \"c8252358442af04b3522aa680915b0a81265fed2df35832d0ac243fc1ce490c0\": container with ID starting with c8252358442af04b3522aa680915b0a81265fed2df35832d0ac243fc1ce490c0 not found: ID does not exist"
Oct 30 21:50:46.721251 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:46.721241    5510 scope.go:122] "RemoveContainer" containerID="be8d014ac774c705b67487af5cf43bac9c0460dc8d8be9cde6f1501fd7fdd9b5"
Oct 30 21:50:46.731405 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:46.731382    5510 scope.go:122] "RemoveContainer" containerID="be8d014ac774c705b67487af5cf43bac9c0460dc8d8be9cde6f1501fd7fdd9b5"
Oct 30 21:50:46.731748 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:50:46.731728    5510 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"be8d014ac774c705b67487af5cf43bac9c0460dc8d8be9cde6f1501fd7fdd9b5\": container with ID starting with be8d014ac774c705b67487af5cf43bac9c0460dc8d8be9cde6f1501fd7fdd9b5 not found: ID does not exist" containerID="be8d014ac774c705b67487af5cf43bac9c0460dc8d8be9cde6f1501fd7fdd9b5"
Oct 30 21:50:46.731813 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:46.731755    5510 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"be8d014ac774c705b67487af5cf43bac9c0460dc8d8be9cde6f1501fd7fdd9b5"} err="failed to get container status \"be8d014ac774c705b67487af5cf43bac9c0460dc8d8be9cde6f1501fd7fdd9b5\": rpc error: code = NotFound desc = could not find container \"be8d014ac774c705b67487af5cf43bac9c0460dc8d8be9cde6f1501fd7fdd9b5\": container with ID starting with be8d014ac774c705b67487af5cf43bac9c0460dc8d8be9cde6f1501fd7fdd9b5 not found: ID does not exist"
Oct 30 21:50:46.872722 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:46.872699    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6d5866d6cd-b6dpl"]
Oct 30 21:50:46.877650 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:46.877631    5510 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6d5866d6cd-b6dpl"]
Oct 30 21:50:46.981327 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:46.981303    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-748df94584-88cjt"]
Oct 30 21:50:46.988568 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:46.988546    5510 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-748df94584-88cjt"]
Oct 30 21:50:47.551069 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:47.551040    5510 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="31a389bb-f36f-4e53-9654-5f4648dae38a" path="/var/lib/kubelet/pods/31a389bb-f36f-4e53-9654-5f4648dae38a/volumes"
Oct 30 21:50:47.551450 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:47.551438    5510 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="42d54b8a-4f48-46a7-8a34-1f0f29cd1602" path="/var/lib/kubelet/pods/42d54b8a-4f48-46a7-8a34-1f0f29cd1602/volumes"
Oct 30 21:50:47.717901 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:47.717857    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c59dcc75-2e4e-4ece-ae39-2e6c5f693942","Type":"ContainerStarted","Data":"cf313c6bf2e1feab995098af537363902b057264e16315ae06b0f04332c7a410"}
Oct 30 21:50:47.718042 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:47.717944    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c59dcc75-2e4e-4ece-ae39-2e6c5f693942","Type":"ContainerStarted","Data":"0bbcf45bb51888f44e7b63007b7c3213cabf916e09d1b9884bf6945216cbb7fe"}
Oct 30 21:50:47.718042 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:47.717981    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c59dcc75-2e4e-4ece-ae39-2e6c5f693942","Type":"ContainerStarted","Data":"19faf926f8eb0d0823b66855380ee91faf1fd7382b18330337d1ad779902443b"}
Oct 30 21:50:47.718042 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:47.718019    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c59dcc75-2e4e-4ece-ae39-2e6c5f693942","Type":"ContainerStarted","Data":"4b95ba2e12fe6639a0987fc7170c61c65ef64be8ae2dbb525a1c99e204082698"}
Oct 30 21:50:47.718172 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:47.718061    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c59dcc75-2e4e-4ece-ae39-2e6c5f693942","Type":"ContainerStarted","Data":"0d20a980311079de97c58cbb0cb804e32fa7e58302f3e721ffb1c0ec91cd441d"}
Oct 30 21:50:47.718172 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:47.718098    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c59dcc75-2e4e-4ece-ae39-2e6c5f693942","Type":"ContainerStarted","Data":"aa863c21ec0b0ce6f3f859ba3742ce10ac7f86670e4e4c962c6c44a85db7f66b"}
Oct 30 21:50:47.845928 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:47.845829    5510 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.87146014 podStartE2EDuration="5.845816103s" totalImagesPullingTime="1.962907985s" totalInitContainerRuntime="11.447978ms" isStatefulPod=false podCreationTimestamp="2026-10-30 21:50:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:50:47.844849476 +0000 UTC m=+28.405659494" watchObservedRunningTime="2026-10-30 21:50:47.845816103 +0000 UTC m=+28.406626107"
Oct 30 21:50:47.867875 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:47.867842    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-b787f5d44-6nhxn"
Oct 30 21:50:47.868047 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:47.867910    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-b787f5d44-6nhxn"
Oct 30 21:50:47.874015 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:47.873994    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-b787f5d44-6nhxn"
Oct 30 21:50:48.725307 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:48.725277    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-b787f5d44-6nhxn"
Oct 30 21:50:49.119604 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:49.119572    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:49.322127 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:49.322074    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:49.837085 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:49.837038    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:50.028421 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:50.028387    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:54.293526 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:54.293489    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:54.502548 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:54.502511    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:55.313805 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:50:55.313775    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-jmbng"
Oct 30 21:51:08.419400 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:51:08.419362    5510 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6gjbf" containerName="extractor"
Oct 30 21:51:13.489468 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:51:13.489423    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:13.725651 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:51:13.725613    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:18.023562 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:51:18.023523    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:33.231458 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:51:33.231420    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:33.421670 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:51:33.421625    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:38.418432 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:51:38.418392    5510 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6gjbf" containerName="extractor"
Oct 30 21:52:01.435459 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:52:01.435422    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:01.624258 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:52:01.624224    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:08.418612 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:52:08.418573    5510 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6gjbf" containerName="extractor"
Oct 30 21:52:18.349531 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:52:18.349491    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:38.418725 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:52:38.418689    5510 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6gjbf" containerName="extractor"
Oct 30 21:52:55.011988 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:52:55.011950    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:08.418006 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:53:08.417969    5510 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6gjbf" containerName="extractor"
Oct 30 21:53:18.696243 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:53:18.696196    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:38.418396 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:53:38.418311    5510 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6gjbf" containerName="extractor"
Oct 30 21:53:46.485076 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:53:46.485039    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:00.007948 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:54:00.007861    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:01.419742 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:54:01.419705    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:02.870298 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:54:02.870262    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:03.077595 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:54:03.077559    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:08.419322 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:54:08.419279    5510 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6gjbf" containerName="extractor"
Oct 30 21:54:19.033838 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:54:19.033794    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:24.358515 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:54:24.358480    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:24.566317 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:54:24.566281    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:38.418754 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:54:38.418713    5510 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6gjbf" containerName="extractor"
Oct 30 21:55:08.418634 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:55:08.418590    5510 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6gjbf" containerName="extractor"
Oct 30 21:55:19.363639 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:55:19.363600    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:25.546790 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:55:25.546751    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:25.743282 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:55:25.743245    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:38.418982 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:55:38.418923    5510 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6gjbf" containerName="extractor"
Oct 30 21:55:49.907784 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:55:49.907738    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:08.418615 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:56:08.418574    5510 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6gjbf" containerName="extractor"
Oct 30 21:56:12.612389 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:56:12.612348    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:19.702957 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:56:19.702922    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:23.308078 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:56:23.308041    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:23.538562 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:56:23.538526    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:24.703542 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:56:24.703503    5510 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644s4rm2"]
Oct 30 21:56:24.704022 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:56:24.703850    5510 cpu_manager.go:414] "containerMap: removing container" podUID="31a389bb-f36f-4e53-9654-5f4648dae38a" containerName="console"
Oct 30 21:56:24.704022 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:56:24.703865    5510 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="31a389bb-f36f-4e53-9654-5f4648dae38a" containerName="console"
Oct 30 21:56:24.704022 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:56:24.703882    5510 cpu_manager.go:414] "containerMap: removing container" podUID="42d54b8a-4f48-46a7-8a34-1f0f29cd1602" containerName="console"
Oct 30 21:56:24.704022 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:56:24.703888    5510 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="42d54b8a-4f48-46a7-8a34-1f0f29cd1602" containerName="console"
Oct 30 21:56:24.704022 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:56:24.703945    5510 memory_manager.go:365] "RemoveStaleState removing state" podUID="31a389bb-f36f-4e53-9654-5f4648dae38a" containerName="console"
Oct 30 21:56:24.704022 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:56:24.703959    5510 memory_manager.go:365] "RemoveStaleState removing state" podUID="42d54b8a-4f48-46a7-8a34-1f0f29cd1602" containerName="console"
Oct 30 21:56:24.707217 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:56:24.707198    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644s4rm2"
Oct 30 21:56:24.759545 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:56:24.759512    5510 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Oct 30 21:56:24.779959 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:56:24.779929    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9aca1c7e-bffa-48a8-af15-fa01c47eb344-util\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644s4rm2\" (UID: \"9aca1c7e-bffa-48a8-af15-fa01c47eb344\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644s4rm2"
Oct 30 21:56:24.780074 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:56:24.779979    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9aca1c7e-bffa-48a8-af15-fa01c47eb344-bundle\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644s4rm2\" (UID: \"9aca1c7e-bffa-48a8-af15-fa01c47eb344\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644s4rm2"
Oct 30 21:56:24.780136 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:56:24.780087    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5p7z4\" (UniqueName: \"kubernetes.io/projected/9aca1c7e-bffa-48a8-af15-fa01c47eb344-kube-api-access-5p7z4\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644s4rm2\" (UID: \"9aca1c7e-bffa-48a8-af15-fa01c47eb344\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644s4rm2"
Oct 30 21:56:24.805815 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:56:24.805790    5510 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Oct 30 21:56:24.805815 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:56:24.805806    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"c4959aeb960c8e20c88ac2de924a35b8221494-145dcd5a-dockercfg-fmpt6\""
Oct 30 21:56:24.813473 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:56:24.813453    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644s4rm2"]
Oct 30 21:56:24.881146 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:56:24.881113    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5p7z4\" (UniqueName: \"kubernetes.io/projected/9aca1c7e-bffa-48a8-af15-fa01c47eb344-kube-api-access-5p7z4\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644s4rm2\" (UID: \"9aca1c7e-bffa-48a8-af15-fa01c47eb344\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644s4rm2"
Oct 30 21:56:24.881336 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:56:24.881200    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9aca1c7e-bffa-48a8-af15-fa01c47eb344-util\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644s4rm2\" (UID: \"9aca1c7e-bffa-48a8-af15-fa01c47eb344\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644s4rm2"
Oct 30 21:56:24.881336 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:56:24.881234    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9aca1c7e-bffa-48a8-af15-fa01c47eb344-bundle\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644s4rm2\" (UID: \"9aca1c7e-bffa-48a8-af15-fa01c47eb344\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644s4rm2"
Oct 30 21:56:24.881586 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:56:24.881561    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9aca1c7e-bffa-48a8-af15-fa01c47eb344-util\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644s4rm2\" (UID: \"9aca1c7e-bffa-48a8-af15-fa01c47eb344\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644s4rm2"
Oct 30 21:56:24.881699 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:56:24.881608    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9aca1c7e-bffa-48a8-af15-fa01c47eb344-bundle\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644s4rm2\" (UID: \"9aca1c7e-bffa-48a8-af15-fa01c47eb344\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644s4rm2"
Oct 30 21:56:24.941256 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:56:24.941223    5510 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be715q8sf"]
Oct 30 21:56:24.943354 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:56:24.943333    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5p7z4\" (UniqueName: \"kubernetes.io/projected/9aca1c7e-bffa-48a8-af15-fa01c47eb344-kube-api-access-5p7z4\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644s4rm2\" (UID: \"9aca1c7e-bffa-48a8-af15-fa01c47eb344\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644s4rm2"
Oct 30 21:56:24.944799 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:56:24.944776    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be715q8sf"
Oct 30 21:56:24.982484 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:56:24.982453    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pdfqm\" (UniqueName: \"kubernetes.io/projected/e51dbfed-0488-4731-8e77-ba61fd4e9c7b-kube-api-access-pdfqm\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be715q8sf\" (UID: \"e51dbfed-0488-4731-8e77-ba61fd4e9c7b\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be715q8sf"
Oct 30 21:56:24.982613 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:56:24.982516    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e51dbfed-0488-4731-8e77-ba61fd4e9c7b-bundle\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be715q8sf\" (UID: \"e51dbfed-0488-4731-8e77-ba61fd4e9c7b\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be715q8sf"
Oct 30 21:56:24.982698 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:56:24.982640    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e51dbfed-0488-4731-8e77-ba61fd4e9c7b-util\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be715q8sf\" (UID: \"e51dbfed-0488-4731-8e77-ba61fd4e9c7b\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be715q8sf"
Oct 30 21:56:24.993953 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:56:24.993925    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"e644421527c169d876fdb59235c927986d6bd2-9d2815fc-dockercfg-zbtc4\""
Oct 30 21:56:25.008584 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:56:25.008559    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be715q8sf"]
Oct 30 21:56:25.022969 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:56:25.022948    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644s4rm2"
Oct 30 21:56:25.083443 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:56:25.083406    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pdfqm\" (UniqueName: \"kubernetes.io/projected/e51dbfed-0488-4731-8e77-ba61fd4e9c7b-kube-api-access-pdfqm\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be715q8sf\" (UID: \"e51dbfed-0488-4731-8e77-ba61fd4e9c7b\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be715q8sf"
Oct 30 21:56:25.083597 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:56:25.083498    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e51dbfed-0488-4731-8e77-ba61fd4e9c7b-bundle\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be715q8sf\" (UID: \"e51dbfed-0488-4731-8e77-ba61fd4e9c7b\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be715q8sf"
Oct 30 21:56:25.083597 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:56:25.083548    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e51dbfed-0488-4731-8e77-ba61fd4e9c7b-util\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be715q8sf\" (UID: \"e51dbfed-0488-4731-8e77-ba61fd4e9c7b\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be715q8sf"
Oct 30 21:56:25.083976 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:56:25.083954    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e51dbfed-0488-4731-8e77-ba61fd4e9c7b-util\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be715q8sf\" (UID: \"e51dbfed-0488-4731-8e77-ba61fd4e9c7b\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be715q8sf"
Oct 30 21:56:25.084090 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:56:25.084067    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e51dbfed-0488-4731-8e77-ba61fd4e9c7b-bundle\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be715q8sf\" (UID: \"e51dbfed-0488-4731-8e77-ba61fd4e9c7b\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be715q8sf"
Oct 30 21:56:25.142320 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:56:25.142291    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pdfqm\" (UniqueName: \"kubernetes.io/projected/e51dbfed-0488-4731-8e77-ba61fd4e9c7b-kube-api-access-pdfqm\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be715q8sf\" (UID: \"e51dbfed-0488-4731-8e77-ba61fd4e9c7b\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be715q8sf"
Oct 30 21:56:25.222586 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:56:25.222502    5510 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-ll8zm"]
Oct 30 21:56:25.226801 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:56:25.226780    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-ll8zm"
Oct 30 21:56:25.263463 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:56:25.263408    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be715q8sf"
Oct 30 21:56:25.284357 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:56:25.284310    5510 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Oct 30 21:56:25.284357 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:56:25.284341    5510 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Oct 30 21:56:25.284357 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:56:25.284341    5510 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Oct 30 21:56:25.284612 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:56:25.284562    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Oct 30 21:56:25.284612 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:56:25.284599    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Oct 30 21:56:25.284886 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:56:25.284613    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e3e9d86d-58cf-41b5-9932-03fa0ef94828-local-certs\") pod \"istiod-openshift-gateway-7876bfcb74-ll8zm\" (UID: \"e3e9d86d-58cf-41b5-9932-03fa0ef94828\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-ll8zm"
Oct 30 21:56:25.284886 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:56:25.284633    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Oct 30 21:56:25.284886 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:56:25.284622    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-wzxhd\""
Oct 30 21:56:25.284886 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:56:25.284705    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/e3e9d86d-58cf-41b5-9932-03fa0ef94828-cacerts\") pod \"istiod-openshift-gateway-7876bfcb74-ll8zm\" (UID: \"e3e9d86d-58cf-41b5-9932-03fa0ef94828\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-ll8zm"
Oct 30 21:56:25.284886 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:56:25.284735    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e3e9d86d-58cf-41b5-9932-03fa0ef94828-istio-kubeconfig\") pod \"istiod-openshift-gateway-7876bfcb74-ll8zm\" (UID: \"e3e9d86d-58cf-41b5-9932-03fa0ef94828\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-ll8zm"
Oct 30 21:56:25.284886 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:56:25.284811    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e3e9d86d-58cf-41b5-9932-03fa0ef94828-istio-token\") pod \"istiod-openshift-gateway-7876bfcb74-ll8zm\" (UID: \"e3e9d86d-58cf-41b5-9932-03fa0ef94828\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-ll8zm"
Oct 30 21:56:25.284886 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:56:25.284838    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e3e9d86d-58cf-41b5-9932-03fa0ef94828-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7876bfcb74-ll8zm\" (UID: \"e3e9d86d-58cf-41b5-9932-03fa0ef94828\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-ll8zm"
Oct 30 21:56:25.284886 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:56:25.284882    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qmd9v\" (UniqueName: \"kubernetes.io/projected/e3e9d86d-58cf-41b5-9932-03fa0ef94828-kube-api-access-qmd9v\") pod \"istiod-openshift-gateway-7876bfcb74-ll8zm\" (UID: \"e3e9d86d-58cf-41b5-9932-03fa0ef94828\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-ll8zm"
Oct 30 21:56:25.285231 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:56:25.284926    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e3e9d86d-58cf-41b5-9932-03fa0ef94828-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7876bfcb74-ll8zm\" (UID: \"e3e9d86d-58cf-41b5-9932-03fa0ef94828\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-ll8zm"
Oct 30 21:56:25.286800 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:56:25.286777    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:25.298424 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:56:25.298402    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-ll8zm"]
Oct 30 21:56:25.385780 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:56:25.385746    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e3e9d86d-58cf-41b5-9932-03fa0ef94828-istio-kubeconfig\") pod \"istiod-openshift-gateway-7876bfcb74-ll8zm\" (UID: \"e3e9d86d-58cf-41b5-9932-03fa0ef94828\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-ll8zm"
Oct 30 21:56:25.385780 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:56:25.385784    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e3e9d86d-58cf-41b5-9932-03fa0ef94828-istio-token\") pod \"istiod-openshift-gateway-7876bfcb74-ll8zm\" (UID: \"e3e9d86d-58cf-41b5-9932-03fa0ef94828\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-ll8zm"
Oct 30 21:56:25.385992 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:56:25.385807    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e3e9d86d-58cf-41b5-9932-03fa0ef94828-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7876bfcb74-ll8zm\" (UID: \"e3e9d86d-58cf-41b5-9932-03fa0ef94828\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-ll8zm"
Oct 30 21:56:25.385992 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:56:25.385832    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qmd9v\" (UniqueName: \"kubernetes.io/projected/e3e9d86d-58cf-41b5-9932-03fa0ef94828-kube-api-access-qmd9v\") pod \"istiod-openshift-gateway-7876bfcb74-ll8zm\" (UID: \"e3e9d86d-58cf-41b5-9932-03fa0ef94828\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-ll8zm"
Oct 30 21:56:25.385992 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:56:25.385863    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e3e9d86d-58cf-41b5-9932-03fa0ef94828-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7876bfcb74-ll8zm\" (UID: \"e3e9d86d-58cf-41b5-9932-03fa0ef94828\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-ll8zm"
Oct 30 21:56:25.385992 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:56:25.385884    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e3e9d86d-58cf-41b5-9932-03fa0ef94828-local-certs\") pod \"istiod-openshift-gateway-7876bfcb74-ll8zm\" (UID: \"e3e9d86d-58cf-41b5-9932-03fa0ef94828\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-ll8zm"
Oct 30 21:56:25.385992 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:56:25.385907    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/e3e9d86d-58cf-41b5-9932-03fa0ef94828-cacerts\") pod \"istiod-openshift-gateway-7876bfcb74-ll8zm\" (UID: \"e3e9d86d-58cf-41b5-9932-03fa0ef94828\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-ll8zm"
Oct 30 21:56:25.386530 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:56:25.386502    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e3e9d86d-58cf-41b5-9932-03fa0ef94828-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7876bfcb74-ll8zm\" (UID: \"e3e9d86d-58cf-41b5-9932-03fa0ef94828\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-ll8zm"
Oct 30 21:56:25.388217 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:56:25.388192    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e3e9d86d-58cf-41b5-9932-03fa0ef94828-local-certs\") pod \"istiod-openshift-gateway-7876bfcb74-ll8zm\" (UID: \"e3e9d86d-58cf-41b5-9932-03fa0ef94828\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-ll8zm"
Oct 30 21:56:25.388357 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:56:25.388329    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e3e9d86d-58cf-41b5-9932-03fa0ef94828-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7876bfcb74-ll8zm\" (UID: \"e3e9d86d-58cf-41b5-9932-03fa0ef94828\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-ll8zm"
Oct 30 21:56:25.388583 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:56:25.388564    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e3e9d86d-58cf-41b5-9932-03fa0ef94828-istio-kubeconfig\") pod \"istiod-openshift-gateway-7876bfcb74-ll8zm\" (UID: \"e3e9d86d-58cf-41b5-9932-03fa0ef94828\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-ll8zm"
Oct 30 21:56:25.388676 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:56:25.388628    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/e3e9d86d-58cf-41b5-9932-03fa0ef94828-cacerts\") pod \"istiod-openshift-gateway-7876bfcb74-ll8zm\" (UID: \"e3e9d86d-58cf-41b5-9932-03fa0ef94828\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-ll8zm"
Oct 30 21:56:25.412839 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:56:25.412812    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644s4rm2"]
Oct 30 21:56:25.415973 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:56:25.415942    5510 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9aca1c7e_bffa_48a8_af15_fa01c47eb344.slice/crio-afe0aed9e6f9e704300537783afdef2becb795f7b7ddfd9ecc6f4ca57f4f06dd WatchSource:0}: Error finding container afe0aed9e6f9e704300537783afdef2becb795f7b7ddfd9ecc6f4ca57f4f06dd: Status 404 returned error can't find the container with id afe0aed9e6f9e704300537783afdef2becb795f7b7ddfd9ecc6f4ca57f4f06dd
Oct 30 21:56:25.418047 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:56:25.418031    5510 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 21:56:25.444594 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:56:25.444564    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e3e9d86d-58cf-41b5-9932-03fa0ef94828-istio-token\") pod \"istiod-openshift-gateway-7876bfcb74-ll8zm\" (UID: \"e3e9d86d-58cf-41b5-9932-03fa0ef94828\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-ll8zm"
Oct 30 21:56:25.444746 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:56:25.444630    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qmd9v\" (UniqueName: \"kubernetes.io/projected/e3e9d86d-58cf-41b5-9932-03fa0ef94828-kube-api-access-qmd9v\") pod \"istiod-openshift-gateway-7876bfcb74-ll8zm\" (UID: \"e3e9d86d-58cf-41b5-9932-03fa0ef94828\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-ll8zm"
Oct 30 21:56:25.513574 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:56:25.513489    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:25.538265 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:56:25.538237    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-ll8zm"
Oct 30 21:56:25.632976 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:56:25.632944    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be715q8sf"]
Oct 30 21:56:25.636131 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:56:25.636102    5510 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode51dbfed_0488_4731_8e77_ba61fd4e9c7b.slice/crio-03cc11b63be4539c9aef3e87d06d6ea591a64368b2f580120b5f40a9649b0ae2 WatchSource:0}: Error finding container 03cc11b63be4539c9aef3e87d06d6ea591a64368b2f580120b5f40a9649b0ae2: Status 404 returned error can't find the container with id 03cc11b63be4539c9aef3e87d06d6ea591a64368b2f580120b5f40a9649b0ae2
Oct 30 21:56:25.859222 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:56:25.859182    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be715q8sf" event={"ID":"e51dbfed-0488-4731-8e77-ba61fd4e9c7b","Type":"ContainerStarted","Data":"03cc11b63be4539c9aef3e87d06d6ea591a64368b2f580120b5f40a9649b0ae2"}
Oct 30 21:56:25.860370 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:56:25.860349    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644s4rm2" event={"ID":"9aca1c7e-bffa-48a8-af15-fa01c47eb344","Type":"ContainerStarted","Data":"afe0aed9e6f9e704300537783afdef2becb795f7b7ddfd9ecc6f4ca57f4f06dd"}
Oct 30 21:56:25.930200 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:56:25.930177    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-ll8zm"]
Oct 30 21:56:25.932473 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:56:25.932449    5510 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode3e9d86d_58cf_41b5_9932_03fa0ef94828.slice/crio-701aea56080d745b74e01a00b5312651d130bb6066bcb402693b207c0aa5bf07 WatchSource:0}: Error finding container 701aea56080d745b74e01a00b5312651d130bb6066bcb402693b207c0aa5bf07: Status 404 returned error can't find the container with id 701aea56080d745b74e01a00b5312651d130bb6066bcb402693b207c0aa5bf07
Oct 30 21:56:26.784145 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:56:26.784103    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:26.866599 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:56:26.866554    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-ll8zm" event={"ID":"e3e9d86d-58cf-41b5-9932-03fa0ef94828","Type":"ContainerStarted","Data":"701aea56080d745b74e01a00b5312651d130bb6066bcb402693b207c0aa5bf07"}
Oct 30 21:56:30.191862 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:56:30.191819    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:56:30.192279 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:56:30.191909    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:56:30.898409 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:56:30.898367    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-ll8zm" event={"ID":"e3e9d86d-58cf-41b5-9932-03fa0ef94828","Type":"ContainerStarted","Data":"98914b5bd78a64ddc66a74e2b90ebaec15719f75ce8a322dbcfd0fafecd00b7b"}
Oct 30 21:56:30.898744 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:56:30.898693    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-ll8zm"
Oct 30 21:56:31.061756 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:56:31.061690    5510 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-ll8zm" podStartSLOduration=1.805969874 podStartE2EDuration="6.061672466s" totalImagesPullingTime="4.255702592s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:56:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:56:31.060433764 +0000 UTC m=+371.621243797" watchObservedRunningTime="2026-10-30 21:56:31.061672466 +0000 UTC m=+371.622482469"
Oct 30 21:56:31.794590 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:56:31.794557    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:31.904133 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:56:31.904096    5510 patch_prober.go:29] interesting pod/istiod-openshift-gateway-7876bfcb74-ll8zm container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 21:56:31.904301 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:56:31.904174    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-ll8zm" podUID="e3e9d86d-58cf-41b5-9932-03fa0ef94828" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 21:56:31.987112 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:56:31.987080    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:33.436538 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:56:33.436507    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:33.690249 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:56:33.690216    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:33.873589 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:56:33.873545    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:33.911914 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:56:33.911843    5510 generic.go:356] "Generic (PLEG): container finished" podID="e51dbfed-0488-4731-8e77-ba61fd4e9c7b" containerID="14ceff73ea3e3001f28adfad89cdb62fb595296a7738b193196dcec17494f944" exitCode=0
Oct 30 21:56:33.912043 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:56:33.911930    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be715q8sf" event={"ID":"e51dbfed-0488-4731-8e77-ba61fd4e9c7b","Type":"ContainerDied","Data":"14ceff73ea3e3001f28adfad89cdb62fb595296a7738b193196dcec17494f944"}
Oct 30 21:56:33.913393 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:56:33.913368    5510 generic.go:356] "Generic (PLEG): container finished" podID="9aca1c7e-bffa-48a8-af15-fa01c47eb344" containerID="a1f346e09e92932518228b39964e97ebb7d368626c45cf84585c6beb0518631e" exitCode=0
Oct 30 21:56:33.913496 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:56:33.913425    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644s4rm2" event={"ID":"9aca1c7e-bffa-48a8-af15-fa01c47eb344","Type":"ContainerDied","Data":"a1f346e09e92932518228b39964e97ebb7d368626c45cf84585c6beb0518631e"}
Oct 30 21:56:34.903426 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:56:34.903395    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-ll8zm"
Oct 30 21:56:35.922981 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:56:35.922950    5510 generic.go:356] "Generic (PLEG): container finished" podID="e51dbfed-0488-4731-8e77-ba61fd4e9c7b" containerID="583bd64e0dd0fc2d987eb6f3fad10506d824559d8385ea90d3645b93fb405641" exitCode=0
Oct 30 21:56:35.923353 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:56:35.923000    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be715q8sf" event={"ID":"e51dbfed-0488-4731-8e77-ba61fd4e9c7b","Type":"ContainerDied","Data":"583bd64e0dd0fc2d987eb6f3fad10506d824559d8385ea90d3645b93fb405641"}
Oct 30 21:56:37.600994 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:56:37.600956    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:37.933443 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:56:37.933391    5510 generic.go:356] "Generic (PLEG): container finished" podID="9aca1c7e-bffa-48a8-af15-fa01c47eb344" containerID="b59f996eb47a4c177da8c4faa282971c0f18bf065b34e2aa7218c2a297a61201" exitCode=0
Oct 30 21:56:37.933630 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:56:37.933479    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644s4rm2" event={"ID":"9aca1c7e-bffa-48a8-af15-fa01c47eb344","Type":"ContainerDied","Data":"b59f996eb47a4c177da8c4faa282971c0f18bf065b34e2aa7218c2a297a61201"}
Oct 30 21:56:38.423476 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:56:38.423410    5510 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6gjbf" containerName="extractor"
Oct 30 21:56:43.958588 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:56:43.958555    5510 generic.go:356] "Generic (PLEG): container finished" podID="e51dbfed-0488-4731-8e77-ba61fd4e9c7b" containerID="7ee7506a593aa70657d7c388a50eaec7c6f8d604f20c64302a97375b926e02aa" exitCode=0
Oct 30 21:56:43.959111 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:56:43.958642    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be715q8sf" event={"ID":"e51dbfed-0488-4731-8e77-ba61fd4e9c7b","Type":"ContainerDied","Data":"7ee7506a593aa70657d7c388a50eaec7c6f8d604f20c64302a97375b926e02aa"}
Oct 30 21:56:43.960699 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:56:43.960674    5510 generic.go:356] "Generic (PLEG): container finished" podID="9aca1c7e-bffa-48a8-af15-fa01c47eb344" containerID="823762b4778dca4af5220a268d81ffcbc2e2e4e41f8cedb7cf000de9d0d7e33c" exitCode=0
Oct 30 21:56:43.960805 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:56:43.960748    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644s4rm2" event={"ID":"9aca1c7e-bffa-48a8-af15-fa01c47eb344","Type":"ContainerDied","Data":"823762b4778dca4af5220a268d81ffcbc2e2e4e41f8cedb7cf000de9d0d7e33c"}
Oct 30 21:56:45.095610 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:56:45.095586    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be715q8sf"
Oct 30 21:56:45.108299 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:56:45.108278    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644s4rm2"
Oct 30 21:56:45.177083 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:56:45.177052    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e51dbfed-0488-4731-8e77-ba61fd4e9c7b-bundle\" (UniqueName: \"kubernetes.io/empty-dir/e51dbfed-0488-4731-8e77-ba61fd4e9c7b-bundle\") pod \"e51dbfed-0488-4731-8e77-ba61fd4e9c7b\" (UID: \"e51dbfed-0488-4731-8e77-ba61fd4e9c7b\") "
Oct 30 21:56:45.177249 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:56:45.177098    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9aca1c7e-bffa-48a8-af15-fa01c47eb344-util\" (UniqueName: \"kubernetes.io/empty-dir/9aca1c7e-bffa-48a8-af15-fa01c47eb344-util\") pod \"9aca1c7e-bffa-48a8-af15-fa01c47eb344\" (UID: \"9aca1c7e-bffa-48a8-af15-fa01c47eb344\") "
Oct 30 21:56:45.177249 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:56:45.177142    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e51dbfed-0488-4731-8e77-ba61fd4e9c7b-util\" (UniqueName: \"kubernetes.io/empty-dir/e51dbfed-0488-4731-8e77-ba61fd4e9c7b-util\") pod \"e51dbfed-0488-4731-8e77-ba61fd4e9c7b\" (UID: \"e51dbfed-0488-4731-8e77-ba61fd4e9c7b\") "
Oct 30 21:56:45.177249 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:56:45.177190    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9aca1c7e-bffa-48a8-af15-fa01c47eb344-bundle\" (UniqueName: \"kubernetes.io/empty-dir/9aca1c7e-bffa-48a8-af15-fa01c47eb344-bundle\") pod \"9aca1c7e-bffa-48a8-af15-fa01c47eb344\" (UID: \"9aca1c7e-bffa-48a8-af15-fa01c47eb344\") "
Oct 30 21:56:45.177249 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:56:45.177224    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e51dbfed-0488-4731-8e77-ba61fd4e9c7b-kube-api-access-pdfqm\" (UniqueName: \"kubernetes.io/projected/e51dbfed-0488-4731-8e77-ba61fd4e9c7b-kube-api-access-pdfqm\") pod \"e51dbfed-0488-4731-8e77-ba61fd4e9c7b\" (UID: \"e51dbfed-0488-4731-8e77-ba61fd4e9c7b\") "
Oct 30 21:56:45.177451 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:56:45.177264    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9aca1c7e-bffa-48a8-af15-fa01c47eb344-kube-api-access-5p7z4\" (UniqueName: \"kubernetes.io/projected/9aca1c7e-bffa-48a8-af15-fa01c47eb344-kube-api-access-5p7z4\") pod \"9aca1c7e-bffa-48a8-af15-fa01c47eb344\" (UID: \"9aca1c7e-bffa-48a8-af15-fa01c47eb344\") "
Oct 30 21:56:45.177647 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:56:45.177618    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9aca1c7e-bffa-48a8-af15-fa01c47eb344-bundle" pod "9aca1c7e-bffa-48a8-af15-fa01c47eb344" (UID: "9aca1c7e-bffa-48a8-af15-fa01c47eb344"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:56:45.177832 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:56:45.177800    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e51dbfed-0488-4731-8e77-ba61fd4e9c7b-bundle" pod "e51dbfed-0488-4731-8e77-ba61fd4e9c7b" (UID: "e51dbfed-0488-4731-8e77-ba61fd4e9c7b"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:56:45.179532 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:56:45.179510    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9aca1c7e-bffa-48a8-af15-fa01c47eb344-kube-api-access-5p7z4" pod "9aca1c7e-bffa-48a8-af15-fa01c47eb344" (UID: "9aca1c7e-bffa-48a8-af15-fa01c47eb344"). InnerVolumeSpecName "kube-api-access-5p7z4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:56:45.179730 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:56:45.179700    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e51dbfed-0488-4731-8e77-ba61fd4e9c7b-kube-api-access-pdfqm" pod "e51dbfed-0488-4731-8e77-ba61fd4e9c7b" (UID: "e51dbfed-0488-4731-8e77-ba61fd4e9c7b"). InnerVolumeSpecName "kube-api-access-pdfqm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:56:45.182351 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:56:45.182330    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9aca1c7e-bffa-48a8-af15-fa01c47eb344-util" pod "9aca1c7e-bffa-48a8-af15-fa01c47eb344" (UID: "9aca1c7e-bffa-48a8-af15-fa01c47eb344"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:56:45.182773 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:56:45.182755    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e51dbfed-0488-4731-8e77-ba61fd4e9c7b-util" pod "e51dbfed-0488-4731-8e77-ba61fd4e9c7b" (UID: "e51dbfed-0488-4731-8e77-ba61fd4e9c7b"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:56:45.278239 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:56:45.278211    5510 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9aca1c7e-bffa-48a8-af15-fa01c47eb344-bundle\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:56:45.278382 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:56:45.278240    5510 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pdfqm\" (UniqueName: \"kubernetes.io/projected/e51dbfed-0488-4731-8e77-ba61fd4e9c7b-kube-api-access-pdfqm\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:56:45.278382 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:56:45.278252    5510 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5p7z4\" (UniqueName: \"kubernetes.io/projected/9aca1c7e-bffa-48a8-af15-fa01c47eb344-kube-api-access-5p7z4\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:56:45.278382 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:56:45.278261    5510 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e51dbfed-0488-4731-8e77-ba61fd4e9c7b-bundle\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:56:45.278382 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:56:45.278270    5510 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9aca1c7e-bffa-48a8-af15-fa01c47eb344-util\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:56:45.278382 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:56:45.278277    5510 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e51dbfed-0488-4731-8e77-ba61fd4e9c7b-util\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:56:45.969310 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:56:45.969234    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be715q8sf"
Oct 30 21:56:45.969471 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:56:45.969234    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be715q8sf" event={"ID":"e51dbfed-0488-4731-8e77-ba61fd4e9c7b","Type":"ContainerDied","Data":"03cc11b63be4539c9aef3e87d06d6ea591a64368b2f580120b5f40a9649b0ae2"}
Oct 30 21:56:45.969471 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:56:45.969369    5510 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="03cc11b63be4539c9aef3e87d06d6ea591a64368b2f580120b5f40a9649b0ae2"
Oct 30 21:56:45.971267 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:56:45.971234    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644s4rm2" event={"ID":"9aca1c7e-bffa-48a8-af15-fa01c47eb344","Type":"ContainerDied","Data":"afe0aed9e6f9e704300537783afdef2becb795f7b7ddfd9ecc6f4ca57f4f06dd"}
Oct 30 21:56:45.971386 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:56:45.971277    5510 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="afe0aed9e6f9e704300537783afdef2becb795f7b7ddfd9ecc6f4ca57f4f06dd"
Oct 30 21:56:45.971386 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:56:45.971346    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644s4rm2"
Oct 30 21:57:05.492068 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:57:05.492028    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:05.731451 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:57:05.731412    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:08.419208 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:57:08.419174    5510 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6gjbf" containerName="extractor"
Oct 30 21:57:13.038553 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:57:13.038516    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:13.241681 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:57:13.241631    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:14.044056 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:57:14.044020    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:14.311925 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:57:14.311850    5510 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-748948fb58-sqks5"]
Oct 30 21:57:14.312159 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:57:14.312136    5510 cpu_manager.go:414] "containerMap: removing container" podUID="e51dbfed-0488-4731-8e77-ba61fd4e9c7b" containerName="pull"
Oct 30 21:57:14.312159 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:57:14.312152    5510 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e51dbfed-0488-4731-8e77-ba61fd4e9c7b" containerName="pull"
Oct 30 21:57:14.312288 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:57:14.312166    5510 cpu_manager.go:414] "containerMap: removing container" podUID="9aca1c7e-bffa-48a8-af15-fa01c47eb344" containerName="pull"
Oct 30 21:57:14.312288 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:57:14.312171    5510 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9aca1c7e-bffa-48a8-af15-fa01c47eb344" containerName="pull"
Oct 30 21:57:14.312288 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:57:14.312188    5510 cpu_manager.go:414] "containerMap: removing container" podUID="e51dbfed-0488-4731-8e77-ba61fd4e9c7b" containerName="util"
Oct 30 21:57:14.312288 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:57:14.312193    5510 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e51dbfed-0488-4731-8e77-ba61fd4e9c7b" containerName="util"
Oct 30 21:57:14.312288 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:57:14.312198    5510 cpu_manager.go:414] "containerMap: removing container" podUID="9aca1c7e-bffa-48a8-af15-fa01c47eb344" containerName="extract"
Oct 30 21:57:14.312288 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:57:14.312202    5510 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9aca1c7e-bffa-48a8-af15-fa01c47eb344" containerName="extract"
Oct 30 21:57:14.312288 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:57:14.312216    5510 cpu_manager.go:414] "containerMap: removing container" podUID="e51dbfed-0488-4731-8e77-ba61fd4e9c7b" containerName="extract"
Oct 30 21:57:14.312288 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:57:14.312221    5510 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e51dbfed-0488-4731-8e77-ba61fd4e9c7b" containerName="extract"
Oct 30 21:57:14.312288 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:57:14.312231    5510 cpu_manager.go:414] "containerMap: removing container" podUID="9aca1c7e-bffa-48a8-af15-fa01c47eb344" containerName="util"
Oct 30 21:57:14.312288 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:57:14.312236    5510 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9aca1c7e-bffa-48a8-af15-fa01c47eb344" containerName="util"
Oct 30 21:57:14.312288 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:57:14.312292    5510 memory_manager.go:365] "RemoveStaleState removing state" podUID="9aca1c7e-bffa-48a8-af15-fa01c47eb344" containerName="extract"
Oct 30 21:57:14.312820 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:57:14.312302    5510 memory_manager.go:365] "RemoveStaleState removing state" podUID="e51dbfed-0488-4731-8e77-ba61fd4e9c7b" containerName="extract"
Oct 30 21:57:14.322101 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:57:14.322077    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-748948fb58-sqks5"
Oct 30 21:57:14.394817 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:57:14.394773    5510 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub-operator-system\"/\"openshift-service-ca.crt\""
Oct 30 21:57:14.411421 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:57:14.411387    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/02cb5ea5-df80-417b-a769-f78b3b3c8798-tmp\") pod \"opendatahub-operator-controller-manager-748948fb58-sqks5\" (UID: \"02cb5ea5-df80-417b-a769-f78b3b3c8798\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-748948fb58-sqks5"
Oct 30 21:57:14.411592 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:57:14.411444    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/02cb5ea5-df80-417b-a769-f78b3b3c8798-webhook-cert\") pod \"opendatahub-operator-controller-manager-748948fb58-sqks5\" (UID: \"02cb5ea5-df80-417b-a769-f78b3b3c8798\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-748948fb58-sqks5"
Oct 30 21:57:14.411592 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:57:14.411514    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/02cb5ea5-df80-417b-a769-f78b3b3c8798-apiservice-cert\") pod \"opendatahub-operator-controller-manager-748948fb58-sqks5\" (UID: \"02cb5ea5-df80-417b-a769-f78b3b3c8798\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-748948fb58-sqks5"
Oct 30 21:57:14.411698 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:57:14.411612    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/02cb5ea5-df80-417b-a769-f78b3b3c8798-metrics-cert\") pod \"opendatahub-operator-controller-manager-748948fb58-sqks5\" (UID: \"02cb5ea5-df80-417b-a769-f78b3b3c8798\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-748948fb58-sqks5"
Oct 30 21:57:14.411698 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:57:14.411637    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/02cb5ea5-df80-417b-a769-f78b3b3c8798-manifests\") pod \"opendatahub-operator-controller-manager-748948fb58-sqks5\" (UID: \"02cb5ea5-df80-417b-a769-f78b3b3c8798\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-748948fb58-sqks5"
Oct 30 21:57:14.411771 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:57:14.411702    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6nhk4\" (UniqueName: \"kubernetes.io/projected/02cb5ea5-df80-417b-a769-f78b3b3c8798-kube-api-access-6nhk4\") pod \"opendatahub-operator-controller-manager-748948fb58-sqks5\" (UID: \"02cb5ea5-df80-417b-a769-f78b3b3c8798\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-748948fb58-sqks5"
Oct 30 21:57:14.412523 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:57:14.412494    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:14.418427 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:57:14.418403    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-webhook-cert\""
Oct 30 21:57:14.418546 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:57:14.418439    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-manager-dockercfg-ng7kl\""
Oct 30 21:57:14.418546 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:57:14.418464    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-metrics-tls\""
Oct 30 21:57:14.418546 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:57:14.418472    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-manager-service-cert\""
Oct 30 21:57:14.418546 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:57:14.418493    5510 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub-operator-system\"/\"kube-root-ca.crt\""
Oct 30 21:57:14.419898 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:57:14.419881    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-748948fb58-sqks5"]
Oct 30 21:57:14.512907 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:57:14.512870    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/02cb5ea5-df80-417b-a769-f78b3b3c8798-apiservice-cert\") pod \"opendatahub-operator-controller-manager-748948fb58-sqks5\" (UID: \"02cb5ea5-df80-417b-a769-f78b3b3c8798\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-748948fb58-sqks5"
Oct 30 21:57:14.513122 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:57:14.512914    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/02cb5ea5-df80-417b-a769-f78b3b3c8798-metrics-cert\") pod \"opendatahub-operator-controller-manager-748948fb58-sqks5\" (UID: \"02cb5ea5-df80-417b-a769-f78b3b3c8798\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-748948fb58-sqks5"
Oct 30 21:57:14.513122 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:57:14.512934    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/02cb5ea5-df80-417b-a769-f78b3b3c8798-manifests\") pod \"opendatahub-operator-controller-manager-748948fb58-sqks5\" (UID: \"02cb5ea5-df80-417b-a769-f78b3b3c8798\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-748948fb58-sqks5"
Oct 30 21:57:14.513122 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:57:14.512959    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6nhk4\" (UniqueName: \"kubernetes.io/projected/02cb5ea5-df80-417b-a769-f78b3b3c8798-kube-api-access-6nhk4\") pod \"opendatahub-operator-controller-manager-748948fb58-sqks5\" (UID: \"02cb5ea5-df80-417b-a769-f78b3b3c8798\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-748948fb58-sqks5"
Oct 30 21:57:14.513122 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:57:14.513085    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/02cb5ea5-df80-417b-a769-f78b3b3c8798-tmp\") pod \"opendatahub-operator-controller-manager-748948fb58-sqks5\" (UID: \"02cb5ea5-df80-417b-a769-f78b3b3c8798\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-748948fb58-sqks5"
Oct 30 21:57:14.513363 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:57:14.513170    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/02cb5ea5-df80-417b-a769-f78b3b3c8798-webhook-cert\") pod \"opendatahub-operator-controller-manager-748948fb58-sqks5\" (UID: \"02cb5ea5-df80-417b-a769-f78b3b3c8798\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-748948fb58-sqks5"
Oct 30 21:57:14.513897 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:57:14.513818    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/02cb5ea5-df80-417b-a769-f78b3b3c8798-manifests\") pod \"opendatahub-operator-controller-manager-748948fb58-sqks5\" (UID: \"02cb5ea5-df80-417b-a769-f78b3b3c8798\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-748948fb58-sqks5"
Oct 30 21:57:14.516561 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:57:14.516528    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/02cb5ea5-df80-417b-a769-f78b3b3c8798-tmp\") pod \"opendatahub-operator-controller-manager-748948fb58-sqks5\" (UID: \"02cb5ea5-df80-417b-a769-f78b3b3c8798\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-748948fb58-sqks5"
Oct 30 21:57:14.516706 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:57:14.516626    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/02cb5ea5-df80-417b-a769-f78b3b3c8798-apiservice-cert\") pod \"opendatahub-operator-controller-manager-748948fb58-sqks5\" (UID: \"02cb5ea5-df80-417b-a769-f78b3b3c8798\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-748948fb58-sqks5"
Oct 30 21:57:14.518212 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:57:14.518073    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/02cb5ea5-df80-417b-a769-f78b3b3c8798-webhook-cert\") pod \"opendatahub-operator-controller-manager-748948fb58-sqks5\" (UID: \"02cb5ea5-df80-417b-a769-f78b3b3c8798\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-748948fb58-sqks5"
Oct 30 21:57:14.520717 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:57:14.520696    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/02cb5ea5-df80-417b-a769-f78b3b3c8798-metrics-cert\") pod \"opendatahub-operator-controller-manager-748948fb58-sqks5\" (UID: \"02cb5ea5-df80-417b-a769-f78b3b3c8798\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-748948fb58-sqks5"
Oct 30 21:57:14.571565 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:57:14.571492    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6nhk4\" (UniqueName: \"kubernetes.io/projected/02cb5ea5-df80-417b-a769-f78b3b3c8798-kube-api-access-6nhk4\") pod \"opendatahub-operator-controller-manager-748948fb58-sqks5\" (UID: \"02cb5ea5-df80-417b-a769-f78b3b3c8798\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-748948fb58-sqks5"
Oct 30 21:57:14.658558 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:57:14.658524    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-748948fb58-sqks5"
Oct 30 21:57:14.721177 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:57:14.721137    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:14.948025 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:57:14.947940    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:15.219793 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:57:15.219707    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-748948fb58-sqks5"]
Oct 30 21:57:15.269552 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:57:15.269518    5510 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02cb5ea5_df80_417b_a769_f78b3b3c8798.slice/crio-eedfa25b7d5468d8e99d293df4b8b8dcf41526843bb587af09b5f408c6fc3e6a WatchSource:0}: Error finding container eedfa25b7d5468d8e99d293df4b8b8dcf41526843bb587af09b5f408c6fc3e6a: Status 404 returned error can't find the container with id eedfa25b7d5468d8e99d293df4b8b8dcf41526843bb587af09b5f408c6fc3e6a
Oct 30 21:57:16.079689 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:57:16.079640    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-748948fb58-sqks5" event={"ID":"02cb5ea5-df80-417b-a769-f78b3b3c8798","Type":"ContainerStarted","Data":"eedfa25b7d5468d8e99d293df4b8b8dcf41526843bb587af09b5f408c6fc3e6a"}
Oct 30 21:57:17.416413 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:57:17.416366    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:17.872392 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:57:17.872353    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:18.644935 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:57:18.644896    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:19.093123 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:57:19.093086    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-748948fb58-sqks5" event={"ID":"02cb5ea5-df80-417b-a769-f78b3b3c8798","Type":"ContainerStarted","Data":"67e8fda1f8219c0a11eef0475fdcfa8455bfec6f6d8362e9605e2ea512f04f1e"}
Oct 30 21:57:19.804816 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:57:19.804783    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:20.098890 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:57:20.098809    5510 generic.go:356] "Generic (PLEG): container finished" podID="02cb5ea5-df80-417b-a769-f78b3b3c8798" containerID="67e8fda1f8219c0a11eef0475fdcfa8455bfec6f6d8362e9605e2ea512f04f1e" exitCode=0
Oct 30 21:57:20.098890 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:57:20.098847    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-748948fb58-sqks5" event={"ID":"02cb5ea5-df80-417b-a769-f78b3b3c8798","Type":"ContainerDied","Data":"67e8fda1f8219c0a11eef0475fdcfa8455bfec6f6d8362e9605e2ea512f04f1e"}
Oct 30 21:57:20.141350 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:57:20.141320    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:21.105021 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:57:21.104982    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-748948fb58-sqks5" event={"ID":"02cb5ea5-df80-417b-a769-f78b3b3c8798","Type":"ContainerStarted","Data":"6b46337fd199a8197cb14484d344297892de373eddcdc67c418a2c5d59d7d7f9"}
Oct 30 21:57:21.105496 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:57:21.105100    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-748948fb58-sqks5"
Oct 30 21:57:21.325442 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:57:21.325383    5510 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-748948fb58-sqks5" podStartSLOduration=2.627559784 podStartE2EDuration="7.325368784s" totalImagesPullingTime="4.276100404s" totalInitContainerRuntime="421.708596ms" isStatefulPod=false podCreationTimestamp="2026-10-30 21:57:14 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:57:21.301174787 +0000 UTC m=+421.861984812" watchObservedRunningTime="2026-10-30 21:57:21.325368784 +0000 UTC m=+421.886178855"
Oct 30 21:57:23.001614 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:57:23.001581    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:25.287308 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:57:25.287271    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:27.069927 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:57:27.069891    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:27.277440 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:57:27.277397    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:28.037526 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:57:28.037485    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:29.909202 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:57:29.909163    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:30.413771 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:57:30.413740    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:31.006790 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:57:31.006744    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:32.111400 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:57:32.111370    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-748948fb58-sqks5"
Oct 30 21:57:34.230431 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:57:34.230395    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:34.686026 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:57:34.685997    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:34.894428 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:57:34.894395    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:35.116510 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:57:35.116479    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:36.788431 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:57:36.788400    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:37.113032 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:57:37.112949    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:37.690569 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:57:37.690531    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:37.959226 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:57:37.959141    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:38.167303 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:57:38.167263    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:38.418802 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:57:38.418766    5510 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6gjbf" containerName="extractor"
Oct 30 21:57:38.904489 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:57:38.904433    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:39.159273 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:57:39.159179    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:41.607822 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:57:41.607788    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:41.838825 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:57:41.838783    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:42.033156 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:57:42.033126    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:42.426651 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:57:42.426567    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:42.611126 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:57:42.611086    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:45.330950 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:57:45.330917    5510 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-597b95cb84-pjs7v"]
Oct 30 21:57:45.334281 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:57:45.334257    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-597b95cb84-pjs7v"
Oct 30 21:57:45.387615 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:57:45.387588    5510 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Oct 30 21:57:45.430567 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:57:45.430519    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-bbf9d\""
Oct 30 21:57:45.430567 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:57:45.430569    5510 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Oct 30 21:57:45.431451 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:57:45.431429    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-597b95cb84-pjs7v"]
Oct 30 21:57:45.483494 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:57:45.483460    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h4lv5\" (UniqueName: \"kubernetes.io/projected/2f34351e-2719-4661-aab5-d242e8d2243e-kube-api-access-h4lv5\") pod \"cert-manager-597b95cb84-pjs7v\" (UID: \"2f34351e-2719-4661-aab5-d242e8d2243e\") " pod="cert-manager/cert-manager-597b95cb84-pjs7v"
Oct 30 21:57:45.483618 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:57:45.483511    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2f34351e-2719-4661-aab5-d242e8d2243e-bound-sa-token\") pod \"cert-manager-597b95cb84-pjs7v\" (UID: \"2f34351e-2719-4661-aab5-d242e8d2243e\") " pod="cert-manager/cert-manager-597b95cb84-pjs7v"
Oct 30 21:57:45.545981 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:57:45.545942    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:45.584093 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:57:45.584021    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h4lv5\" (UniqueName: \"kubernetes.io/projected/2f34351e-2719-4661-aab5-d242e8d2243e-kube-api-access-h4lv5\") pod \"cert-manager-597b95cb84-pjs7v\" (UID: \"2f34351e-2719-4661-aab5-d242e8d2243e\") " pod="cert-manager/cert-manager-597b95cb84-pjs7v"
Oct 30 21:57:45.584093 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:57:45.584067    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2f34351e-2719-4661-aab5-d242e8d2243e-bound-sa-token\") pod \"cert-manager-597b95cb84-pjs7v\" (UID: \"2f34351e-2719-4661-aab5-d242e8d2243e\") " pod="cert-manager/cert-manager-597b95cb84-pjs7v"
Oct 30 21:57:45.647571 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:57:45.647537    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2f34351e-2719-4661-aab5-d242e8d2243e-bound-sa-token\") pod \"cert-manager-597b95cb84-pjs7v\" (UID: \"2f34351e-2719-4661-aab5-d242e8d2243e\") " pod="cert-manager/cert-manager-597b95cb84-pjs7v"
Oct 30 21:57:45.648219 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:57:45.648192    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h4lv5\" (UniqueName: \"kubernetes.io/projected/2f34351e-2719-4661-aab5-d242e8d2243e-kube-api-access-h4lv5\") pod \"cert-manager-597b95cb84-pjs7v\" (UID: \"2f34351e-2719-4661-aab5-d242e8d2243e\") " pod="cert-manager/cert-manager-597b95cb84-pjs7v"
Oct 30 21:57:45.654049 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:57:45.654030    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-597b95cb84-pjs7v"
Oct 30 21:57:46.029143 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:57:46.028970    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-597b95cb84-pjs7v"]
Oct 30 21:57:46.035616 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:57:46.035587    5510 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2f34351e_2719_4661_aab5_d242e8d2243e.slice/crio-131e4e2e7963d98d9a63399c6a90d623b18b35a4712024891655d2cd5a1c8b0e WatchSource:0}: Error finding container 131e4e2e7963d98d9a63399c6a90d623b18b35a4712024891655d2cd5a1c8b0e: Status 404 returned error can't find the container with id 131e4e2e7963d98d9a63399c6a90d623b18b35a4712024891655d2cd5a1c8b0e
Oct 30 21:57:46.195100 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:57:46.195070    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-597b95cb84-pjs7v" event={"ID":"2f34351e-2719-4661-aab5-d242e8d2243e","Type":"ContainerStarted","Data":"131e4e2e7963d98d9a63399c6a90d623b18b35a4712024891655d2cd5a1c8b0e"}
Oct 30 21:57:47.261102 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:57:47.261064    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:47.606700 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:57:47.606636    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:47.910263 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:57:47.910175    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:48.382632 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:57:48.382589    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:48.601893 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:57:48.601859    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:49.445548 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:57:49.445515    5510 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-84877cfc-8nxg5"]
Oct 30 21:57:49.449991 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:57:49.449971    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-84877cfc-8nxg5"
Oct 30 21:57:49.464702 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:57:49.464678    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:49.503121 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:57:49.503093    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Oct 30 21:57:49.503570 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:57:49.503549    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-hnqxt\""
Oct 30 21:57:49.517378 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:57:49.517347    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bae2a39a-a18e-4b74-b6ef-3879493afb2a-tls-certs\") pod \"kube-auth-proxy-84877cfc-8nxg5\" (UID: \"bae2a39a-a18e-4b74-b6ef-3879493afb2a\") " pod="openshift-ingress/kube-auth-proxy-84877cfc-8nxg5"
Oct 30 21:57:49.517592 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:57:49.517566    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bae2a39a-a18e-4b74-b6ef-3879493afb2a-tmp\") pod \"kube-auth-proxy-84877cfc-8nxg5\" (UID: \"bae2a39a-a18e-4b74-b6ef-3879493afb2a\") " pod="openshift-ingress/kube-auth-proxy-84877cfc-8nxg5"
Oct 30 21:57:49.517776 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:57:49.517612    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bbhm2\" (UniqueName: \"kubernetes.io/projected/bae2a39a-a18e-4b74-b6ef-3879493afb2a-kube-api-access-bbhm2\") pod \"kube-auth-proxy-84877cfc-8nxg5\" (UID: \"bae2a39a-a18e-4b74-b6ef-3879493afb2a\") " pod="openshift-ingress/kube-auth-proxy-84877cfc-8nxg5"
Oct 30 21:57:49.549745 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:57:49.549717    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Oct 30 21:57:49.551404 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:57:49.551382    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-84877cfc-8nxg5"]
Oct 30 21:57:49.618355 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:57:49.618322    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bae2a39a-a18e-4b74-b6ef-3879493afb2a-tls-certs\") pod \"kube-auth-proxy-84877cfc-8nxg5\" (UID: \"bae2a39a-a18e-4b74-b6ef-3879493afb2a\") " pod="openshift-ingress/kube-auth-proxy-84877cfc-8nxg5"
Oct 30 21:57:49.618355 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:57:49.618357    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bae2a39a-a18e-4b74-b6ef-3879493afb2a-tmp\") pod \"kube-auth-proxy-84877cfc-8nxg5\" (UID: \"bae2a39a-a18e-4b74-b6ef-3879493afb2a\") " pod="openshift-ingress/kube-auth-proxy-84877cfc-8nxg5"
Oct 30 21:57:49.618602 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:57:49.618399    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bbhm2\" (UniqueName: \"kubernetes.io/projected/bae2a39a-a18e-4b74-b6ef-3879493afb2a-kube-api-access-bbhm2\") pod \"kube-auth-proxy-84877cfc-8nxg5\" (UID: \"bae2a39a-a18e-4b74-b6ef-3879493afb2a\") " pod="openshift-ingress/kube-auth-proxy-84877cfc-8nxg5"
Oct 30 21:57:49.620724 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:57:49.620699    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bae2a39a-a18e-4b74-b6ef-3879493afb2a-tmp\") pod \"kube-auth-proxy-84877cfc-8nxg5\" (UID: \"bae2a39a-a18e-4b74-b6ef-3879493afb2a\") " pod="openshift-ingress/kube-auth-proxy-84877cfc-8nxg5"
Oct 30 21:57:49.620939 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:57:49.620919    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bae2a39a-a18e-4b74-b6ef-3879493afb2a-tls-certs\") pod \"kube-auth-proxy-84877cfc-8nxg5\" (UID: \"bae2a39a-a18e-4b74-b6ef-3879493afb2a\") " pod="openshift-ingress/kube-auth-proxy-84877cfc-8nxg5"
Oct 30 21:57:49.672110 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:57:49.672064    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:49.677697 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:57:49.677649    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bbhm2\" (UniqueName: \"kubernetes.io/projected/bae2a39a-a18e-4b74-b6ef-3879493afb2a-kube-api-access-bbhm2\") pod \"kube-auth-proxy-84877cfc-8nxg5\" (UID: \"bae2a39a-a18e-4b74-b6ef-3879493afb2a\") " pod="openshift-ingress/kube-auth-proxy-84877cfc-8nxg5"
Oct 30 21:57:49.761728 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:57:49.761635    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-84877cfc-8nxg5"
Oct 30 21:57:49.929024 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:57:49.928984    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:50.129539 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:57:50.129502    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-84877cfc-8nxg5"]
Oct 30 21:57:50.159123 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:57:50.159076    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:50.211355 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:57:50.211320    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-84877cfc-8nxg5" event={"ID":"bae2a39a-a18e-4b74-b6ef-3879493afb2a","Type":"ContainerStarted","Data":"d55867445a4935b47f6be5ecf78dc3edb9ec20059293409a2bbb315fc9b70afd"}
Oct 30 21:57:50.354520 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:57:50.354481    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:54.235901 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:57:54.235863    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-84877cfc-8nxg5" event={"ID":"bae2a39a-a18e-4b74-b6ef-3879493afb2a","Type":"ContainerStarted","Data":"859dffddccc7756142b172952e53a6414c34b315ded5678e3ab72d61904b637e"}
Oct 30 21:57:54.349027 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:57:54.348976    5510 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-84877cfc-8nxg5" podStartSLOduration=2.315928046 podStartE2EDuration="5.348961489s" totalImagesPullingTime="3.033033443s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:57:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:57:54.348189798 +0000 UTC m=+454.908999828" watchObservedRunningTime="2026-10-30 21:57:54.348961489 +0000 UTC m=+454.909771550"
Oct 30 21:57:54.863467 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:57:54.863311    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:55.086116 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:57:55.086075    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:57.250730 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:57:57.250695    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-597b95cb84-pjs7v" event={"ID":"2f34351e-2719-4661-aab5-d242e8d2243e","Type":"ContainerStarted","Data":"c9214d8d198cb7c11a29f1d0107e3eae93d073b1cd5302bcba02949667d430a3"}
Oct 30 21:57:57.381911 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:57:57.381856    5510 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="cert-manager/cert-manager-597b95cb84-pjs7v" podStartSLOduration=2.043315779 podStartE2EDuration="12.381842107s" totalImagesPullingTime="10.338526328s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:57:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:57:57.381437255 +0000 UTC m=+457.942247261" watchObservedRunningTime="2026-10-30 21:57:57.381842107 +0000 UTC m=+457.942652109"
Oct 30 21:57:57.943331 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:57:57.943293    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:58.233137 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:57:58.233045    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:59.334042 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:57:59.333999    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:59.602804 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:57:59.602725    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:59.785274 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:57:59.785237    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:58:03.120675 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:58:03.120606    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:58:03.301155 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:58:03.301111    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:58:03.500327 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:58:03.500240    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:58:03.688814 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:58:03.688786    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:58:08.418812 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:58:08.418772    5510 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6gjbf" containerName="extractor"
Oct 30 21:58:09.704059 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:58:09.704022    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:58:09.894794 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:58:09.894759    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:58:16.234988 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:58:16.234937    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:58:16.467585 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:58:16.467549    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:58:17.295178 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:58:17.295143    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:58:17.326971 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:58:17.326935    5510 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-nt8mp"]
Oct 30 21:58:17.330459 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:58:17.330435    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-74dfcfb894-nt8mp"
Oct 30 21:58:17.426355 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:58:17.426324    5510 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Oct 30 21:58:17.426355 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:58:17.426342    5510 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Oct 30 21:58:17.426738 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:58:17.426342    5510 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Oct 30 21:58:17.426738 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:58:17.426394    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-7m7zw\""
Oct 30 21:58:17.427564 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:58:17.427538    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-nt8mp"]
Oct 30 21:58:17.452916 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:58:17.452882    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qsqh4\" (UniqueName: \"kubernetes.io/projected/9d6febf4-e9f9-47ce-ac73-9e2a296ad567-kube-api-access-qsqh4\") pod \"limitador-limitador-74dfcfb894-nt8mp\" (UID: \"9d6febf4-e9f9-47ce-ac73-9e2a296ad567\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-nt8mp"
Oct 30 21:58:17.453054 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:58:17.452933    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/9d6febf4-e9f9-47ce-ac73-9e2a296ad567-config-file\") pod \"limitador-limitador-74dfcfb894-nt8mp\" (UID: \"9d6febf4-e9f9-47ce-ac73-9e2a296ad567\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-nt8mp"
Oct 30 21:58:17.554110 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:58:17.554035    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qsqh4\" (UniqueName: \"kubernetes.io/projected/9d6febf4-e9f9-47ce-ac73-9e2a296ad567-kube-api-access-qsqh4\") pod \"limitador-limitador-74dfcfb894-nt8mp\" (UID: \"9d6febf4-e9f9-47ce-ac73-9e2a296ad567\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-nt8mp"
Oct 30 21:58:17.554110 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:58:17.554073    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/9d6febf4-e9f9-47ce-ac73-9e2a296ad567-config-file\") pod \"limitador-limitador-74dfcfb894-nt8mp\" (UID: \"9d6febf4-e9f9-47ce-ac73-9e2a296ad567\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-nt8mp"
Oct 30 21:58:17.554684 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:58:17.554640    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/9d6febf4-e9f9-47ce-ac73-9e2a296ad567-config-file\") pod \"limitador-limitador-74dfcfb894-nt8mp\" (UID: \"9d6febf4-e9f9-47ce-ac73-9e2a296ad567\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-nt8mp"
Oct 30 21:58:17.566808 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:58:17.566778    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-nt8mp"]
Oct 30 21:58:17.612478 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:58:17.612447    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qsqh4\" (UniqueName: \"kubernetes.io/projected/9d6febf4-e9f9-47ce-ac73-9e2a296ad567-kube-api-access-qsqh4\") pod \"limitador-limitador-74dfcfb894-nt8mp\" (UID: \"9d6febf4-e9f9-47ce-ac73-9e2a296ad567\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-nt8mp"
Oct 30 21:58:17.643283 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:58:17.643255    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-74dfcfb894-nt8mp"
Oct 30 21:58:18.007200 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:58:18.007122    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-nt8mp"]
Oct 30 21:58:18.011031 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:58:18.010986    5510 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d6febf4_e9f9_47ce_ac73_9e2a296ad567.slice/crio-25b34b365da5c9d0aed4c312a870fb584c6f43b6dcd9897bf518c5b14151daf9 WatchSource:0}: Error finding container 25b34b365da5c9d0aed4c312a870fb584c6f43b6dcd9897bf518c5b14151daf9: Status 404 returned error can't find the container with id 25b34b365da5c9d0aed4c312a870fb584c6f43b6dcd9897bf518c5b14151daf9
Oct 30 21:58:18.336128 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:58:18.336093    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-74dfcfb894-nt8mp" event={"ID":"9d6febf4-e9f9-47ce-ac73-9e2a296ad567","Type":"ContainerStarted","Data":"25b34b365da5c9d0aed4c312a870fb584c6f43b6dcd9897bf518c5b14151daf9"}
Oct 30 21:58:19.233206 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:58:19.233168    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:58:19.425251 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:58:19.425214    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:58:20.384961 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:58:20.384924    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:58:23.366048 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:58:23.366004    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-74dfcfb894-nt8mp" event={"ID":"9d6febf4-e9f9-47ce-ac73-9e2a296ad567","Type":"ContainerStarted","Data":"d095ceb11814e488fa545150f012d640c63a9ae22ffff1c2c84057bc59df777b"}
Oct 30 21:58:23.366532 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:58:23.366088    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-74dfcfb894-nt8mp"
Oct 30 21:58:23.477131 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:58:23.477078    5510 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-74dfcfb894-nt8mp" podStartSLOduration=1.757473399 podStartE2EDuration="6.477063745s" totalImagesPullingTime="4.719590346s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:58:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:58:23.476435338 +0000 UTC m=+484.037245343" watchObservedRunningTime="2026-10-30 21:58:23.477063745 +0000 UTC m=+484.037873749"
Oct 30 21:58:34.371643 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:58:34.371609    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-74dfcfb894-nt8mp"
Oct 30 21:58:37.046392 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:58:37.046349    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-nt8mp"]
Oct 30 21:58:37.046920 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:58:37.046623    5510 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-74dfcfb894-nt8mp" podUID="9d6febf4-e9f9-47ce-ac73-9e2a296ad567" containerName="limitador" containerID="cri-o://d095ceb11814e488fa545150f012d640c63a9ae22ffff1c2c84057bc59df777b" gracePeriod=30
Oct 30 21:58:37.408816 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:58:37.408782    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:58:37.588927 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:58:37.588900    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-74dfcfb894-nt8mp"
Oct 30 21:58:37.725394 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:58:37.725320    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9d6febf4-e9f9-47ce-ac73-9e2a296ad567-kube-api-access-qsqh4\" (UniqueName: \"kubernetes.io/projected/9d6febf4-e9f9-47ce-ac73-9e2a296ad567-kube-api-access-qsqh4\") pod \"9d6febf4-e9f9-47ce-ac73-9e2a296ad567\" (UID: \"9d6febf4-e9f9-47ce-ac73-9e2a296ad567\") "
Oct 30 21:58:37.725394 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:58:37.725388    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9d6febf4-e9f9-47ce-ac73-9e2a296ad567-config-file\" (UniqueName: \"kubernetes.io/configmap/9d6febf4-e9f9-47ce-ac73-9e2a296ad567-config-file\") pod \"9d6febf4-e9f9-47ce-ac73-9e2a296ad567\" (UID: \"9d6febf4-e9f9-47ce-ac73-9e2a296ad567\") "
Oct 30 21:58:37.725763 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:58:37.725742    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9d6febf4-e9f9-47ce-ac73-9e2a296ad567-config-file" pod "9d6febf4-e9f9-47ce-ac73-9e2a296ad567" (UID: "9d6febf4-e9f9-47ce-ac73-9e2a296ad567"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:58:37.727474 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:58:37.727452    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9d6febf4-e9f9-47ce-ac73-9e2a296ad567-kube-api-access-qsqh4" pod "9d6febf4-e9f9-47ce-ac73-9e2a296ad567" (UID: "9d6febf4-e9f9-47ce-ac73-9e2a296ad567"). InnerVolumeSpecName "kube-api-access-qsqh4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:58:37.826920 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:58:37.826882    5510 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qsqh4\" (UniqueName: \"kubernetes.io/projected/9d6febf4-e9f9-47ce-ac73-9e2a296ad567-kube-api-access-qsqh4\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:58:37.826920 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:58:37.826913    5510 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/9d6febf4-e9f9-47ce-ac73-9e2a296ad567-config-file\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:58:38.419275 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:58:38.419234    5510 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6gjbf" containerName="extractor"
Oct 30 21:58:38.424938 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:58:38.424908    5510 generic.go:356] "Generic (PLEG): container finished" podID="9d6febf4-e9f9-47ce-ac73-9e2a296ad567" containerID="d095ceb11814e488fa545150f012d640c63a9ae22ffff1c2c84057bc59df777b" exitCode=0
Oct 30 21:58:38.425069 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:58:38.424962    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-74dfcfb894-nt8mp"
Oct 30 21:58:38.425069 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:58:38.424990    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-74dfcfb894-nt8mp" event={"ID":"9d6febf4-e9f9-47ce-ac73-9e2a296ad567","Type":"ContainerDied","Data":"d095ceb11814e488fa545150f012d640c63a9ae22ffff1c2c84057bc59df777b"}
Oct 30 21:58:38.425069 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:58:38.425046    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-74dfcfb894-nt8mp" event={"ID":"9d6febf4-e9f9-47ce-ac73-9e2a296ad567","Type":"ContainerDied","Data":"25b34b365da5c9d0aed4c312a870fb584c6f43b6dcd9897bf518c5b14151daf9"}
Oct 30 21:58:38.425170 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:58:38.425070    5510 scope.go:122] "RemoveContainer" containerID="d095ceb11814e488fa545150f012d640c63a9ae22ffff1c2c84057bc59df777b"
Oct 30 21:58:38.435306 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:58:38.435284    5510 scope.go:122] "RemoveContainer" containerID="d095ceb11814e488fa545150f012d640c63a9ae22ffff1c2c84057bc59df777b"
Oct 30 21:58:38.435573 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:58:38.435555    5510 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d095ceb11814e488fa545150f012d640c63a9ae22ffff1c2c84057bc59df777b\": container with ID starting with d095ceb11814e488fa545150f012d640c63a9ae22ffff1c2c84057bc59df777b not found: ID does not exist" containerID="d095ceb11814e488fa545150f012d640c63a9ae22ffff1c2c84057bc59df777b"
Oct 30 21:58:38.435647 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:58:38.435581    5510 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d095ceb11814e488fa545150f012d640c63a9ae22ffff1c2c84057bc59df777b"} err="failed to get container status \"d095ceb11814e488fa545150f012d640c63a9ae22ffff1c2c84057bc59df777b\": rpc error: code = NotFound desc = could not find container \"d095ceb11814e488fa545150f012d640c63a9ae22ffff1c2c84057bc59df777b\": container with ID starting with d095ceb11814e488fa545150f012d640c63a9ae22ffff1c2c84057bc59df777b not found: ID does not exist"
Oct 30 21:58:38.589065 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:58:38.589029    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-nt8mp"]
Oct 30 21:58:38.635676 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:58:38.635629    5510 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-nt8mp"]
Oct 30 21:58:39.551021 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:58:39.550986    5510 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9d6febf4-e9f9-47ce-ac73-9e2a296ad567" path="/var/lib/kubelet/pods/9d6febf4-e9f9-47ce-ac73-9e2a296ad567/volumes"
Oct 30 21:58:46.149456 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:58:46.149417    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:58:46.403153 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:58:46.403065    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:58:46.821865 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:58:46.821832    5510 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-7cdbf6dc44-s67k8"]
Oct 30 21:58:46.822164 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:58:46.822153    5510 cpu_manager.go:414] "containerMap: removing container" podUID="9d6febf4-e9f9-47ce-ac73-9e2a296ad567" containerName="limitador"
Oct 30 21:58:46.822208 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:58:46.822166    5510 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9d6febf4-e9f9-47ce-ac73-9e2a296ad567" containerName="limitador"
Oct 30 21:58:46.822256 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:58:46.822246    5510 memory_manager.go:365] "RemoveStaleState removing state" podUID="9d6febf4-e9f9-47ce-ac73-9e2a296ad567" containerName="limitador"
Oct 30 21:58:46.825079 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:58:46.825041    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-s67k8"
Oct 30 21:58:46.874446 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:58:46.874414    5510 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 30 21:58:46.895096 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:58:46.895064    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tf2rd\" (UniqueName: \"kubernetes.io/projected/03f1ca8f-6c09-4771-aff9-f217eddb47c0-kube-api-access-tf2rd\") pod \"postgres-7cdbf6dc44-s67k8\" (UID: \"03f1ca8f-6c09-4771-aff9-f217eddb47c0\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-s67k8"
Oct 30 21:58:46.895096 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:58:46.895106    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/03f1ca8f-6c09-4771-aff9-f217eddb47c0-data\") pod \"postgres-7cdbf6dc44-s67k8\" (UID: \"03f1ca8f-6c09-4771-aff9-f217eddb47c0\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-s67k8"
Oct 30 21:58:46.920978 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:58:46.920942    5510 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 30 21:58:46.920978 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:58:46.920967    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Oct 30 21:58:46.921204 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:58:46.921057    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-xhsjw\""
Oct 30 21:58:46.921925 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:58:46.921907    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-7cdbf6dc44-s67k8"]
Oct 30 21:58:46.951246 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:58:46.951217    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:58:46.996286 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:58:46.996251    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tf2rd\" (UniqueName: \"kubernetes.io/projected/03f1ca8f-6c09-4771-aff9-f217eddb47c0-kube-api-access-tf2rd\") pod \"postgres-7cdbf6dc44-s67k8\" (UID: \"03f1ca8f-6c09-4771-aff9-f217eddb47c0\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-s67k8"
Oct 30 21:58:46.996286 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:58:46.996289    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/03f1ca8f-6c09-4771-aff9-f217eddb47c0-data\") pod \"postgres-7cdbf6dc44-s67k8\" (UID: \"03f1ca8f-6c09-4771-aff9-f217eddb47c0\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-s67k8"
Oct 30 21:58:46.996633 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:58:46.996617    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/03f1ca8f-6c09-4771-aff9-f217eddb47c0-data\") pod \"postgres-7cdbf6dc44-s67k8\" (UID: \"03f1ca8f-6c09-4771-aff9-f217eddb47c0\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-s67k8"
Oct 30 21:58:47.055211 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:58:47.055175    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tf2rd\" (UniqueName: \"kubernetes.io/projected/03f1ca8f-6c09-4771-aff9-f217eddb47c0-kube-api-access-tf2rd\") pod \"postgres-7cdbf6dc44-s67k8\" (UID: \"03f1ca8f-6c09-4771-aff9-f217eddb47c0\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-s67k8"
Oct 30 21:58:47.137159 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:58:47.137032    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-s67k8"
Oct 30 21:58:47.499081 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:58:47.499002    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-7cdbf6dc44-s67k8"]
Oct 30 21:58:48.464934 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:58:48.464895    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-s67k8" event={"ID":"03f1ca8f-6c09-4771-aff9-f217eddb47c0","Type":"ContainerStarted","Data":"f81085296d8d3b689a23536f9c3d88a7cb96b60cb7dc2d377c18c922b18fbbe5"}
Oct 30 21:58:49.313275 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:58:49.313237    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:58:56.501702 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:58:56.501665    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-s67k8" event={"ID":"03f1ca8f-6c09-4771-aff9-f217eddb47c0","Type":"ContainerStarted","Data":"fc211010c5373afc4fc84aadc4563d8683bc69c80e395225ef99615c975211e1"}
Oct 30 21:58:56.502183 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:58:56.501831    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-s67k8"
Oct 30 21:58:56.612375 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:58:56.612325    5510 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-s67k8" podStartSLOduration=2.585552712 podStartE2EDuration="10.612308786s" totalImagesPullingTime="8.026756074s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:58:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:58:56.611127155 +0000 UTC m=+517.171937160" watchObservedRunningTime="2026-10-30 21:58:56.612308786 +0000 UTC m=+517.173118791"
Oct 30 21:59:02.533747 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:02.533716    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-s67k8"
Oct 30 21:59:08.418363 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:59:08.418324    5510 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6gjbf" containerName="extractor"
Oct 30 21:59:10.246536 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:10.246494    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:59:10.261590 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:10.261553    5510 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-579cf7c6d-lt42v"]
Oct 30 21:59:10.264956 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:10.264935    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-579cf7c6d-lt42v"
Oct 30 21:59:10.292210 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:10.292173    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rc7wl\" (UniqueName: \"kubernetes.io/projected/fe5a6ce7-18af-41b7-b53e-6441258afb73-kube-api-access-rc7wl\") pod \"maas-controller-579cf7c6d-lt42v\" (UID: \"fe5a6ce7-18af-41b7-b53e-6441258afb73\") " pod="opendatahub/maas-controller-579cf7c6d-lt42v"
Oct 30 21:59:10.292402 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:10.292307    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fe5a6ce7-18af-41b7-b53e-6441258afb73-webhook-cert\") pod \"maas-controller-579cf7c6d-lt42v\" (UID: \"fe5a6ce7-18af-41b7-b53e-6441258afb73\") " pod="opendatahub/maas-controller-579cf7c6d-lt42v"
Oct 30 21:59:10.292402 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:10.292344    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fe5a6ce7-18af-41b7-b53e-6441258afb73-metrics-certs\") pod \"maas-controller-579cf7c6d-lt42v\" (UID: \"fe5a6ce7-18af-41b7-b53e-6441258afb73\") " pod="opendatahub/maas-controller-579cf7c6d-lt42v"
Oct 30 21:59:10.315237 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:10.315204    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Oct 30 21:59:10.315431 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:10.315255    5510 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Oct 30 21:59:10.315431 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:10.315267    5510 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Oct 30 21:59:10.315431 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:10.315422    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Oct 30 21:59:10.362125 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:10.362090    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-fzht9\""
Oct 30 21:59:10.362125 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:10.362096    5510 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Oct 30 21:59:10.363464 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:10.363434    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-579cf7c6d-lt42v"]
Oct 30 21:59:10.393145 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:10.393113    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fe5a6ce7-18af-41b7-b53e-6441258afb73-webhook-cert\") pod \"maas-controller-579cf7c6d-lt42v\" (UID: \"fe5a6ce7-18af-41b7-b53e-6441258afb73\") " pod="opendatahub/maas-controller-579cf7c6d-lt42v"
Oct 30 21:59:10.393350 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:10.393167    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fe5a6ce7-18af-41b7-b53e-6441258afb73-metrics-certs\") pod \"maas-controller-579cf7c6d-lt42v\" (UID: \"fe5a6ce7-18af-41b7-b53e-6441258afb73\") " pod="opendatahub/maas-controller-579cf7c6d-lt42v"
Oct 30 21:59:10.393350 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:10.393203    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rc7wl\" (UniqueName: \"kubernetes.io/projected/fe5a6ce7-18af-41b7-b53e-6441258afb73-kube-api-access-rc7wl\") pod \"maas-controller-579cf7c6d-lt42v\" (UID: \"fe5a6ce7-18af-41b7-b53e-6441258afb73\") " pod="opendatahub/maas-controller-579cf7c6d-lt42v"
Oct 30 21:59:10.395762 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:10.395736    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fe5a6ce7-18af-41b7-b53e-6441258afb73-webhook-cert\") pod \"maas-controller-579cf7c6d-lt42v\" (UID: \"fe5a6ce7-18af-41b7-b53e-6441258afb73\") " pod="opendatahub/maas-controller-579cf7c6d-lt42v"
Oct 30 21:59:10.395882 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:10.395739    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fe5a6ce7-18af-41b7-b53e-6441258afb73-metrics-certs\") pod \"maas-controller-579cf7c6d-lt42v\" (UID: \"fe5a6ce7-18af-41b7-b53e-6441258afb73\") " pod="opendatahub/maas-controller-579cf7c6d-lt42v"
Oct 30 21:59:10.454698 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:10.454647    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rc7wl\" (UniqueName: \"kubernetes.io/projected/fe5a6ce7-18af-41b7-b53e-6441258afb73-kube-api-access-rc7wl\") pod \"maas-controller-579cf7c6d-lt42v\" (UID: \"fe5a6ce7-18af-41b7-b53e-6441258afb73\") " pod="opendatahub/maas-controller-579cf7c6d-lt42v"
Oct 30 21:59:10.576985 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:10.576948    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-579cf7c6d-lt42v"
Oct 30 21:59:10.702250 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:10.702212    5510 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-869c494b8f-cck5w"]
Oct 30 21:59:10.705534 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:10.705507    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-869c494b8f-cck5w"
Oct 30 21:59:10.774206 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:10.774173    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-869c494b8f-cck5w"]
Oct 30 21:59:10.796528 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:10.796503    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cc77e063-6bda-4c8b-818c-9ba4ac8fb3db-webhook-cert\") pod \"maas-controller-869c494b8f-cck5w\" (UID: \"cc77e063-6bda-4c8b-818c-9ba4ac8fb3db\") " pod="opendatahub/maas-controller-869c494b8f-cck5w"
Oct 30 21:59:10.796691 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:10.796539    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5c8kh\" (UniqueName: \"kubernetes.io/projected/cc77e063-6bda-4c8b-818c-9ba4ac8fb3db-kube-api-access-5c8kh\") pod \"maas-controller-869c494b8f-cck5w\" (UID: \"cc77e063-6bda-4c8b-818c-9ba4ac8fb3db\") " pod="opendatahub/maas-controller-869c494b8f-cck5w"
Oct 30 21:59:10.796691 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:10.796570    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cc77e063-6bda-4c8b-818c-9ba4ac8fb3db-metrics-certs\") pod \"maas-controller-869c494b8f-cck5w\" (UID: \"cc77e063-6bda-4c8b-818c-9ba4ac8fb3db\") " pod="opendatahub/maas-controller-869c494b8f-cck5w"
Oct 30 21:59:10.897102 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:10.897017    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cc77e063-6bda-4c8b-818c-9ba4ac8fb3db-metrics-certs\") pod \"maas-controller-869c494b8f-cck5w\" (UID: \"cc77e063-6bda-4c8b-818c-9ba4ac8fb3db\") " pod="opendatahub/maas-controller-869c494b8f-cck5w"
Oct 30 21:59:10.897260 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:10.897103    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cc77e063-6bda-4c8b-818c-9ba4ac8fb3db-webhook-cert\") pod \"maas-controller-869c494b8f-cck5w\" (UID: \"cc77e063-6bda-4c8b-818c-9ba4ac8fb3db\") " pod="opendatahub/maas-controller-869c494b8f-cck5w"
Oct 30 21:59:10.897260 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:10.897130    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5c8kh\" (UniqueName: \"kubernetes.io/projected/cc77e063-6bda-4c8b-818c-9ba4ac8fb3db-kube-api-access-5c8kh\") pod \"maas-controller-869c494b8f-cck5w\" (UID: \"cc77e063-6bda-4c8b-818c-9ba4ac8fb3db\") " pod="opendatahub/maas-controller-869c494b8f-cck5w"
Oct 30 21:59:10.899480 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:10.899455    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cc77e063-6bda-4c8b-818c-9ba4ac8fb3db-metrics-certs\") pod \"maas-controller-869c494b8f-cck5w\" (UID: \"cc77e063-6bda-4c8b-818c-9ba4ac8fb3db\") " pod="opendatahub/maas-controller-869c494b8f-cck5w"
Oct 30 21:59:10.899480 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:10.899466    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cc77e063-6bda-4c8b-818c-9ba4ac8fb3db-webhook-cert\") pod \"maas-controller-869c494b8f-cck5w\" (UID: \"cc77e063-6bda-4c8b-818c-9ba4ac8fb3db\") " pod="opendatahub/maas-controller-869c494b8f-cck5w"
Oct 30 21:59:10.963316 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:10.963284    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-579cf7c6d-lt42v"]
Oct 30 21:59:10.966824 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:59:10.966782    5510 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe5a6ce7_18af_41b7_b53e_6441258afb73.slice/crio-d8c6d8dce4e5123280b6541bab0f69d3ad5a7965a983f4e21e6b5a3af3207fa8 WatchSource:0}: Error finding container d8c6d8dce4e5123280b6541bab0f69d3ad5a7965a983f4e21e6b5a3af3207fa8: Status 404 returned error can't find the container with id d8c6d8dce4e5123280b6541bab0f69d3ad5a7965a983f4e21e6b5a3af3207fa8
Oct 30 21:59:10.969381 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:10.969355    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5c8kh\" (UniqueName: \"kubernetes.io/projected/cc77e063-6bda-4c8b-818c-9ba4ac8fb3db-kube-api-access-5c8kh\") pod \"maas-controller-869c494b8f-cck5w\" (UID: \"cc77e063-6bda-4c8b-818c-9ba4ac8fb3db\") " pod="opendatahub/maas-controller-869c494b8f-cck5w"
Oct 30 21:59:11.017247 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:11.017219    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-869c494b8f-cck5w"
Oct 30 21:59:11.420479 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:11.420445    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-869c494b8f-cck5w"]
Oct 30 21:59:11.424918 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:59:11.424894    5510 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc77e063_6bda_4c8b_818c_9ba4ac8fb3db.slice/crio-190983fb99b4d1bbb400a0a6d995204d68fd2c61e80cb81eea46a04652730f49 WatchSource:0}: Error finding container 190983fb99b4d1bbb400a0a6d995204d68fd2c61e80cb81eea46a04652730f49: Status 404 returned error can't find the container with id 190983fb99b4d1bbb400a0a6d995204d68fd2c61e80cb81eea46a04652730f49
Oct 30 21:59:11.558842 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:11.558806    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-869c494b8f-cck5w" event={"ID":"cc77e063-6bda-4c8b-818c-9ba4ac8fb3db","Type":"ContainerStarted","Data":"190983fb99b4d1bbb400a0a6d995204d68fd2c61e80cb81eea46a04652730f49"}
Oct 30 21:59:11.559839 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:11.559816    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-579cf7c6d-lt42v" event={"ID":"fe5a6ce7-18af-41b7-b53e-6441258afb73","Type":"ContainerStarted","Data":"d8c6d8dce4e5123280b6541bab0f69d3ad5a7965a983f4e21e6b5a3af3207fa8"}
Oct 30 21:59:11.631364 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:11.631325    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:59:15.488736 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:15.488685    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:59:15.588203 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:15.588163    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-579cf7c6d-lt42v" event={"ID":"fe5a6ce7-18af-41b7-b53e-6441258afb73","Type":"ContainerStarted","Data":"63dcbc771938ea82e7cfb92325c5298251c2b742c4be23aa0b04c4c0ae38a64f"}
Oct 30 21:59:15.588390 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:15.588232    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-579cf7c6d-lt42v"
Oct 30 21:59:15.589714 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:15.589688    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-869c494b8f-cck5w" event={"ID":"cc77e063-6bda-4c8b-818c-9ba4ac8fb3db","Type":"ContainerStarted","Data":"388b3d6d50b11ecb842131351e150d862312ecd95f9ca3d384ec5aa4bd2c3234"}
Oct 30 21:59:15.589832 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:15.589740    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-869c494b8f-cck5w"
Oct 30 21:59:15.707465 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:15.707407    5510 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/maas-controller-579cf7c6d-lt42v" podStartSLOduration=2.034689556 podStartE2EDuration="5.707393584s" totalImagesPullingTime="3.672704028s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:59:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:59:15.706139228 +0000 UTC m=+536.266949232" watchObservedRunningTime="2026-10-30 21:59:15.707393584 +0000 UTC m=+536.268203582"
Oct 30 21:59:15.793099 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:15.793062    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:59:15.822460 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:15.822410    5510 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/maas-controller-869c494b8f-cck5w" podStartSLOduration=2.4233466630000002 podStartE2EDuration="5.822394304s" totalImagesPullingTime="3.399047641s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:59:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:59:15.822114904 +0000 UTC m=+536.382924910" watchObservedRunningTime="2026-10-30 21:59:15.822394304 +0000 UTC m=+536.383204308"
Oct 30 21:59:20.293488 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:20.293451    5510 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-zxct4"]
Oct 30 21:59:20.298131 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:20.298107    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-zxct4"
Oct 30 21:59:20.354396 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:20.354363    5510 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"usage-logs-tenancy-proxy-ca-bundle\""
Oct 30 21:59:20.386404 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:20.386372    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/24ea2ad0-bd0f-4e1c-806f-b6962996beba-tls-cert\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-zxct4\" (UID: \"24ea2ad0-bd0f-4e1c-806f-b6962996beba\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-zxct4"
Oct 30 21:59:20.386620 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:20.386414    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sfhbh\" (UniqueName: \"kubernetes.io/projected/24ea2ad0-bd0f-4e1c-806f-b6962996beba-kube-api-access-sfhbh\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-zxct4\" (UID: \"24ea2ad0-bd0f-4e1c-806f-b6962996beba\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-zxct4"
Oct 30 21:59:20.386620 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:20.386538    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-source\" (UniqueName: \"kubernetes.io/configmap/24ea2ad0-bd0f-4e1c-806f-b6962996beba-proxy-source\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-zxct4\" (UID: \"24ea2ad0-bd0f-4e1c-806f-b6962996beba\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-zxct4"
Oct 30 21:59:20.386770 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:20.386632    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/24ea2ad0-bd0f-4e1c-806f-b6962996beba-service-ca\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-zxct4\" (UID: \"24ea2ad0-bd0f-4e1c-806f-b6962996beba\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-zxct4"
Oct 30 21:59:20.398404 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:20.398379    5510 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"usage-tenancy-proxy-source\""
Oct 30 21:59:20.398582 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:20.398402    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"usage-logs-tenancy-proxy-tls\""
Oct 30 21:59:20.398582 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:20.398488    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"usage-tenancy-proxy-dockercfg-6stbd\""
Oct 30 21:59:20.399807 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:20.399788    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-zxct4"]
Oct 30 21:59:20.488021 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:20.487989    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-source\" (UniqueName: \"kubernetes.io/configmap/24ea2ad0-bd0f-4e1c-806f-b6962996beba-proxy-source\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-zxct4\" (UID: \"24ea2ad0-bd0f-4e1c-806f-b6962996beba\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-zxct4"
Oct 30 21:59:20.488196 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:20.488036    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/24ea2ad0-bd0f-4e1c-806f-b6962996beba-service-ca\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-zxct4\" (UID: \"24ea2ad0-bd0f-4e1c-806f-b6962996beba\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-zxct4"
Oct 30 21:59:20.488196 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:20.488059    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/24ea2ad0-bd0f-4e1c-806f-b6962996beba-tls-cert\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-zxct4\" (UID: \"24ea2ad0-bd0f-4e1c-806f-b6962996beba\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-zxct4"
Oct 30 21:59:20.488196 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:20.488080    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sfhbh\" (UniqueName: \"kubernetes.io/projected/24ea2ad0-bd0f-4e1c-806f-b6962996beba-kube-api-access-sfhbh\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-zxct4\" (UID: \"24ea2ad0-bd0f-4e1c-806f-b6962996beba\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-zxct4"
Oct 30 21:59:20.488196 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:59:20.488126    5510 configmap.go:193] Couldn't get configMap opendatahub/usage-tenancy-proxy-source: configmap "usage-tenancy-proxy-source" not found
Oct 30 21:59:20.488401 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:59:20.488213    5510 secret.go:189] Couldn't get secret opendatahub/usage-logs-tenancy-proxy-tls: secret "usage-logs-tenancy-proxy-tls" not found
Oct 30 21:59:20.488401 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:59:20.488218    5510 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/24ea2ad0-bd0f-4e1c-806f-b6962996beba-proxy-source podName:24ea2ad0-bd0f-4e1c-806f-b6962996beba nodeName:}" failed. No retries permitted until 2026-10-30 21:59:20.988194848 +0000 UTC m=+541.549004853 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-source" (UniqueName: "kubernetes.io/configmap/24ea2ad0-bd0f-4e1c-806f-b6962996beba-proxy-source") pod "usage-logs-tenancy-proxy-9c5d49bcd-zxct4" (UID: "24ea2ad0-bd0f-4e1c-806f-b6962996beba") : configmap "usage-tenancy-proxy-source" not found
Oct 30 21:59:20.488401 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:59:20.488278    5510 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24ea2ad0-bd0f-4e1c-806f-b6962996beba-tls-cert podName:24ea2ad0-bd0f-4e1c-806f-b6962996beba nodeName:}" failed. No retries permitted until 2026-10-30 21:59:20.988264258 +0000 UTC m=+541.549074241 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-cert" (UniqueName: "kubernetes.io/secret/24ea2ad0-bd0f-4e1c-806f-b6962996beba-tls-cert") pod "usage-logs-tenancy-proxy-9c5d49bcd-zxct4" (UID: "24ea2ad0-bd0f-4e1c-806f-b6962996beba") : secret "usage-logs-tenancy-proxy-tls" not found
Oct 30 21:59:20.488631 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:20.488611    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/24ea2ad0-bd0f-4e1c-806f-b6962996beba-service-ca\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-zxct4\" (UID: \"24ea2ad0-bd0f-4e1c-806f-b6962996beba\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-zxct4"
Oct 30 21:59:20.550133 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:20.550068    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sfhbh\" (UniqueName: \"kubernetes.io/projected/24ea2ad0-bd0f-4e1c-806f-b6962996beba-kube-api-access-sfhbh\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-zxct4\" (UID: \"24ea2ad0-bd0f-4e1c-806f-b6962996beba\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-zxct4"
Oct 30 21:59:20.581832 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:20.581793    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:59:20.842975 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:20.842893    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:59:20.992590 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:20.992554    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-source\" (UniqueName: \"kubernetes.io/configmap/24ea2ad0-bd0f-4e1c-806f-b6962996beba-proxy-source\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-zxct4\" (UID: \"24ea2ad0-bd0f-4e1c-806f-b6962996beba\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-zxct4"
Oct 30 21:59:20.992797 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:20.992612    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/24ea2ad0-bd0f-4e1c-806f-b6962996beba-tls-cert\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-zxct4\" (UID: \"24ea2ad0-bd0f-4e1c-806f-b6962996beba\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-zxct4"
Oct 30 21:59:20.993426 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:20.993387    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-source\" (UniqueName: \"kubernetes.io/configmap/24ea2ad0-bd0f-4e1c-806f-b6962996beba-proxy-source\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-zxct4\" (UID: \"24ea2ad0-bd0f-4e1c-806f-b6962996beba\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-zxct4"
Oct 30 21:59:20.994988 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:20.994969    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/24ea2ad0-bd0f-4e1c-806f-b6962996beba-tls-cert\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-zxct4\" (UID: \"24ea2ad0-bd0f-4e1c-806f-b6962996beba\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-zxct4"
Oct 30 21:59:21.210176 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:21.210093    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-zxct4"
Oct 30 21:59:21.353639 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:21.353607    5510 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-f68tv"]
Oct 30 21:59:21.358142 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:21.358123    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-f68tv"
Oct 30 21:59:21.462760 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:21.462674    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-f68tv"]
Oct 30 21:59:21.495999 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:21.495961    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-455sr\" (UniqueName: \"kubernetes.io/projected/4407af9b-8cf7-4552-9dfc-8324d9832655-kube-api-access-455sr\") pod \"istiod-openshift-gateway-7ddd9cfdc8-f68tv\" (UID: \"4407af9b-8cf7-4552-9dfc-8324d9832655\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-f68tv"
Oct 30 21:59:21.495999 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:21.495998    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4407af9b-8cf7-4552-9dfc-8324d9832655-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-f68tv\" (UID: \"4407af9b-8cf7-4552-9dfc-8324d9832655\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-f68tv"
Oct 30 21:59:21.496217 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:21.496016    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/4407af9b-8cf7-4552-9dfc-8324d9832655-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-f68tv\" (UID: \"4407af9b-8cf7-4552-9dfc-8324d9832655\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-f68tv"
Oct 30 21:59:21.496217 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:21.496133    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/4407af9b-8cf7-4552-9dfc-8324d9832655-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-f68tv\" (UID: \"4407af9b-8cf7-4552-9dfc-8324d9832655\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-f68tv"
Oct 30 21:59:21.496217 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:21.496158    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/4407af9b-8cf7-4552-9dfc-8324d9832655-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-f68tv\" (UID: \"4407af9b-8cf7-4552-9dfc-8324d9832655\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-f68tv"
Oct 30 21:59:21.496217 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:21.496178    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/4407af9b-8cf7-4552-9dfc-8324d9832655-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-f68tv\" (UID: \"4407af9b-8cf7-4552-9dfc-8324d9832655\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-f68tv"
Oct 30 21:59:21.496408 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:21.496338    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/4407af9b-8cf7-4552-9dfc-8324d9832655-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-f68tv\" (UID: \"4407af9b-8cf7-4552-9dfc-8324d9832655\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-f68tv"
Oct 30 21:59:21.578256 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:21.578225    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-zxct4"]
Oct 30 21:59:21.581805 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:59:21.581775    5510 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24ea2ad0_bd0f_4e1c_806f_b6962996beba.slice/crio-fb48e444496a25989224653ed2a375d50a86d2bd5e0d2a552716edd4465b0c29 WatchSource:0}: Error finding container fb48e444496a25989224653ed2a375d50a86d2bd5e0d2a552716edd4465b0c29: Status 404 returned error can't find the container with id fb48e444496a25989224653ed2a375d50a86d2bd5e0d2a552716edd4465b0c29
Oct 30 21:59:21.597722 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:21.597692    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/4407af9b-8cf7-4552-9dfc-8324d9832655-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-f68tv\" (UID: \"4407af9b-8cf7-4552-9dfc-8324d9832655\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-f68tv"
Oct 30 21:59:21.597832 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:21.597728    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/4407af9b-8cf7-4552-9dfc-8324d9832655-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-f68tv\" (UID: \"4407af9b-8cf7-4552-9dfc-8324d9832655\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-f68tv"
Oct 30 21:59:21.597832 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:21.597759    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/4407af9b-8cf7-4552-9dfc-8324d9832655-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-f68tv\" (UID: \"4407af9b-8cf7-4552-9dfc-8324d9832655\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-f68tv"
Oct 30 21:59:21.597946 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:21.597843    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/4407af9b-8cf7-4552-9dfc-8324d9832655-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-f68tv\" (UID: \"4407af9b-8cf7-4552-9dfc-8324d9832655\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-f68tv"
Oct 30 21:59:21.597946 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:21.597902    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-455sr\" (UniqueName: \"kubernetes.io/projected/4407af9b-8cf7-4552-9dfc-8324d9832655-kube-api-access-455sr\") pod \"istiod-openshift-gateway-7ddd9cfdc8-f68tv\" (UID: \"4407af9b-8cf7-4552-9dfc-8324d9832655\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-f68tv"
Oct 30 21:59:21.597946 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:21.597932    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4407af9b-8cf7-4552-9dfc-8324d9832655-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-f68tv\" (UID: \"4407af9b-8cf7-4552-9dfc-8324d9832655\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-f68tv"
Oct 30 21:59:21.598088 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:21.597980    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/4407af9b-8cf7-4552-9dfc-8324d9832655-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-f68tv\" (UID: \"4407af9b-8cf7-4552-9dfc-8324d9832655\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-f68tv"
Oct 30 21:59:21.598514 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:21.598491    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/4407af9b-8cf7-4552-9dfc-8324d9832655-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-f68tv\" (UID: \"4407af9b-8cf7-4552-9dfc-8324d9832655\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-f68tv"
Oct 30 21:59:21.600143 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:21.600119    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/4407af9b-8cf7-4552-9dfc-8324d9832655-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-f68tv\" (UID: \"4407af9b-8cf7-4552-9dfc-8324d9832655\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-f68tv"
Oct 30 21:59:21.600317 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:21.600291    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/4407af9b-8cf7-4552-9dfc-8324d9832655-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-f68tv\" (UID: \"4407af9b-8cf7-4552-9dfc-8324d9832655\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-f68tv"
Oct 30 21:59:21.600487 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:21.600469    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/4407af9b-8cf7-4552-9dfc-8324d9832655-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-f68tv\" (UID: \"4407af9b-8cf7-4552-9dfc-8324d9832655\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-f68tv"
Oct 30 21:59:21.600544 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:21.600485    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/4407af9b-8cf7-4552-9dfc-8324d9832655-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-f68tv\" (UID: \"4407af9b-8cf7-4552-9dfc-8324d9832655\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-f68tv"
Oct 30 21:59:21.613980 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:21.613954    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-zxct4" event={"ID":"24ea2ad0-bd0f-4e1c-806f-b6962996beba","Type":"ContainerStarted","Data":"fb48e444496a25989224653ed2a375d50a86d2bd5e0d2a552716edd4465b0c29"}
Oct 30 21:59:21.656284 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:21.656254    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4407af9b-8cf7-4552-9dfc-8324d9832655-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-f68tv\" (UID: \"4407af9b-8cf7-4552-9dfc-8324d9832655\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-f68tv"
Oct 30 21:59:21.656405 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:21.656383    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-455sr\" (UniqueName: \"kubernetes.io/projected/4407af9b-8cf7-4552-9dfc-8324d9832655-kube-api-access-455sr\") pod \"istiod-openshift-gateway-7ddd9cfdc8-f68tv\" (UID: \"4407af9b-8cf7-4552-9dfc-8324d9832655\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-f68tv"
Oct 30 21:59:21.670177 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:21.670155    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-f68tv"
Oct 30 21:59:22.084576 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:22.084545    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-f68tv"]
Oct 30 21:59:22.088204 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:59:22.088174    5510 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4407af9b_8cf7_4552_9dfc_8324d9832655.slice/crio-d81f15e85249eadce963c305403b8ea501b1d852f49012e6aa4238d90bb37cb9 WatchSource:0}: Error finding container d81f15e85249eadce963c305403b8ea501b1d852f49012e6aa4238d90bb37cb9: Status 404 returned error can't find the container with id d81f15e85249eadce963c305403b8ea501b1d852f49012e6aa4238d90bb37cb9
Oct 30 21:59:22.090258 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:22.090228    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:59:22.090331 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:22.090290    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:59:22.625552 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:22.625514    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-f68tv" event={"ID":"4407af9b-8cf7-4552-9dfc-8324d9832655","Type":"ContainerStarted","Data":"63bdd7c736f92dca64eb88441088f5d389f31638f5b377962b6df007730106be"}
Oct 30 21:59:22.625952 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:22.625608    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-f68tv"
Oct 30 21:59:22.625952 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:22.625628    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-f68tv" event={"ID":"4407af9b-8cf7-4552-9dfc-8324d9832655","Type":"ContainerStarted","Data":"d81f15e85249eadce963c305403b8ea501b1d852f49012e6aa4238d90bb37cb9"}
Oct 30 21:59:22.751467 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:22.751402    5510 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-f68tv" podStartSLOduration=1.7513898540000001 podStartE2EDuration="1.751389854s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:59:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:59:22.751053494 +0000 UTC m=+543.311863498" watchObservedRunningTime="2026-10-30 21:59:22.751389854 +0000 UTC m=+543.312199859"
Oct 30 21:59:23.631753 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:23.631714    5510 patch_prober.go:29] interesting pod/istiod-openshift-gateway-7ddd9cfdc8-f68tv container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 21:59:23.632128 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:23.631769    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-f68tv" podUID="4407af9b-8cf7-4552-9dfc-8324d9832655" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 21:59:24.647398 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:24.647363    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:59:24.887366 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:24.887330    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:59:25.092180 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:25.092140    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:59:25.281955 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:25.281922    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:59:25.484164 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:25.484080    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:59:25.693596 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:25.693532    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:59:26.596916 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:26.596887    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-579cf7c6d-lt42v"
Oct 30 21:59:26.597111 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:26.596974    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-869c494b8f-cck5w"
Oct 30 21:59:26.631424 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:26.631389    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-f68tv"
Oct 30 21:59:26.825066 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:26.824987    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:59:26.880502 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:26.880423    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-ll8zm"]
Oct 30 21:59:26.880726 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:26.880666    5510 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-ll8zm" podUID="e3e9d86d-58cf-41b5-9932-03fa0ef94828" containerName="discovery" containerID="cri-o://98914b5bd78a64ddc66a74e2b90ebaec15719f75ce8a322dbcfd0fafecd00b7b" gracePeriod=30
Oct 30 21:59:27.001607 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:27.001578    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-579cf7c6d-lt42v"]
Oct 30 21:59:27.002007 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:27.001801    5510 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/maas-controller-579cf7c6d-lt42v" podUID="fe5a6ce7-18af-41b7-b53e-6441258afb73" containerName="manager" containerID="cri-o://63dcbc771938ea82e7cfb92325c5298251c2b742c4be23aa0b04c4c0ae38a64f" gracePeriod=30
Oct 30 21:59:27.276476 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:27.276445    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-ll8zm"
Oct 30 21:59:27.301783 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:27.301753    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-579cf7c6d-lt42v"
Oct 30 21:59:27.355346 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:27.355306    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e3e9d86d-58cf-41b5-9932-03fa0ef94828-local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e3e9d86d-58cf-41b5-9932-03fa0ef94828-local-certs\") pod \"e3e9d86d-58cf-41b5-9932-03fa0ef94828\" (UID: \"e3e9d86d-58cf-41b5-9932-03fa0ef94828\") "
Oct 30 21:59:27.355531 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:27.355367    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e3e9d86d-58cf-41b5-9932-03fa0ef94828-cacerts\" (UniqueName: \"kubernetes.io/secret/e3e9d86d-58cf-41b5-9932-03fa0ef94828-cacerts\") pod \"e3e9d86d-58cf-41b5-9932-03fa0ef94828\" (UID: \"e3e9d86d-58cf-41b5-9932-03fa0ef94828\") "
Oct 30 21:59:27.355531 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:27.355404    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e3e9d86d-58cf-41b5-9932-03fa0ef94828-kube-api-access-qmd9v\" (UniqueName: \"kubernetes.io/projected/e3e9d86d-58cf-41b5-9932-03fa0ef94828-kube-api-access-qmd9v\") pod \"e3e9d86d-58cf-41b5-9932-03fa0ef94828\" (UID: \"e3e9d86d-58cf-41b5-9932-03fa0ef94828\") "
Oct 30 21:59:27.355531 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:27.355491    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e3e9d86d-58cf-41b5-9932-03fa0ef94828-istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e3e9d86d-58cf-41b5-9932-03fa0ef94828-istio-csr-dns-cert\") pod \"e3e9d86d-58cf-41b5-9932-03fa0ef94828\" (UID: \"e3e9d86d-58cf-41b5-9932-03fa0ef94828\") "
Oct 30 21:59:27.355745 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:27.355546    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e3e9d86d-58cf-41b5-9932-03fa0ef94828-istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e3e9d86d-58cf-41b5-9932-03fa0ef94828-istio-kubeconfig\") pod \"e3e9d86d-58cf-41b5-9932-03fa0ef94828\" (UID: \"e3e9d86d-58cf-41b5-9932-03fa0ef94828\") "
Oct 30 21:59:27.355745 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:27.355596    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e3e9d86d-58cf-41b5-9932-03fa0ef94828-istio-token\" (UniqueName: \"kubernetes.io/projected/e3e9d86d-58cf-41b5-9932-03fa0ef94828-istio-token\") pod \"e3e9d86d-58cf-41b5-9932-03fa0ef94828\" (UID: \"e3e9d86d-58cf-41b5-9932-03fa0ef94828\") "
Oct 30 21:59:27.355745 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:27.355647    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e3e9d86d-58cf-41b5-9932-03fa0ef94828-istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e3e9d86d-58cf-41b5-9932-03fa0ef94828-istio-csr-ca-configmap\") pod \"e3e9d86d-58cf-41b5-9932-03fa0ef94828\" (UID: \"e3e9d86d-58cf-41b5-9932-03fa0ef94828\") "
Oct 30 21:59:27.356296 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:27.356267    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e3e9d86d-58cf-41b5-9932-03fa0ef94828-istio-csr-ca-configmap" pod "e3e9d86d-58cf-41b5-9932-03fa0ef94828" (UID: "e3e9d86d-58cf-41b5-9932-03fa0ef94828"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:59:27.358568 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:27.358513    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e3e9d86d-58cf-41b5-9932-03fa0ef94828-local-certs" pod "e3e9d86d-58cf-41b5-9932-03fa0ef94828" (UID: "e3e9d86d-58cf-41b5-9932-03fa0ef94828"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:59:27.358568 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:27.358520    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e3e9d86d-58cf-41b5-9932-03fa0ef94828-cacerts" pod "e3e9d86d-58cf-41b5-9932-03fa0ef94828" (UID: "e3e9d86d-58cf-41b5-9932-03fa0ef94828"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:59:27.359027 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:27.358997    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e3e9d86d-58cf-41b5-9932-03fa0ef94828-istio-token" pod "e3e9d86d-58cf-41b5-9932-03fa0ef94828" (UID: "e3e9d86d-58cf-41b5-9932-03fa0ef94828"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:59:27.359414 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:27.359387    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e3e9d86d-58cf-41b5-9932-03fa0ef94828-istio-kubeconfig" pod "e3e9d86d-58cf-41b5-9932-03fa0ef94828" (UID: "e3e9d86d-58cf-41b5-9932-03fa0ef94828"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:59:27.359914 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:27.359878    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e3e9d86d-58cf-41b5-9932-03fa0ef94828-istio-csr-dns-cert" pod "e3e9d86d-58cf-41b5-9932-03fa0ef94828" (UID: "e3e9d86d-58cf-41b5-9932-03fa0ef94828"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:59:27.361647 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:27.361628    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e3e9d86d-58cf-41b5-9932-03fa0ef94828-kube-api-access-qmd9v" pod "e3e9d86d-58cf-41b5-9932-03fa0ef94828" (UID: "e3e9d86d-58cf-41b5-9932-03fa0ef94828"). InnerVolumeSpecName "kube-api-access-qmd9v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:59:27.456794 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:27.456702    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/fe5a6ce7-18af-41b7-b53e-6441258afb73-metrics-certs\" (UniqueName: \"kubernetes.io/secret/fe5a6ce7-18af-41b7-b53e-6441258afb73-metrics-certs\") pod \"fe5a6ce7-18af-41b7-b53e-6441258afb73\" (UID: \"fe5a6ce7-18af-41b7-b53e-6441258afb73\") "
Oct 30 21:59:27.456974 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:27.456824    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fe5a6ce7-18af-41b7-b53e-6441258afb73-kube-api-access-rc7wl\" (UniqueName: \"kubernetes.io/projected/fe5a6ce7-18af-41b7-b53e-6441258afb73-kube-api-access-rc7wl\") pod \"fe5a6ce7-18af-41b7-b53e-6441258afb73\" (UID: \"fe5a6ce7-18af-41b7-b53e-6441258afb73\") "
Oct 30 21:59:27.456974 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:27.456863    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/fe5a6ce7-18af-41b7-b53e-6441258afb73-webhook-cert\" (UniqueName: \"kubernetes.io/secret/fe5a6ce7-18af-41b7-b53e-6441258afb73-webhook-cert\") pod \"fe5a6ce7-18af-41b7-b53e-6441258afb73\" (UID: \"fe5a6ce7-18af-41b7-b53e-6441258afb73\") "
Oct 30 21:59:27.457201 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:27.457173    5510 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e3e9d86d-58cf-41b5-9932-03fa0ef94828-istio-kubeconfig\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:59:27.457201 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:27.457193    5510 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e3e9d86d-58cf-41b5-9932-03fa0ef94828-istio-token\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:59:27.457201 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:27.457206    5510 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e3e9d86d-58cf-41b5-9932-03fa0ef94828-istio-csr-ca-configmap\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:59:27.457451 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:27.457220    5510 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e3e9d86d-58cf-41b5-9932-03fa0ef94828-local-certs\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:59:27.457451 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:27.457239    5510 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/e3e9d86d-58cf-41b5-9932-03fa0ef94828-cacerts\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:59:27.457451 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:27.457258    5510 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qmd9v\" (UniqueName: \"kubernetes.io/projected/e3e9d86d-58cf-41b5-9932-03fa0ef94828-kube-api-access-qmd9v\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:59:27.457451 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:27.457270    5510 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e3e9d86d-58cf-41b5-9932-03fa0ef94828-istio-csr-dns-cert\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:59:27.459331 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:27.459302    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fe5a6ce7-18af-41b7-b53e-6441258afb73-webhook-cert" pod "fe5a6ce7-18af-41b7-b53e-6441258afb73" (UID: "fe5a6ce7-18af-41b7-b53e-6441258afb73"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:59:27.459430 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:27.459352    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fe5a6ce7-18af-41b7-b53e-6441258afb73-metrics-certs" pod "fe5a6ce7-18af-41b7-b53e-6441258afb73" (UID: "fe5a6ce7-18af-41b7-b53e-6441258afb73"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:59:27.459474 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:27.459452    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fe5a6ce7-18af-41b7-b53e-6441258afb73-kube-api-access-rc7wl" pod "fe5a6ce7-18af-41b7-b53e-6441258afb73" (UID: "fe5a6ce7-18af-41b7-b53e-6441258afb73"). InnerVolumeSpecName "kube-api-access-rc7wl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:59:27.561207 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:27.561126    5510 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fe5a6ce7-18af-41b7-b53e-6441258afb73-metrics-certs\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:59:27.561207 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:27.561160    5510 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rc7wl\" (UniqueName: \"kubernetes.io/projected/fe5a6ce7-18af-41b7-b53e-6441258afb73-kube-api-access-rc7wl\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:59:27.561207 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:27.561177    5510 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fe5a6ce7-18af-41b7-b53e-6441258afb73-webhook-cert\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:59:27.653217 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:27.652595    5510 generic.go:356] "Generic (PLEG): container finished" podID="fe5a6ce7-18af-41b7-b53e-6441258afb73" containerID="63dcbc771938ea82e7cfb92325c5298251c2b742c4be23aa0b04c4c0ae38a64f" exitCode=0
Oct 30 21:59:27.653217 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:27.652683    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-579cf7c6d-lt42v"
Oct 30 21:59:27.653217 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:27.652740    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-579cf7c6d-lt42v" event={"ID":"fe5a6ce7-18af-41b7-b53e-6441258afb73","Type":"ContainerDied","Data":"63dcbc771938ea82e7cfb92325c5298251c2b742c4be23aa0b04c4c0ae38a64f"}
Oct 30 21:59:27.653217 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:27.652792    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-579cf7c6d-lt42v" event={"ID":"fe5a6ce7-18af-41b7-b53e-6441258afb73","Type":"ContainerDied","Data":"d8c6d8dce4e5123280b6541bab0f69d3ad5a7965a983f4e21e6b5a3af3207fa8"}
Oct 30 21:59:27.653217 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:27.652824    5510 scope.go:122] "RemoveContainer" containerID="63dcbc771938ea82e7cfb92325c5298251c2b742c4be23aa0b04c4c0ae38a64f"
Oct 30 21:59:27.655022 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:27.654995    5510 generic.go:356] "Generic (PLEG): container finished" podID="e3e9d86d-58cf-41b5-9932-03fa0ef94828" containerID="98914b5bd78a64ddc66a74e2b90ebaec15719f75ce8a322dbcfd0fafecd00b7b" exitCode=0
Oct 30 21:59:27.655134 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:27.655107    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-ll8zm" event={"ID":"e3e9d86d-58cf-41b5-9932-03fa0ef94828","Type":"ContainerDied","Data":"98914b5bd78a64ddc66a74e2b90ebaec15719f75ce8a322dbcfd0fafecd00b7b"}
Oct 30 21:59:27.655204 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:27.655152    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-ll8zm" event={"ID":"e3e9d86d-58cf-41b5-9932-03fa0ef94828","Type":"ContainerDied","Data":"701aea56080d745b74e01a00b5312651d130bb6066bcb402693b207c0aa5bf07"}
Oct 30 21:59:27.655265 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:27.655229    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-ll8zm"
Oct 30 21:59:27.667772 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:27.667747    5510 scope.go:122] "RemoveContainer" containerID="63dcbc771938ea82e7cfb92325c5298251c2b742c4be23aa0b04c4c0ae38a64f"
Oct 30 21:59:27.668353 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:59:27.668209    5510 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"63dcbc771938ea82e7cfb92325c5298251c2b742c4be23aa0b04c4c0ae38a64f\": container with ID starting with 63dcbc771938ea82e7cfb92325c5298251c2b742c4be23aa0b04c4c0ae38a64f not found: ID does not exist" containerID="63dcbc771938ea82e7cfb92325c5298251c2b742c4be23aa0b04c4c0ae38a64f"
Oct 30 21:59:27.668353 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:27.668249    5510 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"63dcbc771938ea82e7cfb92325c5298251c2b742c4be23aa0b04c4c0ae38a64f"} err="failed to get container status \"63dcbc771938ea82e7cfb92325c5298251c2b742c4be23aa0b04c4c0ae38a64f\": rpc error: code = NotFound desc = could not find container \"63dcbc771938ea82e7cfb92325c5298251c2b742c4be23aa0b04c4c0ae38a64f\": container with ID starting with 63dcbc771938ea82e7cfb92325c5298251c2b742c4be23aa0b04c4c0ae38a64f not found: ID does not exist"
Oct 30 21:59:27.668353 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:27.668273    5510 scope.go:122] "RemoveContainer" containerID="98914b5bd78a64ddc66a74e2b90ebaec15719f75ce8a322dbcfd0fafecd00b7b"
Oct 30 21:59:27.678486 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:27.678464    5510 scope.go:122] "RemoveContainer" containerID="98914b5bd78a64ddc66a74e2b90ebaec15719f75ce8a322dbcfd0fafecd00b7b"
Oct 30 21:59:27.678822 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:59:27.678749    5510 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"98914b5bd78a64ddc66a74e2b90ebaec15719f75ce8a322dbcfd0fafecd00b7b\": container with ID starting with 98914b5bd78a64ddc66a74e2b90ebaec15719f75ce8a322dbcfd0fafecd00b7b not found: ID does not exist" containerID="98914b5bd78a64ddc66a74e2b90ebaec15719f75ce8a322dbcfd0fafecd00b7b"
Oct 30 21:59:27.678822 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:27.678783    5510 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"98914b5bd78a64ddc66a74e2b90ebaec15719f75ce8a322dbcfd0fafecd00b7b"} err="failed to get container status \"98914b5bd78a64ddc66a74e2b90ebaec15719f75ce8a322dbcfd0fafecd00b7b\": rpc error: code = NotFound desc = could not find container \"98914b5bd78a64ddc66a74e2b90ebaec15719f75ce8a322dbcfd0fafecd00b7b\": container with ID starting with 98914b5bd78a64ddc66a74e2b90ebaec15719f75ce8a322dbcfd0fafecd00b7b not found: ID does not exist"
Oct 30 21:59:27.809007 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:27.808966    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-ll8zm"]
Oct 30 21:59:27.815134 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:27.815103    5510 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-ll8zm"]
Oct 30 21:59:27.919715 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:27.919644    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-579cf7c6d-lt42v"]
Oct 30 21:59:27.933844 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:27.928337    5510 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-579cf7c6d-lt42v"]
Oct 30 21:59:29.552915 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:29.552880    5510 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e3e9d86d-58cf-41b5-9932-03fa0ef94828" path="/var/lib/kubelet/pods/e3e9d86d-58cf-41b5-9932-03fa0ef94828/volumes"
Oct 30 21:59:29.553401 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:29.553300    5510 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fe5a6ce7-18af-41b7-b53e-6441258afb73" path="/var/lib/kubelet/pods/fe5a6ce7-18af-41b7-b53e-6441258afb73/volumes"
Oct 30 21:59:31.861826 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:31.861785    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:59:32.102040 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:32.101998    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:59:32.285261 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:32.285220    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:59:32.479728 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:32.479690    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:59:32.670639 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:32.670552    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:59:32.856497 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:32.856463    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:59:33.048260 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:33.048220    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:59:35.694201 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:35.694155    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-zxct4" event={"ID":"24ea2ad0-bd0f-4e1c-806f-b6962996beba","Type":"ContainerStarted","Data":"48daba4cc6e7a3331e9fb444cc10d9560eba855ac38729665789fe23e1579b21"}
Oct 30 21:59:35.694580 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:35.694230    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-zxct4"
Oct 30 21:59:35.807454 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:35.807400    5510 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-zxct4" podStartSLOduration=2.242363939 podStartE2EDuration="15.807385985s" totalImagesPullingTime="13.565022046s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:59:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:59:35.806327708 +0000 UTC m=+556.367137712" watchObservedRunningTime="2026-10-30 21:59:35.807385985 +0000 UTC m=+556.368196052"
Oct 30 21:59:38.419129 ip-10-0-136-210 kubenswrapper[5510]: E1030 21:59:38.419097    5510 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6gjbf" containerName="extractor"
Oct 30 21:59:46.703793 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:46.703760    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-zxct4"
Oct 30 21:59:48.971054 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:48.971009    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:59:49.217023 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:49.216980    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:59:51.109212 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:51.109175    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:59:51.336352 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:51.336302    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:59:51.604899 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:51.604852    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 21:59:52.171924 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:52.171890    5510 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-6d669c7fcd-jhbb2"]
Oct 30 21:59:52.172318 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:52.172225    5510 cpu_manager.go:414] "containerMap: removing container" podUID="e3e9d86d-58cf-41b5-9932-03fa0ef94828" containerName="discovery"
Oct 30 21:59:52.172318 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:52.172237    5510 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e3e9d86d-58cf-41b5-9932-03fa0ef94828" containerName="discovery"
Oct 30 21:59:52.172318 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:52.172259    5510 cpu_manager.go:414] "containerMap: removing container" podUID="fe5a6ce7-18af-41b7-b53e-6441258afb73" containerName="manager"
Oct 30 21:59:52.172318 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:52.172265    5510 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fe5a6ce7-18af-41b7-b53e-6441258afb73" containerName="manager"
Oct 30 21:59:52.172318 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:52.172313    5510 memory_manager.go:365] "RemoveStaleState removing state" podUID="fe5a6ce7-18af-41b7-b53e-6441258afb73" containerName="manager"
Oct 30 21:59:52.172478 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:52.172325    5510 memory_manager.go:365] "RemoveStaleState removing state" podUID="e3e9d86d-58cf-41b5-9932-03fa0ef94828" containerName="discovery"
Oct 30 21:59:52.178555 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:52.178533    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-jhbb2"
Oct 30 21:59:52.274858 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:52.274820    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-jglqk\""
Oct 30 21:59:52.275035 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:52.274868    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Oct 30 21:59:52.275035 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:52.274968    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Oct 30 21:59:52.275967 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:52.275949    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6d669c7fcd-jhbb2"]
Oct 30 21:59:52.285177 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:52.285156    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1bf676b2-7a2c-4ea4-b2fa-e8a6c17d9782-openshift-service-ca\") pod \"maas-api-6d669c7fcd-jhbb2\" (UID: \"1bf676b2-7a2c-4ea4-b2fa-e8a6c17d9782\") " pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-jhbb2"
Oct 30 21:59:52.285234 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:52.285194    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1bf676b2-7a2c-4ea4-b2fa-e8a6c17d9782-metrics-certs\") pod \"maas-api-6d669c7fcd-jhbb2\" (UID: \"1bf676b2-7a2c-4ea4-b2fa-e8a6c17d9782\") " pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-jhbb2"
Oct 30 21:59:52.285334 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:52.285315    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1bf676b2-7a2c-4ea4-b2fa-e8a6c17d9782-maas-api-tls\") pod \"maas-api-6d669c7fcd-jhbb2\" (UID: \"1bf676b2-7a2c-4ea4-b2fa-e8a6c17d9782\") " pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-jhbb2"
Oct 30 21:59:52.285373 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:52.285349    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7xkrk\" (UniqueName: \"kubernetes.io/projected/1bf676b2-7a2c-4ea4-b2fa-e8a6c17d9782-kube-api-access-7xkrk\") pod \"maas-api-6d669c7fcd-jhbb2\" (UID: \"1bf676b2-7a2c-4ea4-b2fa-e8a6c17d9782\") " pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-jhbb2"
Oct 30 21:59:52.386094 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:52.386048    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7xkrk\" (UniqueName: \"kubernetes.io/projected/1bf676b2-7a2c-4ea4-b2fa-e8a6c17d9782-kube-api-access-7xkrk\") pod \"maas-api-6d669c7fcd-jhbb2\" (UID: \"1bf676b2-7a2c-4ea4-b2fa-e8a6c17d9782\") " pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-jhbb2"
Oct 30 21:59:52.386094 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:52.386099    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1bf676b2-7a2c-4ea4-b2fa-e8a6c17d9782-openshift-service-ca\") pod \"maas-api-6d669c7fcd-jhbb2\" (UID: \"1bf676b2-7a2c-4ea4-b2fa-e8a6c17d9782\") " pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-jhbb2"
Oct 30 21:59:52.386318 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:52.386123    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1bf676b2-7a2c-4ea4-b2fa-e8a6c17d9782-metrics-certs\") pod \"maas-api-6d669c7fcd-jhbb2\" (UID: \"1bf676b2-7a2c-4ea4-b2fa-e8a6c17d9782\") " pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-jhbb2"
Oct 30 21:59:52.386318 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:52.386233    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1bf676b2-7a2c-4ea4-b2fa-e8a6c17d9782-maas-api-tls\") pod \"maas-api-6d669c7fcd-jhbb2\" (UID: \"1bf676b2-7a2c-4ea4-b2fa-e8a6c17d9782\") " pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-jhbb2"
Oct 30 21:59:52.386924 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:52.386902    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1bf676b2-7a2c-4ea4-b2fa-e8a6c17d9782-openshift-service-ca\") pod \"maas-api-6d669c7fcd-jhbb2\" (UID: \"1bf676b2-7a2c-4ea4-b2fa-e8a6c17d9782\") " pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-jhbb2"
Oct 30 21:59:52.388690 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:52.388646    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1bf676b2-7a2c-4ea4-b2fa-e8a6c17d9782-maas-api-tls\") pod \"maas-api-6d669c7fcd-jhbb2\" (UID: \"1bf676b2-7a2c-4ea4-b2fa-e8a6c17d9782\") " pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-jhbb2"
Oct 30 21:59:52.388802 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:52.388720    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1bf676b2-7a2c-4ea4-b2fa-e8a6c17d9782-metrics-certs\") pod \"maas-api-6d669c7fcd-jhbb2\" (UID: \"1bf676b2-7a2c-4ea4-b2fa-e8a6c17d9782\") " pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-jhbb2"
Oct 30 21:59:52.445027 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:52.444959    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7xkrk\" (UniqueName: \"kubernetes.io/projected/1bf676b2-7a2c-4ea4-b2fa-e8a6c17d9782-kube-api-access-7xkrk\") pod \"maas-api-6d669c7fcd-jhbb2\" (UID: \"1bf676b2-7a2c-4ea4-b2fa-e8a6c17d9782\") " pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-jhbb2"
Oct 30 21:59:52.491168 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:52.491135    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-jhbb2"
Oct 30 21:59:52.858762 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:52.858730    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6d669c7fcd-jhbb2"]
Oct 30 21:59:52.861847 ip-10-0-136-210 kubenswrapper[5510]: W1030 21:59:52.861804    5510 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1bf676b2_7a2c_4ea4_b2fa_e8a6c17d9782.slice/crio-9ae6364a93394a2e8667b331d0cbb82cc992addcc1d7aab836e4b458633e7c70 WatchSource:0}: Error finding container 9ae6364a93394a2e8667b331d0cbb82cc992addcc1d7aab836e4b458633e7c70: Status 404 returned error can't find the container with id 9ae6364a93394a2e8667b331d0cbb82cc992addcc1d7aab836e4b458633e7c70
Oct 30 21:59:53.770201 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:53.770162    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-jhbb2" event={"ID":"1bf676b2-7a2c-4ea4-b2fa-e8a6c17d9782","Type":"ContainerStarted","Data":"9ae6364a93394a2e8667b331d0cbb82cc992addcc1d7aab836e4b458633e7c70"}
Oct 30 21:59:55.781163 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:55.781126    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-jhbb2" event={"ID":"1bf676b2-7a2c-4ea4-b2fa-e8a6c17d9782","Type":"ContainerStarted","Data":"909aab9125898f8ebd6c3f624bb166b92040e6e1198f8af9eab8202670a01f1c"}
Oct 30 21:59:55.781545 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:55.781240    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-jhbb2"
Oct 30 21:59:55.895041 ip-10-0-136-210 kubenswrapper[5510]: I1030 21:59:55.894984    5510 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-jhbb2" podStartSLOduration=1.899715238 podStartE2EDuration="3.894969463s" totalImagesPullingTime="1.995254225s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:59:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:59:55.894393034 +0000 UTC m=+576.455203059" watchObservedRunningTime="2026-10-30 21:59:55.894969463 +0000 UTC m=+576.455779459"
Oct 30 22:00:01.791500 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:00:01.791472    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-jhbb2"
Oct 30 22:00:02.409965 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:00:02.409933    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:00:02.751140 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:00:02.751051    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:00:02.886428 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:00:02.886390    5510 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-7f4ccfcf79-tlkq8"]
Oct 30 22:00:02.904942 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:00:02.904915    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7f4ccfcf79-tlkq8"
Oct 30 22:00:02.951462 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:00:02.951433    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-7f4ccfcf79-tlkq8"]
Oct 30 22:00:02.986397 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:00:02.986362    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/91c8233a-1602-49b1-8cd2-51a4c47de80a-openshift-service-ca\") pod \"maas-api-7f4ccfcf79-tlkq8\" (UID: \"91c8233a-1602-49b1-8cd2-51a4c47de80a\") " pod="odh-ai-gateway-infra/maas-api-7f4ccfcf79-tlkq8"
Oct 30 22:00:02.986590 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:00:02.986428    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/91c8233a-1602-49b1-8cd2-51a4c47de80a-maas-api-tls\") pod \"maas-api-7f4ccfcf79-tlkq8\" (UID: \"91c8233a-1602-49b1-8cd2-51a4c47de80a\") " pod="odh-ai-gateway-infra/maas-api-7f4ccfcf79-tlkq8"
Oct 30 22:00:02.986590 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:00:02.986539    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/91c8233a-1602-49b1-8cd2-51a4c47de80a-metrics-certs\") pod \"maas-api-7f4ccfcf79-tlkq8\" (UID: \"91c8233a-1602-49b1-8cd2-51a4c47de80a\") " pod="odh-ai-gateway-infra/maas-api-7f4ccfcf79-tlkq8"
Oct 30 22:00:02.986752 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:00:02.986702    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b7qvz\" (UniqueName: \"kubernetes.io/projected/91c8233a-1602-49b1-8cd2-51a4c47de80a-kube-api-access-b7qvz\") pod \"maas-api-7f4ccfcf79-tlkq8\" (UID: \"91c8233a-1602-49b1-8cd2-51a4c47de80a\") " pod="odh-ai-gateway-infra/maas-api-7f4ccfcf79-tlkq8"
Oct 30 22:00:03.087881 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:00:03.087842    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b7qvz\" (UniqueName: \"kubernetes.io/projected/91c8233a-1602-49b1-8cd2-51a4c47de80a-kube-api-access-b7qvz\") pod \"maas-api-7f4ccfcf79-tlkq8\" (UID: \"91c8233a-1602-49b1-8cd2-51a4c47de80a\") " pod="odh-ai-gateway-infra/maas-api-7f4ccfcf79-tlkq8"
Oct 30 22:00:03.087881 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:00:03.087885    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/91c8233a-1602-49b1-8cd2-51a4c47de80a-openshift-service-ca\") pod \"maas-api-7f4ccfcf79-tlkq8\" (UID: \"91c8233a-1602-49b1-8cd2-51a4c47de80a\") " pod="odh-ai-gateway-infra/maas-api-7f4ccfcf79-tlkq8"
Oct 30 22:00:03.088098 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:00:03.088065    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/91c8233a-1602-49b1-8cd2-51a4c47de80a-maas-api-tls\") pod \"maas-api-7f4ccfcf79-tlkq8\" (UID: \"91c8233a-1602-49b1-8cd2-51a4c47de80a\") " pod="odh-ai-gateway-infra/maas-api-7f4ccfcf79-tlkq8"
Oct 30 22:00:03.088155 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:00:03.088110    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/91c8233a-1602-49b1-8cd2-51a4c47de80a-metrics-certs\") pod \"maas-api-7f4ccfcf79-tlkq8\" (UID: \"91c8233a-1602-49b1-8cd2-51a4c47de80a\") " pod="odh-ai-gateway-infra/maas-api-7f4ccfcf79-tlkq8"
Oct 30 22:00:03.088585 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:00:03.088552    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/91c8233a-1602-49b1-8cd2-51a4c47de80a-openshift-service-ca\") pod \"maas-api-7f4ccfcf79-tlkq8\" (UID: \"91c8233a-1602-49b1-8cd2-51a4c47de80a\") " pod="odh-ai-gateway-infra/maas-api-7f4ccfcf79-tlkq8"
Oct 30 22:00:03.090665 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:00:03.090628    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/91c8233a-1602-49b1-8cd2-51a4c47de80a-maas-api-tls\") pod \"maas-api-7f4ccfcf79-tlkq8\" (UID: \"91c8233a-1602-49b1-8cd2-51a4c47de80a\") " pod="odh-ai-gateway-infra/maas-api-7f4ccfcf79-tlkq8"
Oct 30 22:00:03.090783 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:00:03.090668    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/91c8233a-1602-49b1-8cd2-51a4c47de80a-metrics-certs\") pod \"maas-api-7f4ccfcf79-tlkq8\" (UID: \"91c8233a-1602-49b1-8cd2-51a4c47de80a\") " pod="odh-ai-gateway-infra/maas-api-7f4ccfcf79-tlkq8"
Oct 30 22:00:03.146244 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:00:03.146207    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b7qvz\" (UniqueName: \"kubernetes.io/projected/91c8233a-1602-49b1-8cd2-51a4c47de80a-kube-api-access-b7qvz\") pod \"maas-api-7f4ccfcf79-tlkq8\" (UID: \"91c8233a-1602-49b1-8cd2-51a4c47de80a\") " pod="odh-ai-gateway-infra/maas-api-7f4ccfcf79-tlkq8"
Oct 30 22:00:03.227554 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:00:03.227511    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7f4ccfcf79-tlkq8"
Oct 30 22:00:03.615008 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:00:03.614979    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-7f4ccfcf79-tlkq8"]
Oct 30 22:00:03.618482 ip-10-0-136-210 kubenswrapper[5510]: W1030 22:00:03.618448    5510 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91c8233a_1602_49b1_8cd2_51a4c47de80a.slice/crio-735306c1e68276d95f0b07a8baf5e6a2fc0147264f95d9295d0f6d335a0a9960 WatchSource:0}: Error finding container 735306c1e68276d95f0b07a8baf5e6a2fc0147264f95d9295d0f6d335a0a9960: Status 404 returned error can't find the container with id 735306c1e68276d95f0b07a8baf5e6a2fc0147264f95d9295d0f6d335a0a9960
Oct 30 22:00:03.816947 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:00:03.816913    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7f4ccfcf79-tlkq8" event={"ID":"91c8233a-1602-49b1-8cd2-51a4c47de80a","Type":"ContainerStarted","Data":"735306c1e68276d95f0b07a8baf5e6a2fc0147264f95d9295d0f6d335a0a9960"}
Oct 30 22:00:05.826978 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:00:05.826938    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7f4ccfcf79-tlkq8" event={"ID":"91c8233a-1602-49b1-8cd2-51a4c47de80a","Type":"ContainerStarted","Data":"a17d77bf9f4e1d003f9c518d0e291e48e6f1d7097e4442d578626ceee3ce0afc"}
Oct 30 22:00:05.827399 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:00:05.827059    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-7f4ccfcf79-tlkq8"
Oct 30 22:00:08.419524 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:00:08.419486    5510 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6gjbf" containerName="extractor"
Oct 30 22:00:11.835891 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:00:11.835858    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-7f4ccfcf79-tlkq8"
Oct 30 22:00:11.947722 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:00:11.947651    5510 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-7f4ccfcf79-tlkq8" podStartSLOduration=8.769201 podStartE2EDuration="9.947636544s" totalImagesPullingTime="1.178435544s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:00:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:00:05.941788092 +0000 UTC m=+586.502598117" watchObservedRunningTime="2026-10-30 22:00:11.947636544 +0000 UTC m=+592.508446551"
Oct 30 22:00:11.994852 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:00:11.994821    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-6d669c7fcd-jhbb2"]
Oct 30 22:00:11.995062 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:00:11.995027    5510 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-jhbb2" podUID="1bf676b2-7a2c-4ea4-b2fa-e8a6c17d9782" containerName="maas-api" containerID="cri-o://909aab9125898f8ebd6c3f624bb166b92040e6e1198f8af9eab8202670a01f1c" gracePeriod=30
Oct 30 22:00:12.231577 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:00:12.231549    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-jhbb2"
Oct 30 22:00:12.383459 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:00:12.383367    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1bf676b2-7a2c-4ea4-b2fa-e8a6c17d9782-metrics-certs\" (UniqueName: \"kubernetes.io/secret/1bf676b2-7a2c-4ea4-b2fa-e8a6c17d9782-metrics-certs\") pod \"1bf676b2-7a2c-4ea4-b2fa-e8a6c17d9782\" (UID: \"1bf676b2-7a2c-4ea4-b2fa-e8a6c17d9782\") "
Oct 30 22:00:12.383459 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:00:12.383412    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1bf676b2-7a2c-4ea4-b2fa-e8a6c17d9782-kube-api-access-7xkrk\" (UniqueName: \"kubernetes.io/projected/1bf676b2-7a2c-4ea4-b2fa-e8a6c17d9782-kube-api-access-7xkrk\") pod \"1bf676b2-7a2c-4ea4-b2fa-e8a6c17d9782\" (UID: \"1bf676b2-7a2c-4ea4-b2fa-e8a6c17d9782\") "
Oct 30 22:00:12.383732 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:00:12.383525    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1bf676b2-7a2c-4ea4-b2fa-e8a6c17d9782-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1bf676b2-7a2c-4ea4-b2fa-e8a6c17d9782-openshift-service-ca\") pod \"1bf676b2-7a2c-4ea4-b2fa-e8a6c17d9782\" (UID: \"1bf676b2-7a2c-4ea4-b2fa-e8a6c17d9782\") "
Oct 30 22:00:12.383732 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:00:12.383593    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1bf676b2-7a2c-4ea4-b2fa-e8a6c17d9782-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1bf676b2-7a2c-4ea4-b2fa-e8a6c17d9782-maas-api-tls\") pod \"1bf676b2-7a2c-4ea4-b2fa-e8a6c17d9782\" (UID: \"1bf676b2-7a2c-4ea4-b2fa-e8a6c17d9782\") "
Oct 30 22:00:12.383987 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:00:12.383959    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1bf676b2-7a2c-4ea4-b2fa-e8a6c17d9782-openshift-service-ca" pod "1bf676b2-7a2c-4ea4-b2fa-e8a6c17d9782" (UID: "1bf676b2-7a2c-4ea4-b2fa-e8a6c17d9782"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:00:12.385855 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:00:12.385826    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1bf676b2-7a2c-4ea4-b2fa-e8a6c17d9782-maas-api-tls" pod "1bf676b2-7a2c-4ea4-b2fa-e8a6c17d9782" (UID: "1bf676b2-7a2c-4ea4-b2fa-e8a6c17d9782"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:00:12.385855 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:00:12.385847    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1bf676b2-7a2c-4ea4-b2fa-e8a6c17d9782-metrics-certs" pod "1bf676b2-7a2c-4ea4-b2fa-e8a6c17d9782" (UID: "1bf676b2-7a2c-4ea4-b2fa-e8a6c17d9782"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:00:12.385991 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:00:12.385865    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1bf676b2-7a2c-4ea4-b2fa-e8a6c17d9782-kube-api-access-7xkrk" pod "1bf676b2-7a2c-4ea4-b2fa-e8a6c17d9782" (UID: "1bf676b2-7a2c-4ea4-b2fa-e8a6c17d9782"). InnerVolumeSpecName "kube-api-access-7xkrk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:00:12.485191 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:00:12.485155    5510 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1bf676b2-7a2c-4ea4-b2fa-e8a6c17d9782-openshift-service-ca\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:00:12.485191 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:00:12.485188    5510 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1bf676b2-7a2c-4ea4-b2fa-e8a6c17d9782-maas-api-tls\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:00:12.485191 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:00:12.485197    5510 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1bf676b2-7a2c-4ea4-b2fa-e8a6c17d9782-metrics-certs\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:00:12.485436 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:00:12.485207    5510 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7xkrk\" (UniqueName: \"kubernetes.io/projected/1bf676b2-7a2c-4ea4-b2fa-e8a6c17d9782-kube-api-access-7xkrk\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:00:12.856199 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:00:12.856161    5510 generic.go:356] "Generic (PLEG): container finished" podID="1bf676b2-7a2c-4ea4-b2fa-e8a6c17d9782" containerID="909aab9125898f8ebd6c3f624bb166b92040e6e1198f8af9eab8202670a01f1c" exitCode=0
Oct 30 22:00:12.856701 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:00:12.856206    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-jhbb2"
Oct 30 22:00:12.856701 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:00:12.856240    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-jhbb2" event={"ID":"1bf676b2-7a2c-4ea4-b2fa-e8a6c17d9782","Type":"ContainerDied","Data":"909aab9125898f8ebd6c3f624bb166b92040e6e1198f8af9eab8202670a01f1c"}
Oct 30 22:00:12.856701 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:00:12.856293    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-jhbb2" event={"ID":"1bf676b2-7a2c-4ea4-b2fa-e8a6c17d9782","Type":"ContainerDied","Data":"9ae6364a93394a2e8667b331d0cbb82cc992addcc1d7aab836e4b458633e7c70"}
Oct 30 22:00:12.856701 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:00:12.856317    5510 scope.go:122] "RemoveContainer" containerID="909aab9125898f8ebd6c3f624bb166b92040e6e1198f8af9eab8202670a01f1c"
Oct 30 22:00:12.866812 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:00:12.866715    5510 scope.go:122] "RemoveContainer" containerID="909aab9125898f8ebd6c3f624bb166b92040e6e1198f8af9eab8202670a01f1c"
Oct 30 22:00:12.870873 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:00:12.870676    5510 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"909aab9125898f8ebd6c3f624bb166b92040e6e1198f8af9eab8202670a01f1c\": container with ID starting with 909aab9125898f8ebd6c3f624bb166b92040e6e1198f8af9eab8202670a01f1c not found: ID does not exist" containerID="909aab9125898f8ebd6c3f624bb166b92040e6e1198f8af9eab8202670a01f1c"
Oct 30 22:00:12.870873 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:00:12.870727    5510 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"909aab9125898f8ebd6c3f624bb166b92040e6e1198f8af9eab8202670a01f1c"} err="failed to get container status \"909aab9125898f8ebd6c3f624bb166b92040e6e1198f8af9eab8202670a01f1c\": rpc error: code = NotFound desc = could not find container \"909aab9125898f8ebd6c3f624bb166b92040e6e1198f8af9eab8202670a01f1c\": container with ID starting with 909aab9125898f8ebd6c3f624bb166b92040e6e1198f8af9eab8202670a01f1c not found: ID does not exist"
Oct 30 22:00:13.013210 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:00:13.013181    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-6d669c7fcd-jhbb2"]
Oct 30 22:00:13.059961 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:00:13.059929    5510 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-6d669c7fcd-jhbb2"]
Oct 30 22:00:13.551322 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:00:13.551289    5510 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1bf676b2-7a2c-4ea4-b2fa-e8a6c17d9782" path="/var/lib/kubelet/pods/1bf676b2-7a2c-4ea4-b2fa-e8a6c17d9782/volumes"
Oct 30 22:00:21.208343 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:00:21.208301    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:00:38.419177 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:00:38.419135    5510 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6gjbf" containerName="extractor"
Oct 30 22:00:54.923399 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:00:54.923360    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:01:08.418476 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:01:08.418401    5510 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6gjbf" containerName="extractor"
Oct 30 22:01:21.387803 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:01:21.387752    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:01:21.758798 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:01:21.758701    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:01:22.001776 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:01:22.001737    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:01:24.767212 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:01:24.767172    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:01:24.973595 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:01:24.973559    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:01:38.419483 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:01:38.419446    5510 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6gjbf" containerName="extractor"
Oct 30 22:01:44.761177 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:01:44.761142    5510 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-84b4ff656c-vrvcr"]
Oct 30 22:01:44.761793 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:01:44.761740    5510 cpu_manager.go:414] "containerMap: removing container" podUID="1bf676b2-7a2c-4ea4-b2fa-e8a6c17d9782" containerName="maas-api"
Oct 30 22:01:44.761793 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:01:44.761761    5510 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1bf676b2-7a2c-4ea4-b2fa-e8a6c17d9782" containerName="maas-api"
Oct 30 22:01:44.761926 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:01:44.761841    5510 memory_manager.go:365] "RemoveStaleState removing state" podUID="1bf676b2-7a2c-4ea4-b2fa-e8a6c17d9782" containerName="maas-api"
Oct 30 22:01:44.764048 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:01:44.764022    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-vrvcr"
Oct 30 22:01:44.861211 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:01:44.861175    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0cf74802-5189-453b-8b55-801a5f442939-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-vrvcr\" (UID: \"0cf74802-5189-453b-8b55-801a5f442939\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-vrvcr"
Oct 30 22:01:44.861211 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:01:44.861217    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0cf74802-5189-453b-8b55-801a5f442939-dshm\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-vrvcr\" (UID: \"0cf74802-5189-453b-8b55-801a5f442939\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-vrvcr"
Oct 30 22:01:44.861406 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:01:44.861234    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0cf74802-5189-453b-8b55-801a5f442939-model-cache\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-vrvcr\" (UID: \"0cf74802-5189-453b-8b55-801a5f442939\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-vrvcr"
Oct 30 22:01:44.861406 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:01:44.861341    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0cf74802-5189-453b-8b55-801a5f442939-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-vrvcr\" (UID: \"0cf74802-5189-453b-8b55-801a5f442939\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-vrvcr"
Oct 30 22:01:44.861483 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:01:44.861408    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0cf74802-5189-453b-8b55-801a5f442939-home\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-vrvcr\" (UID: \"0cf74802-5189-453b-8b55-801a5f442939\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-vrvcr"
Oct 30 22:01:44.861483 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:01:44.861429    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p65t9\" (UniqueName: \"kubernetes.io/projected/0cf74802-5189-453b-8b55-801a5f442939-kube-api-access-p65t9\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-vrvcr\" (UID: \"0cf74802-5189-453b-8b55-801a5f442939\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-vrvcr"
Oct 30 22:01:44.864931 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:01:44.864912    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-f92sw\""
Oct 30 22:01:44.865000 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:01:44.864912    5510 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Oct 30 22:01:44.865000 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:01:44.864921    5510 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Oct 30 22:01:44.865000 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:01:44.864954    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Oct 30 22:01:44.867221 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:01:44.867202    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-84b4ff656c-vrvcr"]
Oct 30 22:01:44.915332 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:01:44.915298    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:01:44.962354 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:01:44.962323    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0cf74802-5189-453b-8b55-801a5f442939-model-cache\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-vrvcr\" (UID: \"0cf74802-5189-453b-8b55-801a5f442939\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-vrvcr"
Oct 30 22:01:44.962506 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:01:44.962360    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0cf74802-5189-453b-8b55-801a5f442939-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-vrvcr\" (UID: \"0cf74802-5189-453b-8b55-801a5f442939\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-vrvcr"
Oct 30 22:01:44.962506 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:01:44.962403    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0cf74802-5189-453b-8b55-801a5f442939-home\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-vrvcr\" (UID: \"0cf74802-5189-453b-8b55-801a5f442939\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-vrvcr"
Oct 30 22:01:44.962506 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:01:44.962423    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p65t9\" (UniqueName: \"kubernetes.io/projected/0cf74802-5189-453b-8b55-801a5f442939-kube-api-access-p65t9\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-vrvcr\" (UID: \"0cf74802-5189-453b-8b55-801a5f442939\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-vrvcr"
Oct 30 22:01:44.962506 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:01:44.962457    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0cf74802-5189-453b-8b55-801a5f442939-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-vrvcr\" (UID: \"0cf74802-5189-453b-8b55-801a5f442939\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-vrvcr"
Oct 30 22:01:44.962506 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:01:44.962500    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0cf74802-5189-453b-8b55-801a5f442939-dshm\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-vrvcr\" (UID: \"0cf74802-5189-453b-8b55-801a5f442939\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-vrvcr"
Oct 30 22:01:44.962795 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:01:44.962778    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0cf74802-5189-453b-8b55-801a5f442939-model-cache\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-vrvcr\" (UID: \"0cf74802-5189-453b-8b55-801a5f442939\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-vrvcr"
Oct 30 22:01:44.962869 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:01:44.962848    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0cf74802-5189-453b-8b55-801a5f442939-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-vrvcr\" (UID: \"0cf74802-5189-453b-8b55-801a5f442939\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-vrvcr"
Oct 30 22:01:44.962904 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:01:44.962855    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0cf74802-5189-453b-8b55-801a5f442939-home\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-vrvcr\" (UID: \"0cf74802-5189-453b-8b55-801a5f442939\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-vrvcr"
Oct 30 22:01:44.964790 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:01:44.964763    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0cf74802-5189-453b-8b55-801a5f442939-dshm\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-vrvcr\" (UID: \"0cf74802-5189-453b-8b55-801a5f442939\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-vrvcr"
Oct 30 22:01:44.965031 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:01:44.965013    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0cf74802-5189-453b-8b55-801a5f442939-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-vrvcr\" (UID: \"0cf74802-5189-453b-8b55-801a5f442939\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-vrvcr"
Oct 30 22:01:45.020544 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:01:45.020513    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p65t9\" (UniqueName: \"kubernetes.io/projected/0cf74802-5189-453b-8b55-801a5f442939-kube-api-access-p65t9\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-vrvcr\" (UID: \"0cf74802-5189-453b-8b55-801a5f442939\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-vrvcr"
Oct 30 22:01:45.076342 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:01:45.076291    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-vrvcr"
Oct 30 22:01:45.468600 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:01:45.468523    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-84b4ff656c-vrvcr"]
Oct 30 22:01:45.471721 ip-10-0-136-210 kubenswrapper[5510]: W1030 22:01:45.471690    5510 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0cf74802_5189_453b_8b55_801a5f442939.slice/crio-d53d2bb79ba607e600d09677e2998a5542b2e49a53f52de875a7d27ad5fcec16 WatchSource:0}: Error finding container d53d2bb79ba607e600d09677e2998a5542b2e49a53f52de875a7d27ad5fcec16: Status 404 returned error can't find the container with id d53d2bb79ba607e600d09677e2998a5542b2e49a53f52de875a7d27ad5fcec16
Oct 30 22:01:45.473184 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:01:45.473160    5510 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 22:01:46.241287 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:01:46.241208    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-vrvcr" event={"ID":"0cf74802-5189-453b-8b55-801a5f442939","Type":"ContainerStarted","Data":"d53d2bb79ba607e600d09677e2998a5542b2e49a53f52de875a7d27ad5fcec16"}
Oct 30 22:01:47.247217 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:01:47.247136    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-vrvcr" event={"ID":"0cf74802-5189-453b-8b55-801a5f442939","Type":"ContainerStarted","Data":"046d6a7be9506540c1d7df4a776bd0e2388b1eb475e2983f5be1de2ebba34bcc"}
Oct 30 22:01:47.359749 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:01:47.359693    5510 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-vrvcr" podStartSLOduration=1.849559625 podStartE2EDuration="3.359647574s" totalImagesPullingTime="1.510087949s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:01:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:01:47.359212821 +0000 UTC m=+687.920022825" watchObservedRunningTime="2026-10-30 22:01:47.359647574 +0000 UTC m=+687.920457579"
Oct 30 22:01:48.076603 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:01:48.076565    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-vrvcr"
Oct 30 22:01:48.457080 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:01:48.456994    5510 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/echo-simulator-kserve-64b6f8f96c-b62z6"]
Oct 30 22:01:48.459923 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:01:48.459899    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/echo-simulator-kserve-64b6f8f96c-b62z6"
Oct 30 22:01:48.557252 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:01:48.557213    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"echo-simulator-kserve-self-signed-certs\""
Oct 30 22:01:48.558575 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:01:48.558554    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/echo-simulator-kserve-64b6f8f96c-b62z6"]
Oct 30 22:01:48.597086 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:01:48.597044    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/558165fb-690a-4dbf-8362-defbce94ea9b-model-cache\") pod \"echo-simulator-kserve-64b6f8f96c-b62z6\" (UID: \"558165fb-690a-4dbf-8362-defbce94ea9b\") " pod="llm/echo-simulator-kserve-64b6f8f96c-b62z6"
Oct 30 22:01:48.597244 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:01:48.597114    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/558165fb-690a-4dbf-8362-defbce94ea9b-home\") pod \"echo-simulator-kserve-64b6f8f96c-b62z6\" (UID: \"558165fb-690a-4dbf-8362-defbce94ea9b\") " pod="llm/echo-simulator-kserve-64b6f8f96c-b62z6"
Oct 30 22:01:48.597244 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:01:48.597215    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dsj98\" (UniqueName: \"kubernetes.io/projected/558165fb-690a-4dbf-8362-defbce94ea9b-kube-api-access-dsj98\") pod \"echo-simulator-kserve-64b6f8f96c-b62z6\" (UID: \"558165fb-690a-4dbf-8362-defbce94ea9b\") " pod="llm/echo-simulator-kserve-64b6f8f96c-b62z6"
Oct 30 22:01:48.597355 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:01:48.597253    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/558165fb-690a-4dbf-8362-defbce94ea9b-dshm\") pod \"echo-simulator-kserve-64b6f8f96c-b62z6\" (UID: \"558165fb-690a-4dbf-8362-defbce94ea9b\") " pod="llm/echo-simulator-kserve-64b6f8f96c-b62z6"
Oct 30 22:01:48.597408 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:01:48.597377    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/558165fb-690a-4dbf-8362-defbce94ea9b-tmp-dir\") pod \"echo-simulator-kserve-64b6f8f96c-b62z6\" (UID: \"558165fb-690a-4dbf-8362-defbce94ea9b\") " pod="llm/echo-simulator-kserve-64b6f8f96c-b62z6"
Oct 30 22:01:48.597452 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:01:48.597409    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/558165fb-690a-4dbf-8362-defbce94ea9b-tls-certs\") pod \"echo-simulator-kserve-64b6f8f96c-b62z6\" (UID: \"558165fb-690a-4dbf-8362-defbce94ea9b\") " pod="llm/echo-simulator-kserve-64b6f8f96c-b62z6"
Oct 30 22:01:48.618340 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:01:48.618296    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:01:48.698314 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:01:48.698279    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/558165fb-690a-4dbf-8362-defbce94ea9b-tmp-dir\") pod \"echo-simulator-kserve-64b6f8f96c-b62z6\" (UID: \"558165fb-690a-4dbf-8362-defbce94ea9b\") " pod="llm/echo-simulator-kserve-64b6f8f96c-b62z6"
Oct 30 22:01:48.698314 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:01:48.698314    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/558165fb-690a-4dbf-8362-defbce94ea9b-tls-certs\") pod \"echo-simulator-kserve-64b6f8f96c-b62z6\" (UID: \"558165fb-690a-4dbf-8362-defbce94ea9b\") " pod="llm/echo-simulator-kserve-64b6f8f96c-b62z6"
Oct 30 22:01:48.698554 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:01:48.698363    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/558165fb-690a-4dbf-8362-defbce94ea9b-model-cache\") pod \"echo-simulator-kserve-64b6f8f96c-b62z6\" (UID: \"558165fb-690a-4dbf-8362-defbce94ea9b\") " pod="llm/echo-simulator-kserve-64b6f8f96c-b62z6"
Oct 30 22:01:48.698554 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:01:48.698446    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/558165fb-690a-4dbf-8362-defbce94ea9b-home\") pod \"echo-simulator-kserve-64b6f8f96c-b62z6\" (UID: \"558165fb-690a-4dbf-8362-defbce94ea9b\") " pod="llm/echo-simulator-kserve-64b6f8f96c-b62z6"
Oct 30 22:01:48.698554 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:01:48.698490    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dsj98\" (UniqueName: \"kubernetes.io/projected/558165fb-690a-4dbf-8362-defbce94ea9b-kube-api-access-dsj98\") pod \"echo-simulator-kserve-64b6f8f96c-b62z6\" (UID: \"558165fb-690a-4dbf-8362-defbce94ea9b\") " pod="llm/echo-simulator-kserve-64b6f8f96c-b62z6"
Oct 30 22:01:48.698554 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:01:48.698518    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/558165fb-690a-4dbf-8362-defbce94ea9b-dshm\") pod \"echo-simulator-kserve-64b6f8f96c-b62z6\" (UID: \"558165fb-690a-4dbf-8362-defbce94ea9b\") " pod="llm/echo-simulator-kserve-64b6f8f96c-b62z6"
Oct 30 22:01:48.698887 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:01:48.698863    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/558165fb-690a-4dbf-8362-defbce94ea9b-model-cache\") pod \"echo-simulator-kserve-64b6f8f96c-b62z6\" (UID: \"558165fb-690a-4dbf-8362-defbce94ea9b\") " pod="llm/echo-simulator-kserve-64b6f8f96c-b62z6"
Oct 30 22:01:48.698997 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:01:48.698960    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/558165fb-690a-4dbf-8362-defbce94ea9b-home\") pod \"echo-simulator-kserve-64b6f8f96c-b62z6\" (UID: \"558165fb-690a-4dbf-8362-defbce94ea9b\") " pod="llm/echo-simulator-kserve-64b6f8f96c-b62z6"
Oct 30 22:01:48.699060 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:01:48.699039    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/558165fb-690a-4dbf-8362-defbce94ea9b-tmp-dir\") pod \"echo-simulator-kserve-64b6f8f96c-b62z6\" (UID: \"558165fb-690a-4dbf-8362-defbce94ea9b\") " pod="llm/echo-simulator-kserve-64b6f8f96c-b62z6"
Oct 30 22:01:48.700640 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:01:48.700618    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/558165fb-690a-4dbf-8362-defbce94ea9b-dshm\") pod \"echo-simulator-kserve-64b6f8f96c-b62z6\" (UID: \"558165fb-690a-4dbf-8362-defbce94ea9b\") " pod="llm/echo-simulator-kserve-64b6f8f96c-b62z6"
Oct 30 22:01:48.700933 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:01:48.700915    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/558165fb-690a-4dbf-8362-defbce94ea9b-tls-certs\") pod \"echo-simulator-kserve-64b6f8f96c-b62z6\" (UID: \"558165fb-690a-4dbf-8362-defbce94ea9b\") " pod="llm/echo-simulator-kserve-64b6f8f96c-b62z6"
Oct 30 22:01:48.757085 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:01:48.757010    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dsj98\" (UniqueName: \"kubernetes.io/projected/558165fb-690a-4dbf-8362-defbce94ea9b-kube-api-access-dsj98\") pod \"echo-simulator-kserve-64b6f8f96c-b62z6\" (UID: \"558165fb-690a-4dbf-8362-defbce94ea9b\") " pod="llm/echo-simulator-kserve-64b6f8f96c-b62z6"
Oct 30 22:01:48.771840 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:01:48.771813    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/echo-simulator-kserve-64b6f8f96c-b62z6"
Oct 30 22:01:49.140241 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:01:49.140218    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/echo-simulator-kserve-64b6f8f96c-b62z6"]
Oct 30 22:01:49.142900 ip-10-0-136-210 kubenswrapper[5510]: W1030 22:01:49.142860    5510 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod558165fb_690a_4dbf_8362_defbce94ea9b.slice/crio-51fc8912b37fb1387ed24abd26337f5a16d3fbee14685d5d2766506e6d18d760 WatchSource:0}: Error finding container 51fc8912b37fb1387ed24abd26337f5a16d3fbee14685d5d2766506e6d18d760: Status 404 returned error can't find the container with id 51fc8912b37fb1387ed24abd26337f5a16d3fbee14685d5d2766506e6d18d760
Oct 30 22:01:49.255567 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:01:49.255531    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/echo-simulator-kserve-64b6f8f96c-b62z6" event={"ID":"558165fb-690a-4dbf-8362-defbce94ea9b","Type":"ContainerStarted","Data":"51fc8912b37fb1387ed24abd26337f5a16d3fbee14685d5d2766506e6d18d760"}
Oct 30 22:01:52.270567 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:01:52.270530    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/echo-simulator-kserve-64b6f8f96c-b62z6" event={"ID":"558165fb-690a-4dbf-8362-defbce94ea9b","Type":"ContainerStarted","Data":"f96a04292559c61edc293e1ff6c6c6da3258501ff95b50a7e4d6822ab77bb1bc"}
Oct 30 22:01:52.383759 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:01:52.383707    5510 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/echo-simulator-kserve-64b6f8f96c-b62z6" podStartSLOduration=1.7752775170000001 podStartE2EDuration="4.383694684s" totalImagesPullingTime="2.608417167s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:01:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:01:52.382897188 +0000 UTC m=+692.943707203" watchObservedRunningTime="2026-10-30 22:01:52.383694684 +0000 UTC m=+692.944504689"
Oct 30 22:01:52.772785 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:01:52.772757    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/echo-simulator-kserve-64b6f8f96c-b62z6"
Oct 30 22:01:55.076775 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:01:55.076734    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-vrvcr"
Oct 30 22:01:55.089560 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:01:55.089535    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-vrvcr"
Oct 30 22:01:55.294706 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:01:55.294650    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-vrvcr"
Oct 30 22:01:55.485271 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:01:55.485191    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:01:58.772485 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:01:58.772443    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/echo-simulator-kserve-64b6f8f96c-b62z6"
Oct 30 22:01:58.785238 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:01:58.785211    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/echo-simulator-kserve-64b6f8f96c-b62z6"
Oct 30 22:01:59.311803 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:01:59.311773    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/echo-simulator-kserve-64b6f8f96c-b62z6"
Oct 30 22:02:00.190973 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:00.190942    5510 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-cbd4f5586-42mzp"]
Oct 30 22:02:00.193621 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:00.193597    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-42mzp"
Oct 30 22:02:00.283027 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:00.282996    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Oct 30 22:02:00.292786 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:00.292756    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-cbd4f5586-42mzp"]
Oct 30 22:02:00.298702 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:00.298651    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9f899391-6025-4f44-9553-a82cd5e25e69-home\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-42mzp\" (UID: \"9f899391-6025-4f44-9553-a82cd5e25e69\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-42mzp"
Oct 30 22:02:00.298795 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:00.298738    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9f899391-6025-4f44-9553-a82cd5e25e69-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-42mzp\" (UID: \"9f899391-6025-4f44-9553-a82cd5e25e69\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-42mzp"
Oct 30 22:02:00.298795 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:00.298762    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9f899391-6025-4f44-9553-a82cd5e25e69-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-42mzp\" (UID: \"9f899391-6025-4f44-9553-a82cd5e25e69\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-42mzp"
Oct 30 22:02:00.298882 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:00.298812    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9f899391-6025-4f44-9553-a82cd5e25e69-dshm\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-42mzp\" (UID: \"9f899391-6025-4f44-9553-a82cd5e25e69\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-42mzp"
Oct 30 22:02:00.298951 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:00.298934    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z2g4h\" (UniqueName: \"kubernetes.io/projected/9f899391-6025-4f44-9553-a82cd5e25e69-kube-api-access-z2g4h\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-42mzp\" (UID: \"9f899391-6025-4f44-9553-a82cd5e25e69\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-42mzp"
Oct 30 22:02:00.299021 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:00.299009    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9f899391-6025-4f44-9553-a82cd5e25e69-model-cache\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-42mzp\" (UID: \"9f899391-6025-4f44-9553-a82cd5e25e69\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-42mzp"
Oct 30 22:02:00.400455 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:00.400368    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z2g4h\" (UniqueName: \"kubernetes.io/projected/9f899391-6025-4f44-9553-a82cd5e25e69-kube-api-access-z2g4h\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-42mzp\" (UID: \"9f899391-6025-4f44-9553-a82cd5e25e69\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-42mzp"
Oct 30 22:02:00.400455 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:00.400423    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9f899391-6025-4f44-9553-a82cd5e25e69-model-cache\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-42mzp\" (UID: \"9f899391-6025-4f44-9553-a82cd5e25e69\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-42mzp"
Oct 30 22:02:00.400455 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:00.400451    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9f899391-6025-4f44-9553-a82cd5e25e69-home\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-42mzp\" (UID: \"9f899391-6025-4f44-9553-a82cd5e25e69\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-42mzp"
Oct 30 22:02:00.400867 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:00.400469    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9f899391-6025-4f44-9553-a82cd5e25e69-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-42mzp\" (UID: \"9f899391-6025-4f44-9553-a82cd5e25e69\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-42mzp"
Oct 30 22:02:00.400867 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:00.400484    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9f899391-6025-4f44-9553-a82cd5e25e69-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-42mzp\" (UID: \"9f899391-6025-4f44-9553-a82cd5e25e69\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-42mzp"
Oct 30 22:02:00.400867 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:00.400610    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9f899391-6025-4f44-9553-a82cd5e25e69-dshm\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-42mzp\" (UID: \"9f899391-6025-4f44-9553-a82cd5e25e69\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-42mzp"
Oct 30 22:02:00.400867 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:00.400853    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9f899391-6025-4f44-9553-a82cd5e25e69-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-42mzp\" (UID: \"9f899391-6025-4f44-9553-a82cd5e25e69\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-42mzp"
Oct 30 22:02:00.401146 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:00.401128    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9f899391-6025-4f44-9553-a82cd5e25e69-home\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-42mzp\" (UID: \"9f899391-6025-4f44-9553-a82cd5e25e69\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-42mzp"
Oct 30 22:02:00.401255 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:00.401230    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9f899391-6025-4f44-9553-a82cd5e25e69-model-cache\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-42mzp\" (UID: \"9f899391-6025-4f44-9553-a82cd5e25e69\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-42mzp"
Oct 30 22:02:00.402985 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:00.402956    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9f899391-6025-4f44-9553-a82cd5e25e69-dshm\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-42mzp\" (UID: \"9f899391-6025-4f44-9553-a82cd5e25e69\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-42mzp"
Oct 30 22:02:00.403095 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:00.403078    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9f899391-6025-4f44-9553-a82cd5e25e69-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-42mzp\" (UID: \"9f899391-6025-4f44-9553-a82cd5e25e69\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-42mzp"
Oct 30 22:02:00.459815 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:00.459739    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z2g4h\" (UniqueName: \"kubernetes.io/projected/9f899391-6025-4f44-9553-a82cd5e25e69-kube-api-access-z2g4h\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-42mzp\" (UID: \"9f899391-6025-4f44-9553-a82cd5e25e69\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-42mzp"
Oct 30 22:02:00.486138 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:00.486111    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:02:00.505232 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:00.505206    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-42mzp"
Oct 30 22:02:00.885411 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:00.885336    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-cbd4f5586-42mzp"]
Oct 30 22:02:00.887822 ip-10-0-136-210 kubenswrapper[5510]: W1030 22:02:00.887777    5510 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f899391_6025_4f44_9553_a82cd5e25e69.slice/crio-31fc057f4ff1a4cba3e7ec82adff0b8de3e1b3048358e8ca7f8f597f09247595 WatchSource:0}: Error finding container 31fc057f4ff1a4cba3e7ec82adff0b8de3e1b3048358e8ca7f8f597f09247595: Status 404 returned error can't find the container with id 31fc057f4ff1a4cba3e7ec82adff0b8de3e1b3048358e8ca7f8f597f09247595
Oct 30 22:02:01.309725 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:01.309690    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-42mzp" event={"ID":"9f899391-6025-4f44-9553-a82cd5e25e69","Type":"ContainerStarted","Data":"31fc057f4ff1a4cba3e7ec82adff0b8de3e1b3048358e8ca7f8f597f09247595"}
Oct 30 22:02:02.316610 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:02.316578    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-42mzp" event={"ID":"9f899391-6025-4f44-9553-a82cd5e25e69","Type":"ContainerStarted","Data":"eda2565db0e377d7e1ee2775b90fbdf488b29fc5cd1df7f67c345784ae47b372"}
Oct 30 22:02:02.432698 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:02.432623    5510 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-42mzp" podStartSLOduration=1.828949173 podStartE2EDuration="2.432607823s" totalImagesPullingTime="603.65865ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:02:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:02:02.431817969 +0000 UTC m=+702.992627988" watchObservedRunningTime="2026-10-30 22:02:02.432607823 +0000 UTC m=+702.993417900"
Oct 30 22:02:02.506051 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:02.506016    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-42mzp"
Oct 30 22:02:03.325027 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:03.324995    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:02:07.324957 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:07.324920    5510 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-embedding-simulated-kserve-67bf594cd6-zz2gk"]
Oct 30 22:02:07.327751 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:07.327730    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-zz2gk"
Oct 30 22:02:07.359914 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:07.359882    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cb6133b6-d8dc-4687-9974-68874f9a34d4-home\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-zz2gk\" (UID: \"cb6133b6-d8dc-4687-9974-68874f9a34d4\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-zz2gk"
Oct 30 22:02:07.360048 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:07.359923    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pz8tw\" (UniqueName: \"kubernetes.io/projected/cb6133b6-d8dc-4687-9974-68874f9a34d4-kube-api-access-pz8tw\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-zz2gk\" (UID: \"cb6133b6-d8dc-4687-9974-68874f9a34d4\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-zz2gk"
Oct 30 22:02:07.360048 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:07.360010    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cb6133b6-d8dc-4687-9974-68874f9a34d4-model-cache\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-zz2gk\" (UID: \"cb6133b6-d8dc-4687-9974-68874f9a34d4\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-zz2gk"
Oct 30 22:02:07.360119 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:07.360076    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cb6133b6-d8dc-4687-9974-68874f9a34d4-dshm\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-zz2gk\" (UID: \"cb6133b6-d8dc-4687-9974-68874f9a34d4\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-zz2gk"
Oct 30 22:02:07.360119 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:07.360108    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cb6133b6-d8dc-4687-9974-68874f9a34d4-tmp-dir\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-zz2gk\" (UID: \"cb6133b6-d8dc-4687-9974-68874f9a34d4\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-zz2gk"
Oct 30 22:02:07.360184 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:07.360130    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cb6133b6-d8dc-4687-9974-68874f9a34d4-tls-certs\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-zz2gk\" (UID: \"cb6133b6-d8dc-4687-9974-68874f9a34d4\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-zz2gk"
Oct 30 22:02:07.423861 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:07.423820    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-embedding-simulated-kserve-self-signed-certs\""
Oct 30 22:02:07.425138 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:07.425115    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-67bf594cd6-zz2gk"]
Oct 30 22:02:07.461329 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:07.461297    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cb6133b6-d8dc-4687-9974-68874f9a34d4-model-cache\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-zz2gk\" (UID: \"cb6133b6-d8dc-4687-9974-68874f9a34d4\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-zz2gk"
Oct 30 22:02:07.461479 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:07.461349    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cb6133b6-d8dc-4687-9974-68874f9a34d4-dshm\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-zz2gk\" (UID: \"cb6133b6-d8dc-4687-9974-68874f9a34d4\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-zz2gk"
Oct 30 22:02:07.461563 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:07.461471    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cb6133b6-d8dc-4687-9974-68874f9a34d4-tmp-dir\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-zz2gk\" (UID: \"cb6133b6-d8dc-4687-9974-68874f9a34d4\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-zz2gk"
Oct 30 22:02:07.461563 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:07.461515    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cb6133b6-d8dc-4687-9974-68874f9a34d4-tls-certs\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-zz2gk\" (UID: \"cb6133b6-d8dc-4687-9974-68874f9a34d4\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-zz2gk"
Oct 30 22:02:07.461693 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:07.461596    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cb6133b6-d8dc-4687-9974-68874f9a34d4-home\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-zz2gk\" (UID: \"cb6133b6-d8dc-4687-9974-68874f9a34d4\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-zz2gk"
Oct 30 22:02:07.461693 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:07.461639    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pz8tw\" (UniqueName: \"kubernetes.io/projected/cb6133b6-d8dc-4687-9974-68874f9a34d4-kube-api-access-pz8tw\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-zz2gk\" (UID: \"cb6133b6-d8dc-4687-9974-68874f9a34d4\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-zz2gk"
Oct 30 22:02:07.461800 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:07.461762    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cb6133b6-d8dc-4687-9974-68874f9a34d4-model-cache\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-zz2gk\" (UID: \"cb6133b6-d8dc-4687-9974-68874f9a34d4\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-zz2gk"
Oct 30 22:02:07.461854 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:07.461840    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cb6133b6-d8dc-4687-9974-68874f9a34d4-tmp-dir\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-zz2gk\" (UID: \"cb6133b6-d8dc-4687-9974-68874f9a34d4\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-zz2gk"
Oct 30 22:02:07.462045 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:07.462014    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cb6133b6-d8dc-4687-9974-68874f9a34d4-home\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-zz2gk\" (UID: \"cb6133b6-d8dc-4687-9974-68874f9a34d4\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-zz2gk"
Oct 30 22:02:07.463841 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:07.463814    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cb6133b6-d8dc-4687-9974-68874f9a34d4-dshm\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-zz2gk\" (UID: \"cb6133b6-d8dc-4687-9974-68874f9a34d4\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-zz2gk"
Oct 30 22:02:07.464013 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:07.463996    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cb6133b6-d8dc-4687-9974-68874f9a34d4-tls-certs\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-zz2gk\" (UID: \"cb6133b6-d8dc-4687-9974-68874f9a34d4\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-zz2gk"
Oct 30 22:02:07.474337 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:07.474303    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:02:07.519929 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:07.519892    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pz8tw\" (UniqueName: \"kubernetes.io/projected/cb6133b6-d8dc-4687-9974-68874f9a34d4-kube-api-access-pz8tw\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-zz2gk\" (UID: \"cb6133b6-d8dc-4687-9974-68874f9a34d4\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-zz2gk"
Oct 30 22:02:07.639893 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:07.639811    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-zz2gk"
Oct 30 22:02:08.003607 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:08.003529    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-67bf594cd6-zz2gk"]
Oct 30 22:02:08.007356 ip-10-0-136-210 kubenswrapper[5510]: W1030 22:02:08.007324    5510 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb6133b6_d8dc_4687_9974_68874f9a34d4.slice/crio-17cbfc6bf201ebe3aa60bc1263154b7686ce899eb4635a59dcf0ae25d7856511 WatchSource:0}: Error finding container 17cbfc6bf201ebe3aa60bc1263154b7686ce899eb4635a59dcf0ae25d7856511: Status 404 returned error can't find the container with id 17cbfc6bf201ebe3aa60bc1263154b7686ce899eb4635a59dcf0ae25d7856511
Oct 30 22:02:08.343026 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:08.342990    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-zz2gk" event={"ID":"cb6133b6-d8dc-4687-9974-68874f9a34d4","Type":"ContainerStarted","Data":"17cbfc6bf201ebe3aa60bc1263154b7686ce899eb4635a59dcf0ae25d7856511"}
Oct 30 22:02:08.418088 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:02:08.418058    5510 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6gjbf" containerName="extractor"
Oct 30 22:02:09.349043 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:09.349005    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-zz2gk" event={"ID":"cb6133b6-d8dc-4687-9974-68874f9a34d4","Type":"ContainerStarted","Data":"b08549972ea7112b6a4f695950ee0b551f06c1a073853ae65d49b124a7f62649"}
Oct 30 22:02:09.462103 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:09.462047    5510 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-zz2gk" podStartSLOduration=1.8658368429999999 podStartE2EDuration="2.462032512s" totalImagesPullingTime="596.195669ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:02:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:02:09.46073612 +0000 UTC m=+710.021546125" watchObservedRunningTime="2026-10-30 22:02:09.462032512 +0000 UTC m=+710.022842508"
Oct 30 22:02:09.640760 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:09.640682    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-zz2gk"
Oct 30 22:02:10.506182 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:10.506151    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-42mzp"
Oct 30 22:02:10.520158 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:10.520118    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-42mzp"
Oct 30 22:02:10.953379 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:10.953339    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:02:11.369185 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:11.369156    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-42mzp"
Oct 30 22:02:14.666031 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:14.663966    5510 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8kpvjt"]
Oct 30 22:02:14.668826 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:14.668799    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8kpvjt"
Oct 30 22:02:14.729164 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:14.729125    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qh269\" (UniqueName: \"kubernetes.io/projected/fb0f223b-851b-438d-b981-4d0d79ccfe64-kube-api-access-qh269\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8kpvjt\" (UID: \"fb0f223b-851b-438d-b981-4d0d79ccfe64\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8kpvjt"
Oct 30 22:02:14.729469 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:14.729244    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fb0f223b-851b-438d-b981-4d0d79ccfe64-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8kpvjt\" (UID: \"fb0f223b-851b-438d-b981-4d0d79ccfe64\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8kpvjt"
Oct 30 22:02:14.729469 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:14.729284    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fb0f223b-851b-438d-b981-4d0d79ccfe64-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8kpvjt\" (UID: \"fb0f223b-851b-438d-b981-4d0d79ccfe64\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8kpvjt"
Oct 30 22:02:14.729469 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:14.729379    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fb0f223b-851b-438d-b981-4d0d79ccfe64-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8kpvjt\" (UID: \"fb0f223b-851b-438d-b981-4d0d79ccfe64\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8kpvjt"
Oct 30 22:02:14.729636 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:14.729492    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fb0f223b-851b-438d-b981-4d0d79ccfe64-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8kpvjt\" (UID: \"fb0f223b-851b-438d-b981-4d0d79ccfe64\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8kpvjt"
Oct 30 22:02:14.729636 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:14.729532    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fb0f223b-851b-438d-b981-4d0d79ccfe64-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8kpvjt\" (UID: \"fb0f223b-851b-438d-b981-4d0d79ccfe64\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8kpvjt"
Oct 30 22:02:14.736674 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:14.736635    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Oct 30 22:02:14.766362 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:14.766332    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8kpvjt"]
Oct 30 22:02:14.830488 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:14.830455    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fb0f223b-851b-438d-b981-4d0d79ccfe64-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8kpvjt\" (UID: \"fb0f223b-851b-438d-b981-4d0d79ccfe64\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8kpvjt"
Oct 30 22:02:14.830721 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:14.830515    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fb0f223b-851b-438d-b981-4d0d79ccfe64-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8kpvjt\" (UID: \"fb0f223b-851b-438d-b981-4d0d79ccfe64\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8kpvjt"
Oct 30 22:02:14.830721 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:14.830588    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fb0f223b-851b-438d-b981-4d0d79ccfe64-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8kpvjt\" (UID: \"fb0f223b-851b-438d-b981-4d0d79ccfe64\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8kpvjt"
Oct 30 22:02:14.830721 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:14.830615    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fb0f223b-851b-438d-b981-4d0d79ccfe64-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8kpvjt\" (UID: \"fb0f223b-851b-438d-b981-4d0d79ccfe64\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8kpvjt"
Oct 30 22:02:14.830888 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:14.830793    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qh269\" (UniqueName: \"kubernetes.io/projected/fb0f223b-851b-438d-b981-4d0d79ccfe64-kube-api-access-qh269\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8kpvjt\" (UID: \"fb0f223b-851b-438d-b981-4d0d79ccfe64\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8kpvjt"
Oct 30 22:02:14.830888 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:14.830847    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fb0f223b-851b-438d-b981-4d0d79ccfe64-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8kpvjt\" (UID: \"fb0f223b-851b-438d-b981-4d0d79ccfe64\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8kpvjt"
Oct 30 22:02:14.830990 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:14.830971    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fb0f223b-851b-438d-b981-4d0d79ccfe64-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8kpvjt\" (UID: \"fb0f223b-851b-438d-b981-4d0d79ccfe64\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8kpvjt"
Oct 30 22:02:14.830990 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:14.830984    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fb0f223b-851b-438d-b981-4d0d79ccfe64-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8kpvjt\" (UID: \"fb0f223b-851b-438d-b981-4d0d79ccfe64\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8kpvjt"
Oct 30 22:02:14.831178 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:14.831155    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fb0f223b-851b-438d-b981-4d0d79ccfe64-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8kpvjt\" (UID: \"fb0f223b-851b-438d-b981-4d0d79ccfe64\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8kpvjt"
Oct 30 22:02:14.832902 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:14.832886    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fb0f223b-851b-438d-b981-4d0d79ccfe64-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8kpvjt\" (UID: \"fb0f223b-851b-438d-b981-4d0d79ccfe64\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8kpvjt"
Oct 30 22:02:14.833172 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:14.833154    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fb0f223b-851b-438d-b981-4d0d79ccfe64-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8kpvjt\" (UID: \"fb0f223b-851b-438d-b981-4d0d79ccfe64\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8kpvjt"
Oct 30 22:02:14.900014 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:14.899982    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qh269\" (UniqueName: \"kubernetes.io/projected/fb0f223b-851b-438d-b981-4d0d79ccfe64-kube-api-access-qh269\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8kpvjt\" (UID: \"fb0f223b-851b-438d-b981-4d0d79ccfe64\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8kpvjt"
Oct 30 22:02:14.919733 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:14.919641    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:02:14.981696 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:14.981644    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8kpvjt"
Oct 30 22:02:15.347258 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:15.347238    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8kpvjt"]
Oct 30 22:02:15.350128 ip-10-0-136-210 kubenswrapper[5510]: W1030 22:02:15.350095    5510 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb0f223b_851b_438d_b981_4d0d79ccfe64.slice/crio-d8af7d35e826e707193f0ce41ef2a5518f9ec705abb2a217e7bfc3f2700f0273 WatchSource:0}: Error finding container d8af7d35e826e707193f0ce41ef2a5518f9ec705abb2a217e7bfc3f2700f0273: Status 404 returned error can't find the container with id d8af7d35e826e707193f0ce41ef2a5518f9ec705abb2a217e7bfc3f2700f0273
Oct 30 22:02:15.376817 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:15.376785    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8kpvjt" event={"ID":"fb0f223b-851b-438d-b981-4d0d79ccfe64","Type":"ContainerStarted","Data":"d8af7d35e826e707193f0ce41ef2a5518f9ec705abb2a217e7bfc3f2700f0273"}
Oct 30 22:02:16.382245 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:16.382157    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8kpvjt" event={"ID":"fb0f223b-851b-438d-b981-4d0d79ccfe64","Type":"ContainerStarted","Data":"9185eed6f49bfe1a802f8f30b3e47c55635cec7dc2211b08628a0a793df279e1"}
Oct 30 22:02:16.494960 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:16.494905    5510 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8kpvjt" podStartSLOduration=1.813850313 podStartE2EDuration="2.49489067s" totalImagesPullingTime="681.040357ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:02:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:02:16.494576674 +0000 UTC m=+717.055386679" watchObservedRunningTime="2026-10-30 22:02:16.49489067 +0000 UTC m=+717.055700676"
Oct 30 22:02:16.982181 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:16.982133    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8kpvjt"
Oct 30 22:02:17.640941 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:17.640906    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-zz2gk"
Oct 30 22:02:17.653490 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:17.653460    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-zz2gk"
Oct 30 22:02:18.402439 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:18.402412    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-zz2gk"
Oct 30 22:02:21.267358 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:21.267323    5510 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-49rg2"]
Oct 30 22:02:21.270106 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:21.270086    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-49rg2"
Oct 30 22:02:21.320474 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:21.320446    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-v8cwr\""
Oct 30 22:02:21.370371 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:21.370339    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-49rg2"]
Oct 30 22:02:21.393961 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:21.393932    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f98cd426-fb0e-433e-97ff-3fc95c44239b-workload-certs\") pod \"maas-default-gateway-openshift-default-85544d4cd7-49rg2\" (UID: \"f98cd426-fb0e-433e-97ff-3fc95c44239b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-49rg2"
Oct 30 22:02:21.394131 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:21.393965    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f98cd426-fb0e-433e-97ff-3fc95c44239b-istio-podinfo\") pod \"maas-default-gateway-openshift-default-85544d4cd7-49rg2\" (UID: \"f98cd426-fb0e-433e-97ff-3fc95c44239b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-49rg2"
Oct 30 22:02:21.394131 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:21.394043    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f98cd426-fb0e-433e-97ff-3fc95c44239b-istio-envoy\") pod \"maas-default-gateway-openshift-default-85544d4cd7-49rg2\" (UID: \"f98cd426-fb0e-433e-97ff-3fc95c44239b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-49rg2"
Oct 30 22:02:21.394131 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:21.394086    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f98cd426-fb0e-433e-97ff-3fc95c44239b-istio-data\") pod \"maas-default-gateway-openshift-default-85544d4cd7-49rg2\" (UID: \"f98cd426-fb0e-433e-97ff-3fc95c44239b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-49rg2"
Oct 30 22:02:21.394131 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:21.394104    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f98cd426-fb0e-433e-97ff-3fc95c44239b-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-85544d4cd7-49rg2\" (UID: \"f98cd426-fb0e-433e-97ff-3fc95c44239b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-49rg2"
Oct 30 22:02:21.394348 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:21.394217    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6fj9h\" (UniqueName: \"kubernetes.io/projected/f98cd426-fb0e-433e-97ff-3fc95c44239b-kube-api-access-6fj9h\") pod \"maas-default-gateway-openshift-default-85544d4cd7-49rg2\" (UID: \"f98cd426-fb0e-433e-97ff-3fc95c44239b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-49rg2"
Oct 30 22:02:21.394348 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:21.394279    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f98cd426-fb0e-433e-97ff-3fc95c44239b-istio-token\") pod \"maas-default-gateway-openshift-default-85544d4cd7-49rg2\" (UID: \"f98cd426-fb0e-433e-97ff-3fc95c44239b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-49rg2"
Oct 30 22:02:21.394348 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:21.394329    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f98cd426-fb0e-433e-97ff-3fc95c44239b-credential-socket\") pod \"maas-default-gateway-openshift-default-85544d4cd7-49rg2\" (UID: \"f98cd426-fb0e-433e-97ff-3fc95c44239b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-49rg2"
Oct 30 22:02:21.394448 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:21.394403    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f98cd426-fb0e-433e-97ff-3fc95c44239b-workload-socket\") pod \"maas-default-gateway-openshift-default-85544d4cd7-49rg2\" (UID: \"f98cd426-fb0e-433e-97ff-3fc95c44239b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-49rg2"
Oct 30 22:02:21.495608 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:21.495572    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f98cd426-fb0e-433e-97ff-3fc95c44239b-istio-data\") pod \"maas-default-gateway-openshift-default-85544d4cd7-49rg2\" (UID: \"f98cd426-fb0e-433e-97ff-3fc95c44239b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-49rg2"
Oct 30 22:02:21.495608 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:21.495608    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f98cd426-fb0e-433e-97ff-3fc95c44239b-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-85544d4cd7-49rg2\" (UID: \"f98cd426-fb0e-433e-97ff-3fc95c44239b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-49rg2"
Oct 30 22:02:21.495894 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:21.495629    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6fj9h\" (UniqueName: \"kubernetes.io/projected/f98cd426-fb0e-433e-97ff-3fc95c44239b-kube-api-access-6fj9h\") pod \"maas-default-gateway-openshift-default-85544d4cd7-49rg2\" (UID: \"f98cd426-fb0e-433e-97ff-3fc95c44239b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-49rg2"
Oct 30 22:02:21.495894 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:21.495767    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f98cd426-fb0e-433e-97ff-3fc95c44239b-istio-token\") pod \"maas-default-gateway-openshift-default-85544d4cd7-49rg2\" (UID: \"f98cd426-fb0e-433e-97ff-3fc95c44239b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-49rg2"
Oct 30 22:02:21.495894 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:21.495812    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f98cd426-fb0e-433e-97ff-3fc95c44239b-credential-socket\") pod \"maas-default-gateway-openshift-default-85544d4cd7-49rg2\" (UID: \"f98cd426-fb0e-433e-97ff-3fc95c44239b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-49rg2"
Oct 30 22:02:21.495894 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:21.495847    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f98cd426-fb0e-433e-97ff-3fc95c44239b-workload-socket\") pod \"maas-default-gateway-openshift-default-85544d4cd7-49rg2\" (UID: \"f98cd426-fb0e-433e-97ff-3fc95c44239b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-49rg2"
Oct 30 22:02:21.495894 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:21.495870    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f98cd426-fb0e-433e-97ff-3fc95c44239b-workload-certs\") pod \"maas-default-gateway-openshift-default-85544d4cd7-49rg2\" (UID: \"f98cd426-fb0e-433e-97ff-3fc95c44239b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-49rg2"
Oct 30 22:02:21.495894 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:21.495891    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f98cd426-fb0e-433e-97ff-3fc95c44239b-istio-podinfo\") pod \"maas-default-gateway-openshift-default-85544d4cd7-49rg2\" (UID: \"f98cd426-fb0e-433e-97ff-3fc95c44239b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-49rg2"
Oct 30 22:02:21.496172 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:21.495957    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f98cd426-fb0e-433e-97ff-3fc95c44239b-istio-envoy\") pod \"maas-default-gateway-openshift-default-85544d4cd7-49rg2\" (UID: \"f98cd426-fb0e-433e-97ff-3fc95c44239b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-49rg2"
Oct 30 22:02:21.496172 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:21.496077    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f98cd426-fb0e-433e-97ff-3fc95c44239b-istio-data\") pod \"maas-default-gateway-openshift-default-85544d4cd7-49rg2\" (UID: \"f98cd426-fb0e-433e-97ff-3fc95c44239b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-49rg2"
Oct 30 22:02:21.496301 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:21.496276    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f98cd426-fb0e-433e-97ff-3fc95c44239b-credential-socket\") pod \"maas-default-gateway-openshift-default-85544d4cd7-49rg2\" (UID: \"f98cd426-fb0e-433e-97ff-3fc95c44239b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-49rg2"
Oct 30 22:02:21.496367 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:21.496291    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f98cd426-fb0e-433e-97ff-3fc95c44239b-workload-certs\") pod \"maas-default-gateway-openshift-default-85544d4cd7-49rg2\" (UID: \"f98cd426-fb0e-433e-97ff-3fc95c44239b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-49rg2"
Oct 30 22:02:21.496428 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:21.496363    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f98cd426-fb0e-433e-97ff-3fc95c44239b-workload-socket\") pod \"maas-default-gateway-openshift-default-85544d4cd7-49rg2\" (UID: \"f98cd426-fb0e-433e-97ff-3fc95c44239b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-49rg2"
Oct 30 22:02:21.496479 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:21.496464    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f98cd426-fb0e-433e-97ff-3fc95c44239b-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-85544d4cd7-49rg2\" (UID: \"f98cd426-fb0e-433e-97ff-3fc95c44239b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-49rg2"
Oct 30 22:02:21.498199 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:21.498177    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f98cd426-fb0e-433e-97ff-3fc95c44239b-istio-envoy\") pod \"maas-default-gateway-openshift-default-85544d4cd7-49rg2\" (UID: \"f98cd426-fb0e-433e-97ff-3fc95c44239b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-49rg2"
Oct 30 22:02:21.498357 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:21.498339    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f98cd426-fb0e-433e-97ff-3fc95c44239b-istio-podinfo\") pod \"maas-default-gateway-openshift-default-85544d4cd7-49rg2\" (UID: \"f98cd426-fb0e-433e-97ff-3fc95c44239b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-49rg2"
Oct 30 22:02:21.554213 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:21.554141    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f98cd426-fb0e-433e-97ff-3fc95c44239b-istio-token\") pod \"maas-default-gateway-openshift-default-85544d4cd7-49rg2\" (UID: \"f98cd426-fb0e-433e-97ff-3fc95c44239b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-49rg2"
Oct 30 22:02:21.554342 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:21.554303    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6fj9h\" (UniqueName: \"kubernetes.io/projected/f98cd426-fb0e-433e-97ff-3fc95c44239b-kube-api-access-6fj9h\") pod \"maas-default-gateway-openshift-default-85544d4cd7-49rg2\" (UID: \"f98cd426-fb0e-433e-97ff-3fc95c44239b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-49rg2"
Oct 30 22:02:21.581058 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:21.581026    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-49rg2"
Oct 30 22:02:21.728407 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:21.728375    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:02:22.022302 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:22.022277    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-49rg2"]
Oct 30 22:02:22.024916 ip-10-0-136-210 kubenswrapper[5510]: W1030 22:02:22.024890    5510 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf98cd426_fb0e_433e_97ff_3fc95c44239b.slice/crio-47ffca3a29d53f35b833eb9755f14d620a884722760e3cab1787abc0a191c603 WatchSource:0}: Error finding container 47ffca3a29d53f35b833eb9755f14d620a884722760e3cab1787abc0a191c603: Status 404 returned error can't find the container with id 47ffca3a29d53f35b833eb9755f14d620a884722760e3cab1787abc0a191c603
Oct 30 22:02:22.410976 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:22.410890    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-49rg2" event={"ID":"f98cd426-fb0e-433e-97ff-3fc95c44239b","Type":"ContainerStarted","Data":"47ffca3a29d53f35b833eb9755f14d620a884722760e3cab1787abc0a191c603"}
Oct 30 22:02:24.982340 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:24.982305    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8kpvjt"
Oct 30 22:02:24.996250 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:24.996225    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8kpvjt"
Oct 30 22:02:25.439419 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:25.439384    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8kpvjt"
Oct 30 22:02:25.956747 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:25.956717    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:02:25.956829 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:25.956784    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:02:25.956829 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:25.956813    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:02:26.433218 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:26.433179    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-49rg2" event={"ID":"f98cd426-fb0e-433e-97ff-3fc95c44239b","Type":"ContainerStarted","Data":"5416194ed875923b42615c7bd6625c8bfb768b5c3ebcb42640ad0352fb756fcf"}
Oct 30 22:02:26.569682 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:26.551554    5510 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-49rg2" podStartSLOduration=1.6237542249999999 podStartE2EDuration="5.551533565s" totalImagesPullingTime="3.92777934s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:02:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:02:26.550122437 +0000 UTC m=+727.110932468" watchObservedRunningTime="2026-10-30 22:02:26.551533565 +0000 UTC m=+727.112343572"
Oct 30 22:02:26.581927 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:26.581889    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-49rg2"
Oct 30 22:02:27.583329 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:27.583295    5510 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-85544d4cd7-49rg2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" start-of-body=
Oct 30 22:02:27.583721 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:27.583348    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-49rg2" podUID="f98cd426-fb0e-433e-97ff-3fc95c44239b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused"
Oct 30 22:02:28.581544 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:28.581506    5510 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-85544d4cd7-49rg2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" start-of-body=
Oct 30 22:02:28.581737 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:28.581564    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-49rg2" podUID="f98cd426-fb0e-433e-97ff-3fc95c44239b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused"
Oct 30 22:02:29.582175 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:29.582144    5510 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-85544d4cd7-49rg2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" start-of-body=
Oct 30 22:02:29.582563 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:29.582209    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-49rg2" podUID="f98cd426-fb0e-433e-97ff-3fc95c44239b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused"
Oct 30 22:02:30.582166 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:30.582129    5510 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-85544d4cd7-49rg2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" start-of-body=
Oct 30 22:02:30.582380 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:30.582192    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-49rg2" podUID="f98cd426-fb0e-433e-97ff-3fc95c44239b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused"
Oct 30 22:02:31.582448 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:31.582412    5510 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-85544d4cd7-49rg2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" start-of-body=
Oct 30 22:02:31.582901 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:31.582461    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-49rg2" podUID="f98cd426-fb0e-433e-97ff-3fc95c44239b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused"
Oct 30 22:02:32.582471 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:32.582432    5510 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-85544d4cd7-49rg2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" start-of-body=
Oct 30 22:02:32.582999 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:32.582492    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-49rg2" podUID="f98cd426-fb0e-433e-97ff-3fc95c44239b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused"
Oct 30 22:02:34.167124 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:34.167088    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-49rg2"
Oct 30 22:02:34.167596 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:34.167481    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-49rg2"
Oct 30 22:02:34.168199 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:34.168181    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-49rg2"
Oct 30 22:02:38.418026 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:02:38.417983    5510 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6gjbf" containerName="extractor"
Oct 30 22:02:41.539212 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:41.539180    5510 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/ai-gateway-controller-66dd76bf94-84njd"]
Oct 30 22:02:41.541691 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:41.541649    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-66dd76bf94-84njd"
Oct 30 22:02:41.574226 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:41.574193    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:02:41.593337 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:41.593297    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dc619167-ea89-4732-8a8e-ec20d05c7940-metrics-certs\") pod \"ai-gateway-controller-66dd76bf94-84njd\" (UID: \"dc619167-ea89-4732-8a8e-ec20d05c7940\") " pod="opendatahub/ai-gateway-controller-66dd76bf94-84njd"
Oct 30 22:02:41.593508 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:41.593348    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wv5l4\" (UniqueName: \"kubernetes.io/projected/dc619167-ea89-4732-8a8e-ec20d05c7940-kube-api-access-wv5l4\") pod \"ai-gateway-controller-66dd76bf94-84njd\" (UID: \"dc619167-ea89-4732-8a8e-ec20d05c7940\") " pod="opendatahub/ai-gateway-controller-66dd76bf94-84njd"
Oct 30 22:02:41.637958 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:41.637919    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"ai-gateway-controller-metrics-tls\""
Oct 30 22:02:41.638147 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:41.637979    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"ai-gateway-controller-dockercfg-sf4vq\""
Oct 30 22:02:41.638147 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:41.638128    5510 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"ai-gateway-controller-parameters\""
Oct 30 22:02:41.639800 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:41.639778    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/ai-gateway-controller-66dd76bf94-84njd"]
Oct 30 22:02:41.693983 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:41.693958    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dc619167-ea89-4732-8a8e-ec20d05c7940-metrics-certs\") pod \"ai-gateway-controller-66dd76bf94-84njd\" (UID: \"dc619167-ea89-4732-8a8e-ec20d05c7940\") " pod="opendatahub/ai-gateway-controller-66dd76bf94-84njd"
Oct 30 22:02:41.694107 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:41.693998    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wv5l4\" (UniqueName: \"kubernetes.io/projected/dc619167-ea89-4732-8a8e-ec20d05c7940-kube-api-access-wv5l4\") pod \"ai-gateway-controller-66dd76bf94-84njd\" (UID: \"dc619167-ea89-4732-8a8e-ec20d05c7940\") " pod="opendatahub/ai-gateway-controller-66dd76bf94-84njd"
Oct 30 22:02:41.696388 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:41.696361    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dc619167-ea89-4732-8a8e-ec20d05c7940-metrics-certs\") pod \"ai-gateway-controller-66dd76bf94-84njd\" (UID: \"dc619167-ea89-4732-8a8e-ec20d05c7940\") " pod="opendatahub/ai-gateway-controller-66dd76bf94-84njd"
Oct 30 22:02:41.752989 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:41.752959    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wv5l4\" (UniqueName: \"kubernetes.io/projected/dc619167-ea89-4732-8a8e-ec20d05c7940-kube-api-access-wv5l4\") pod \"ai-gateway-controller-66dd76bf94-84njd\" (UID: \"dc619167-ea89-4732-8a8e-ec20d05c7940\") " pod="opendatahub/ai-gateway-controller-66dd76bf94-84njd"
Oct 30 22:02:41.773747 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:41.773717    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:02:41.853599 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:41.853518    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-66dd76bf94-84njd"
Oct 30 22:02:42.223624 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:42.223548    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/ai-gateway-controller-66dd76bf94-84njd"]
Oct 30 22:02:42.226574 ip-10-0-136-210 kubenswrapper[5510]: W1030 22:02:42.226541    5510 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc619167_ea89_4732_8a8e_ec20d05c7940.slice/crio-98b562fb14668862b2ec1f1b6bb9614626fc0239d84279360d74bc72457ae4d9 WatchSource:0}: Error finding container 98b562fb14668862b2ec1f1b6bb9614626fc0239d84279360d74bc72457ae4d9: Status 404 returned error can't find the container with id 98b562fb14668862b2ec1f1b6bb9614626fc0239d84279360d74bc72457ae4d9
Oct 30 22:02:42.540795 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:42.540753    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-66dd76bf94-84njd" event={"ID":"dc619167-ea89-4732-8a8e-ec20d05c7940","Type":"ContainerStarted","Data":"98b562fb14668862b2ec1f1b6bb9614626fc0239d84279360d74bc72457ae4d9"}
Oct 30 22:02:44.552951 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:44.552918    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-66dd76bf94-84njd" event={"ID":"dc619167-ea89-4732-8a8e-ec20d05c7940","Type":"ContainerStarted","Data":"632ddd7a489dfb65532726ce40e33a82f30b68b80c5c1d9b19da6014a0f152c3"}
Oct 30 22:02:44.553401 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:44.553012    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/ai-gateway-controller-66dd76bf94-84njd"
Oct 30 22:02:44.665296 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:44.665180    5510 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/ai-gateway-controller-66dd76bf94-84njd" podStartSLOduration=1.5613356440000001 podStartE2EDuration="3.665163894s" totalImagesPullingTime="2.10382825s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:02:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:02:44.664180125 +0000 UTC m=+745.224990130" watchObservedRunningTime="2026-10-30 22:02:44.665163894 +0000 UTC m=+745.225973900"
Oct 30 22:02:48.201927 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:48.201890    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:02:48.434476 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:48.434435    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:02:48.579675 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:48.579633    5510 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7b98bb678c-slj64"]
Oct 30 22:02:48.582623 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:48.582601    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7b98bb678c-slj64"
Oct 30 22:02:48.638580 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:48.638549    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-sk4hf\""
Oct 30 22:02:48.638960 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:48.638925    5510 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Oct 30 22:02:48.643692 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:48.643648    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7b98bb678c-slj64"]
Oct 30 22:02:48.658908 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:48.658876    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m7ns4\" (UniqueName: \"kubernetes.io/projected/6a38438c-669a-4bb0-af27-0d7b661d5ba0-kube-api-access-m7ns4\") pod \"payload-pre-processing-7b98bb678c-slj64\" (UID: \"6a38438c-669a-4bb0-af27-0d7b661d5ba0\") " pod="openshift-ingress/payload-pre-processing-7b98bb678c-slj64"
Oct 30 22:02:48.659037 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:48.658928    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6a38438c-669a-4bb0-af27-0d7b661d5ba0-plugins-config-volume\") pod \"payload-pre-processing-7b98bb678c-slj64\" (UID: \"6a38438c-669a-4bb0-af27-0d7b661d5ba0\") " pod="openshift-ingress/payload-pre-processing-7b98bb678c-slj64"
Oct 30 22:02:48.740762 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:48.740728    5510 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-55fbbbf9dc-d72tk"]
Oct 30 22:02:48.743189 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:48.743164    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-55fbbbf9dc-d72tk"
Oct 30 22:02:48.760434 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:48.760403    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vhjq7\" (UniqueName: \"kubernetes.io/projected/c08b8a69-8878-4042-82ee-d9eed9bcb960-kube-api-access-vhjq7\") pod \"payload-processing-55fbbbf9dc-d72tk\" (UID: \"c08b8a69-8878-4042-82ee-d9eed9bcb960\") " pod="openshift-ingress/payload-processing-55fbbbf9dc-d72tk"
Oct 30 22:02:48.760565 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:48.760440    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6a38438c-669a-4bb0-af27-0d7b661d5ba0-plugins-config-volume\") pod \"payload-pre-processing-7b98bb678c-slj64\" (UID: \"6a38438c-669a-4bb0-af27-0d7b661d5ba0\") " pod="openshift-ingress/payload-pre-processing-7b98bb678c-slj64"
Oct 30 22:02:48.760606 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:48.760567    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c08b8a69-8878-4042-82ee-d9eed9bcb960-plugins-config-volume\") pod \"payload-processing-55fbbbf9dc-d72tk\" (UID: \"c08b8a69-8878-4042-82ee-d9eed9bcb960\") " pod="openshift-ingress/payload-processing-55fbbbf9dc-d72tk"
Oct 30 22:02:48.760606 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:48.760592    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m7ns4\" (UniqueName: \"kubernetes.io/projected/6a38438c-669a-4bb0-af27-0d7b661d5ba0-kube-api-access-m7ns4\") pod \"payload-pre-processing-7b98bb678c-slj64\" (UID: \"6a38438c-669a-4bb0-af27-0d7b661d5ba0\") " pod="openshift-ingress/payload-pre-processing-7b98bb678c-slj64"
Oct 30 22:02:48.761091 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:48.761069    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6a38438c-669a-4bb0-af27-0d7b661d5ba0-plugins-config-volume\") pod \"payload-pre-processing-7b98bb678c-slj64\" (UID: \"6a38438c-669a-4bb0-af27-0d7b661d5ba0\") " pod="openshift-ingress/payload-pre-processing-7b98bb678c-slj64"
Oct 30 22:02:48.807598 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:48.807555    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-55fbbbf9dc-d72tk"]
Oct 30 22:02:48.821446 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:48.821410    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m7ns4\" (UniqueName: \"kubernetes.io/projected/6a38438c-669a-4bb0-af27-0d7b661d5ba0-kube-api-access-m7ns4\") pod \"payload-pre-processing-7b98bb678c-slj64\" (UID: \"6a38438c-669a-4bb0-af27-0d7b661d5ba0\") " pod="openshift-ingress/payload-pre-processing-7b98bb678c-slj64"
Oct 30 22:02:48.861416 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:48.861329    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c08b8a69-8878-4042-82ee-d9eed9bcb960-plugins-config-volume\") pod \"payload-processing-55fbbbf9dc-d72tk\" (UID: \"c08b8a69-8878-4042-82ee-d9eed9bcb960\") " pod="openshift-ingress/payload-processing-55fbbbf9dc-d72tk"
Oct 30 22:02:48.861416 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:48.861391    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vhjq7\" (UniqueName: \"kubernetes.io/projected/c08b8a69-8878-4042-82ee-d9eed9bcb960-kube-api-access-vhjq7\") pod \"payload-processing-55fbbbf9dc-d72tk\" (UID: \"c08b8a69-8878-4042-82ee-d9eed9bcb960\") " pod="openshift-ingress/payload-processing-55fbbbf9dc-d72tk"
Oct 30 22:02:48.862402 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:48.862374    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c08b8a69-8878-4042-82ee-d9eed9bcb960-plugins-config-volume\") pod \"payload-processing-55fbbbf9dc-d72tk\" (UID: \"c08b8a69-8878-4042-82ee-d9eed9bcb960\") " pod="openshift-ingress/payload-processing-55fbbbf9dc-d72tk"
Oct 30 22:02:48.894236 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:48.894204    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7b98bb678c-slj64"
Oct 30 22:02:48.919942 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:48.919903    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vhjq7\" (UniqueName: \"kubernetes.io/projected/c08b8a69-8878-4042-82ee-d9eed9bcb960-kube-api-access-vhjq7\") pod \"payload-processing-55fbbbf9dc-d72tk\" (UID: \"c08b8a69-8878-4042-82ee-d9eed9bcb960\") " pod="openshift-ingress/payload-processing-55fbbbf9dc-d72tk"
Oct 30 22:02:49.055652 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:49.055617    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-55fbbbf9dc-d72tk"
Oct 30 22:02:49.265979 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:49.265943    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7b98bb678c-slj64"]
Oct 30 22:02:49.269648 ip-10-0-136-210 kubenswrapper[5510]: W1030 22:02:49.269621    5510 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a38438c_669a_4bb0_af27_0d7b661d5ba0.slice/crio-af827731c2cc232ca0c14a5bc93270d5ff57f683f6d281dde9d598f5eb419ff7 WatchSource:0}: Error finding container af827731c2cc232ca0c14a5bc93270d5ff57f683f6d281dde9d598f5eb419ff7: Status 404 returned error can't find the container with id af827731c2cc232ca0c14a5bc93270d5ff57f683f6d281dde9d598f5eb419ff7
Oct 30 22:02:49.425961 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:49.425930    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-55fbbbf9dc-d72tk"]
Oct 30 22:02:49.429373 ip-10-0-136-210 kubenswrapper[5510]: W1030 22:02:49.429345    5510 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc08b8a69_8878_4042_82ee_d9eed9bcb960.slice/crio-8d2fb87bfcacbd558bfe892af4239d0abf480fcc0cac8c96504bc57953e530d5 WatchSource:0}: Error finding container 8d2fb87bfcacbd558bfe892af4239d0abf480fcc0cac8c96504bc57953e530d5: Status 404 returned error can't find the container with id 8d2fb87bfcacbd558bfe892af4239d0abf480fcc0cac8c96504bc57953e530d5
Oct 30 22:02:49.581869 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:49.581822    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7b98bb678c-slj64" event={"ID":"6a38438c-669a-4bb0-af27-0d7b661d5ba0","Type":"ContainerStarted","Data":"af827731c2cc232ca0c14a5bc93270d5ff57f683f6d281dde9d598f5eb419ff7"}
Oct 30 22:02:49.583256 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:49.583218    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-55fbbbf9dc-d72tk" event={"ID":"c08b8a69-8878-4042-82ee-d9eed9bcb960","Type":"ContainerStarted","Data":"8d2fb87bfcacbd558bfe892af4239d0abf480fcc0cac8c96504bc57953e530d5"}
Oct 30 22:02:52.601258 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:52.601224    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-55fbbbf9dc-d72tk" event={"ID":"c08b8a69-8878-4042-82ee-d9eed9bcb960","Type":"ContainerStarted","Data":"53f9b9d6b73b7a8d71e964c2a7e82caa9d4745771e4b319be3b26b17f8d1de20"}
Oct 30 22:02:52.601697 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:52.601368    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-55fbbbf9dc-d72tk"
Oct 30 22:02:52.602866 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:52.602844    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7b98bb678c-slj64" event={"ID":"6a38438c-669a-4bb0-af27-0d7b661d5ba0","Type":"ContainerStarted","Data":"160225793d4878cf8fcc8908471b688e3207f789019212724a88375c8cace997"}
Oct 30 22:02:52.602981 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:52.602915    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7b98bb678c-slj64"
Oct 30 22:02:52.842880 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:52.842833    5510 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7b98bb678c-slj64" podStartSLOduration=1.7290003619999998 podStartE2EDuration="4.842821417s" totalImagesPullingTime="3.113821055s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:02:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:02:52.84218424 +0000 UTC m=+753.402994245" watchObservedRunningTime="2026-10-30 22:02:52.842821417 +0000 UTC m=+753.403631400"
Oct 30 22:02:52.843604 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:52.843580    5510 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-55fbbbf9dc-d72tk" podStartSLOduration=1.896643444 podStartE2EDuration="4.843572043s" totalImagesPullingTime="2.946928599s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:02:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:02:52.72918721 +0000 UTC m=+753.289997215" watchObservedRunningTime="2026-10-30 22:02:52.843572043 +0000 UTC m=+753.404382103"
Oct 30 22:02:55.559283 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:55.559255    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/ai-gateway-controller-66dd76bf94-84njd"
Oct 30 22:02:58.436157 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:58.436117    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:02:58.622784 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:02:58.622744    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:03:03.610399 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:03.610367    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-55fbbbf9dc-d72tk"
Oct 30 22:03:03.610824 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:03.610436    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7b98bb678c-slj64"
Oct 30 22:03:08.419295 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:03:08.419254    5510 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6gjbf" containerName="extractor"
Oct 30 22:03:10.887693 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:10.887644    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/ai-gateway-controller-66dd76bf94-84njd"]
Oct 30 22:03:10.888112 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:10.887876    5510 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/ai-gateway-controller-66dd76bf94-84njd" podUID="dc619167-ea89-4732-8a8e-ec20d05c7940" containerName="manager" containerID="cri-o://632ddd7a489dfb65532726ce40e33a82f30b68b80c5c1d9b19da6014a0f152c3" gracePeriod=30
Oct 30 22:03:11.143299 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:11.143238    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-66dd76bf94-84njd"
Oct 30 22:03:11.288397 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:11.288367    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/dc619167-ea89-4732-8a8e-ec20d05c7940-metrics-certs\" (UniqueName: \"kubernetes.io/secret/dc619167-ea89-4732-8a8e-ec20d05c7940-metrics-certs\") pod \"dc619167-ea89-4732-8a8e-ec20d05c7940\" (UID: \"dc619167-ea89-4732-8a8e-ec20d05c7940\") "
Oct 30 22:03:11.288567 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:11.288482    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/dc619167-ea89-4732-8a8e-ec20d05c7940-kube-api-access-wv5l4\" (UniqueName: \"kubernetes.io/projected/dc619167-ea89-4732-8a8e-ec20d05c7940-kube-api-access-wv5l4\") pod \"dc619167-ea89-4732-8a8e-ec20d05c7940\" (UID: \"dc619167-ea89-4732-8a8e-ec20d05c7940\") "
Oct 30 22:03:11.290565 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:11.290533    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dc619167-ea89-4732-8a8e-ec20d05c7940-kube-api-access-wv5l4" pod "dc619167-ea89-4732-8a8e-ec20d05c7940" (UID: "dc619167-ea89-4732-8a8e-ec20d05c7940"). InnerVolumeSpecName "kube-api-access-wv5l4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:03:11.290707 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:11.290612    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dc619167-ea89-4732-8a8e-ec20d05c7940-metrics-certs" pod "dc619167-ea89-4732-8a8e-ec20d05c7940" (UID: "dc619167-ea89-4732-8a8e-ec20d05c7940"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:03:11.390139 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:11.390105    5510 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wv5l4\" (UniqueName: \"kubernetes.io/projected/dc619167-ea89-4732-8a8e-ec20d05c7940-kube-api-access-wv5l4\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:03:11.390139 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:11.390134    5510 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dc619167-ea89-4732-8a8e-ec20d05c7940-metrics-certs\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:03:11.690231 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:11.690201    5510 generic.go:356] "Generic (PLEG): container finished" podID="dc619167-ea89-4732-8a8e-ec20d05c7940" containerID="632ddd7a489dfb65532726ce40e33a82f30b68b80c5c1d9b19da6014a0f152c3" exitCode=0
Oct 30 22:03:11.690396 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:11.690248    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-66dd76bf94-84njd"
Oct 30 22:03:11.690396 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:11.690261    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-66dd76bf94-84njd" event={"ID":"dc619167-ea89-4732-8a8e-ec20d05c7940","Type":"ContainerDied","Data":"632ddd7a489dfb65532726ce40e33a82f30b68b80c5c1d9b19da6014a0f152c3"}
Oct 30 22:03:11.690396 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:11.690302    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-66dd76bf94-84njd" event={"ID":"dc619167-ea89-4732-8a8e-ec20d05c7940","Type":"ContainerDied","Data":"98b562fb14668862b2ec1f1b6bb9614626fc0239d84279360d74bc72457ae4d9"}
Oct 30 22:03:11.690396 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:11.690322    5510 scope.go:122] "RemoveContainer" containerID="632ddd7a489dfb65532726ce40e33a82f30b68b80c5c1d9b19da6014a0f152c3"
Oct 30 22:03:11.698799 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:11.698780    5510 scope.go:122] "RemoveContainer" containerID="632ddd7a489dfb65532726ce40e33a82f30b68b80c5c1d9b19da6014a0f152c3"
Oct 30 22:03:11.699063 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:03:11.699041    5510 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"632ddd7a489dfb65532726ce40e33a82f30b68b80c5c1d9b19da6014a0f152c3\": container with ID starting with 632ddd7a489dfb65532726ce40e33a82f30b68b80c5c1d9b19da6014a0f152c3 not found: ID does not exist" containerID="632ddd7a489dfb65532726ce40e33a82f30b68b80c5c1d9b19da6014a0f152c3"
Oct 30 22:03:11.699166 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:11.699075    5510 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"632ddd7a489dfb65532726ce40e33a82f30b68b80c5c1d9b19da6014a0f152c3"} err="failed to get container status \"632ddd7a489dfb65532726ce40e33a82f30b68b80c5c1d9b19da6014a0f152c3\": rpc error: code = NotFound desc = could not find container \"632ddd7a489dfb65532726ce40e33a82f30b68b80c5c1d9b19da6014a0f152c3\": container with ID starting with 632ddd7a489dfb65532726ce40e33a82f30b68b80c5c1d9b19da6014a0f152c3 not found: ID does not exist"
Oct 30 22:03:11.845182 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:11.845106    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/ai-gateway-controller-66dd76bf94-84njd"]
Oct 30 22:03:11.891741 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:11.891706    5510 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/ai-gateway-controller-66dd76bf94-84njd"]
Oct 30 22:03:12.675613 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:12.675574    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:03:13.199007 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:13.198972    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:03:13.551927 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:13.551893    5510 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="dc619167-ea89-4732-8a8e-ec20d05c7940" path="/var/lib/kubelet/pods/dc619167-ea89-4732-8a8e-ec20d05c7940/volumes"
Oct 30 22:03:14.930580 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:14.930545    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:03:15.129766 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:15.129732    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:03:16.731324 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:16.731289    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:03:16.922011 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:16.921971    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:03:21.775119 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:21.775090    5510 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-fwfzh"]
Oct 30 22:03:21.775616 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:21.775504    5510 cpu_manager.go:414] "containerMap: removing container" podUID="dc619167-ea89-4732-8a8e-ec20d05c7940" containerName="manager"
Oct 30 22:03:21.775616 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:21.775518    5510 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="dc619167-ea89-4732-8a8e-ec20d05c7940" containerName="manager"
Oct 30 22:03:21.775731 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:21.775635    5510 memory_manager.go:365] "RemoveStaleState removing state" podUID="dc619167-ea89-4732-8a8e-ec20d05c7940" containerName="manager"
Oct 30 22:03:21.779182 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:21.779162    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-fwfzh"
Oct 30 22:03:21.821521 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:21.821491    5510 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-gbtm2"]
Oct 30 22:03:21.823896 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:21.823875    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-gbtm2"
Oct 30 22:03:21.876243 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:21.876204    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-56njb\" (UniqueName: \"kubernetes.io/projected/bbd4c2ea-a56a-4f1f-9e37-a77b71326592-kube-api-access-56njb\") pod \"maas-default-gateway-openshift-default-85544d4cd7-gbtm2\" (UID: \"bbd4c2ea-a56a-4f1f-9e37-a77b71326592\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-gbtm2"
Oct 30 22:03:21.876243 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:21.876245    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/edc8b181-2f27-423e-96c9-8057cf0d1851-istio-envoy\") pod \"maas-default-gateway-openshift-default-85544d4cd7-fwfzh\" (UID: \"edc8b181-2f27-423e-96c9-8057cf0d1851\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-fwfzh"
Oct 30 22:03:21.876463 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:21.876265    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/edc8b181-2f27-423e-96c9-8057cf0d1851-istio-podinfo\") pod \"maas-default-gateway-openshift-default-85544d4cd7-fwfzh\" (UID: \"edc8b181-2f27-423e-96c9-8057cf0d1851\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-fwfzh"
Oct 30 22:03:21.876463 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:21.876346    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bbd4c2ea-a56a-4f1f-9e37-a77b71326592-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-85544d4cd7-gbtm2\" (UID: \"bbd4c2ea-a56a-4f1f-9e37-a77b71326592\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-gbtm2"
Oct 30 22:03:21.876463 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:21.876392    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bbd4c2ea-a56a-4f1f-9e37-a77b71326592-workload-certs\") pod \"maas-default-gateway-openshift-default-85544d4cd7-gbtm2\" (UID: \"bbd4c2ea-a56a-4f1f-9e37-a77b71326592\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-gbtm2"
Oct 30 22:03:21.876463 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:21.876461    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bbd4c2ea-a56a-4f1f-9e37-a77b71326592-istio-data\") pod \"maas-default-gateway-openshift-default-85544d4cd7-gbtm2\" (UID: \"bbd4c2ea-a56a-4f1f-9e37-a77b71326592\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-gbtm2"
Oct 30 22:03:21.876595 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:21.876482    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bbd4c2ea-a56a-4f1f-9e37-a77b71326592-workload-socket\") pod \"maas-default-gateway-openshift-default-85544d4cd7-gbtm2\" (UID: \"bbd4c2ea-a56a-4f1f-9e37-a77b71326592\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-gbtm2"
Oct 30 22:03:21.876595 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:21.876502    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/edc8b181-2f27-423e-96c9-8057cf0d1851-workload-socket\") pod \"maas-default-gateway-openshift-default-85544d4cd7-fwfzh\" (UID: \"edc8b181-2f27-423e-96c9-8057cf0d1851\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-fwfzh"
Oct 30 22:03:21.876595 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:21.876552    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/edc8b181-2f27-423e-96c9-8057cf0d1851-workload-certs\") pod \"maas-default-gateway-openshift-default-85544d4cd7-fwfzh\" (UID: \"edc8b181-2f27-423e-96c9-8057cf0d1851\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-fwfzh"
Oct 30 22:03:21.876595 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:21.876588    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/edc8b181-2f27-423e-96c9-8057cf0d1851-credential-socket\") pod \"maas-default-gateway-openshift-default-85544d4cd7-fwfzh\" (UID: \"edc8b181-2f27-423e-96c9-8057cf0d1851\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-fwfzh"
Oct 30 22:03:21.876786 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:21.876703    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/edc8b181-2f27-423e-96c9-8057cf0d1851-istio-token\") pod \"maas-default-gateway-openshift-default-85544d4cd7-fwfzh\" (UID: \"edc8b181-2f27-423e-96c9-8057cf0d1851\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-fwfzh"
Oct 30 22:03:21.876786 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:21.876730    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bbd4c2ea-a56a-4f1f-9e37-a77b71326592-istio-envoy\") pod \"maas-default-gateway-openshift-default-85544d4cd7-gbtm2\" (UID: \"bbd4c2ea-a56a-4f1f-9e37-a77b71326592\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-gbtm2"
Oct 30 22:03:21.876786 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:21.876750    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bbd4c2ea-a56a-4f1f-9e37-a77b71326592-istio-podinfo\") pod \"maas-default-gateway-openshift-default-85544d4cd7-gbtm2\" (UID: \"bbd4c2ea-a56a-4f1f-9e37-a77b71326592\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-gbtm2"
Oct 30 22:03:21.876786 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:21.876773    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/edc8b181-2f27-423e-96c9-8057cf0d1851-istio-data\") pod \"maas-default-gateway-openshift-default-85544d4cd7-fwfzh\" (UID: \"edc8b181-2f27-423e-96c9-8057cf0d1851\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-fwfzh"
Oct 30 22:03:21.876905 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:21.876826    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/edc8b181-2f27-423e-96c9-8057cf0d1851-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-85544d4cd7-fwfzh\" (UID: \"edc8b181-2f27-423e-96c9-8057cf0d1851\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-fwfzh"
Oct 30 22:03:21.876905 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:21.876845    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6jw8w\" (UniqueName: \"kubernetes.io/projected/edc8b181-2f27-423e-96c9-8057cf0d1851-kube-api-access-6jw8w\") pod \"maas-default-gateway-openshift-default-85544d4cd7-fwfzh\" (UID: \"edc8b181-2f27-423e-96c9-8057cf0d1851\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-fwfzh"
Oct 30 22:03:21.876905 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:21.876861    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bbd4c2ea-a56a-4f1f-9e37-a77b71326592-credential-socket\") pod \"maas-default-gateway-openshift-default-85544d4cd7-gbtm2\" (UID: \"bbd4c2ea-a56a-4f1f-9e37-a77b71326592\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-gbtm2"
Oct 30 22:03:21.876905 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:21.876876    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bbd4c2ea-a56a-4f1f-9e37-a77b71326592-istio-token\") pod \"maas-default-gateway-openshift-default-85544d4cd7-gbtm2\" (UID: \"bbd4c2ea-a56a-4f1f-9e37-a77b71326592\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-gbtm2"
Oct 30 22:03:21.882260 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:21.882239    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-fwfzh"]
Oct 30 22:03:21.936486 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:21.936453    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-gbtm2"]
Oct 30 22:03:21.977725 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:21.977691    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bbd4c2ea-a56a-4f1f-9e37-a77b71326592-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-85544d4cd7-gbtm2\" (UID: \"bbd4c2ea-a56a-4f1f-9e37-a77b71326592\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-gbtm2"
Oct 30 22:03:21.977915 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:21.977742    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bbd4c2ea-a56a-4f1f-9e37-a77b71326592-workload-certs\") pod \"maas-default-gateway-openshift-default-85544d4cd7-gbtm2\" (UID: \"bbd4c2ea-a56a-4f1f-9e37-a77b71326592\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-gbtm2"
Oct 30 22:03:21.977915 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:21.977774    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bbd4c2ea-a56a-4f1f-9e37-a77b71326592-istio-data\") pod \"maas-default-gateway-openshift-default-85544d4cd7-gbtm2\" (UID: \"bbd4c2ea-a56a-4f1f-9e37-a77b71326592\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-gbtm2"
Oct 30 22:03:21.977915 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:21.977804    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bbd4c2ea-a56a-4f1f-9e37-a77b71326592-workload-socket\") pod \"maas-default-gateway-openshift-default-85544d4cd7-gbtm2\" (UID: \"bbd4c2ea-a56a-4f1f-9e37-a77b71326592\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-gbtm2"
Oct 30 22:03:21.977915 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:21.977825    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/edc8b181-2f27-423e-96c9-8057cf0d1851-workload-socket\") pod \"maas-default-gateway-openshift-default-85544d4cd7-fwfzh\" (UID: \"edc8b181-2f27-423e-96c9-8057cf0d1851\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-fwfzh"
Oct 30 22:03:21.978118 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:21.977974    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/edc8b181-2f27-423e-96c9-8057cf0d1851-workload-certs\") pod \"maas-default-gateway-openshift-default-85544d4cd7-fwfzh\" (UID: \"edc8b181-2f27-423e-96c9-8057cf0d1851\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-fwfzh"
Oct 30 22:03:21.978118 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:21.978025    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/edc8b181-2f27-423e-96c9-8057cf0d1851-credential-socket\") pod \"maas-default-gateway-openshift-default-85544d4cd7-fwfzh\" (UID: \"edc8b181-2f27-423e-96c9-8057cf0d1851\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-fwfzh"
Oct 30 22:03:21.978118 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:21.978065    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/edc8b181-2f27-423e-96c9-8057cf0d1851-istio-token\") pod \"maas-default-gateway-openshift-default-85544d4cd7-fwfzh\" (UID: \"edc8b181-2f27-423e-96c9-8057cf0d1851\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-fwfzh"
Oct 30 22:03:21.978118 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:21.978094    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bbd4c2ea-a56a-4f1f-9e37-a77b71326592-istio-envoy\") pod \"maas-default-gateway-openshift-default-85544d4cd7-gbtm2\" (UID: \"bbd4c2ea-a56a-4f1f-9e37-a77b71326592\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-gbtm2"
Oct 30 22:03:21.978336 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:21.978127    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bbd4c2ea-a56a-4f1f-9e37-a77b71326592-istio-podinfo\") pod \"maas-default-gateway-openshift-default-85544d4cd7-gbtm2\" (UID: \"bbd4c2ea-a56a-4f1f-9e37-a77b71326592\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-gbtm2"
Oct 30 22:03:21.978336 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:21.978152    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/edc8b181-2f27-423e-96c9-8057cf0d1851-istio-data\") pod \"maas-default-gateway-openshift-default-85544d4cd7-fwfzh\" (UID: \"edc8b181-2f27-423e-96c9-8057cf0d1851\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-fwfzh"
Oct 30 22:03:21.978336 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:21.978179    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/edc8b181-2f27-423e-96c9-8057cf0d1851-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-85544d4cd7-fwfzh\" (UID: \"edc8b181-2f27-423e-96c9-8057cf0d1851\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-fwfzh"
Oct 30 22:03:21.978336 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:21.978211    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6jw8w\" (UniqueName: \"kubernetes.io/projected/edc8b181-2f27-423e-96c9-8057cf0d1851-kube-api-access-6jw8w\") pod \"maas-default-gateway-openshift-default-85544d4cd7-fwfzh\" (UID: \"edc8b181-2f27-423e-96c9-8057cf0d1851\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-fwfzh"
Oct 30 22:03:21.978336 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:21.978226    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/edc8b181-2f27-423e-96c9-8057cf0d1851-workload-socket\") pod \"maas-default-gateway-openshift-default-85544d4cd7-fwfzh\" (UID: \"edc8b181-2f27-423e-96c9-8057cf0d1851\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-fwfzh"
Oct 30 22:03:21.978336 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:21.978239    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bbd4c2ea-a56a-4f1f-9e37-a77b71326592-credential-socket\") pod \"maas-default-gateway-openshift-default-85544d4cd7-gbtm2\" (UID: \"bbd4c2ea-a56a-4f1f-9e37-a77b71326592\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-gbtm2"
Oct 30 22:03:21.978336 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:21.978240    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bbd4c2ea-a56a-4f1f-9e37-a77b71326592-workload-certs\") pod \"maas-default-gateway-openshift-default-85544d4cd7-gbtm2\" (UID: \"bbd4c2ea-a56a-4f1f-9e37-a77b71326592\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-gbtm2"
Oct 30 22:03:21.978336 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:21.978267    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bbd4c2ea-a56a-4f1f-9e37-a77b71326592-workload-socket\") pod \"maas-default-gateway-openshift-default-85544d4cd7-gbtm2\" (UID: \"bbd4c2ea-a56a-4f1f-9e37-a77b71326592\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-gbtm2"
Oct 30 22:03:21.978336 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:21.978276    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bbd4c2ea-a56a-4f1f-9e37-a77b71326592-istio-token\") pod \"maas-default-gateway-openshift-default-85544d4cd7-gbtm2\" (UID: \"bbd4c2ea-a56a-4f1f-9e37-a77b71326592\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-gbtm2"
Oct 30 22:03:21.978336 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:21.978315    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-56njb\" (UniqueName: \"kubernetes.io/projected/bbd4c2ea-a56a-4f1f-9e37-a77b71326592-kube-api-access-56njb\") pod \"maas-default-gateway-openshift-default-85544d4cd7-gbtm2\" (UID: \"bbd4c2ea-a56a-4f1f-9e37-a77b71326592\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-gbtm2"
Oct 30 22:03:21.978336 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:21.978329    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/edc8b181-2f27-423e-96c9-8057cf0d1851-credential-socket\") pod \"maas-default-gateway-openshift-default-85544d4cd7-fwfzh\" (UID: \"edc8b181-2f27-423e-96c9-8057cf0d1851\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-fwfzh"
Oct 30 22:03:21.978967 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:21.978346    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bbd4c2ea-a56a-4f1f-9e37-a77b71326592-istio-data\") pod \"maas-default-gateway-openshift-default-85544d4cd7-gbtm2\" (UID: \"bbd4c2ea-a56a-4f1f-9e37-a77b71326592\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-gbtm2"
Oct 30 22:03:21.978967 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:21.978356    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/edc8b181-2f27-423e-96c9-8057cf0d1851-istio-envoy\") pod \"maas-default-gateway-openshift-default-85544d4cd7-fwfzh\" (UID: \"edc8b181-2f27-423e-96c9-8057cf0d1851\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-fwfzh"
Oct 30 22:03:21.978967 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:21.978408    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/edc8b181-2f27-423e-96c9-8057cf0d1851-istio-podinfo\") pod \"maas-default-gateway-openshift-default-85544d4cd7-fwfzh\" (UID: \"edc8b181-2f27-423e-96c9-8057cf0d1851\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-fwfzh"
Oct 30 22:03:21.978967 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:21.978496    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bbd4c2ea-a56a-4f1f-9e37-a77b71326592-credential-socket\") pod \"maas-default-gateway-openshift-default-85544d4cd7-gbtm2\" (UID: \"bbd4c2ea-a56a-4f1f-9e37-a77b71326592\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-gbtm2"
Oct 30 22:03:21.978967 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:21.978553    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bbd4c2ea-a56a-4f1f-9e37-a77b71326592-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-85544d4cd7-gbtm2\" (UID: \"bbd4c2ea-a56a-4f1f-9e37-a77b71326592\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-gbtm2"
Oct 30 22:03:21.978967 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:21.978785    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/edc8b181-2f27-423e-96c9-8057cf0d1851-istio-data\") pod \"maas-default-gateway-openshift-default-85544d4cd7-fwfzh\" (UID: \"edc8b181-2f27-423e-96c9-8057cf0d1851\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-fwfzh"
Oct 30 22:03:21.979350 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:21.979122    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/edc8b181-2f27-423e-96c9-8057cf0d1851-workload-certs\") pod \"maas-default-gateway-openshift-default-85544d4cd7-fwfzh\" (UID: \"edc8b181-2f27-423e-96c9-8057cf0d1851\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-fwfzh"
Oct 30 22:03:21.979350 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:21.979179    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/edc8b181-2f27-423e-96c9-8057cf0d1851-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-85544d4cd7-fwfzh\" (UID: \"edc8b181-2f27-423e-96c9-8057cf0d1851\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-fwfzh"
Oct 30 22:03:21.980708 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:21.980675    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/edc8b181-2f27-423e-96c9-8057cf0d1851-istio-envoy\") pod \"maas-default-gateway-openshift-default-85544d4cd7-fwfzh\" (UID: \"edc8b181-2f27-423e-96c9-8057cf0d1851\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-fwfzh"
Oct 30 22:03:21.980848 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:21.980718    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bbd4c2ea-a56a-4f1f-9e37-a77b71326592-istio-envoy\") pod \"maas-default-gateway-openshift-default-85544d4cd7-gbtm2\" (UID: \"bbd4c2ea-a56a-4f1f-9e37-a77b71326592\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-gbtm2"
Oct 30 22:03:21.980960 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:21.980939    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bbd4c2ea-a56a-4f1f-9e37-a77b71326592-istio-podinfo\") pod \"maas-default-gateway-openshift-default-85544d4cd7-gbtm2\" (UID: \"bbd4c2ea-a56a-4f1f-9e37-a77b71326592\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-gbtm2"
Oct 30 22:03:21.981010 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:21.980943    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/edc8b181-2f27-423e-96c9-8057cf0d1851-istio-podinfo\") pod \"maas-default-gateway-openshift-default-85544d4cd7-fwfzh\" (UID: \"edc8b181-2f27-423e-96c9-8057cf0d1851\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-fwfzh"
Oct 30 22:03:22.038171 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:22.038104    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/edc8b181-2f27-423e-96c9-8057cf0d1851-istio-token\") pod \"maas-default-gateway-openshift-default-85544d4cd7-fwfzh\" (UID: \"edc8b181-2f27-423e-96c9-8057cf0d1851\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-fwfzh"
Oct 30 22:03:22.038315 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:22.038258    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bbd4c2ea-a56a-4f1f-9e37-a77b71326592-istio-token\") pod \"maas-default-gateway-openshift-default-85544d4cd7-gbtm2\" (UID: \"bbd4c2ea-a56a-4f1f-9e37-a77b71326592\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-gbtm2"
Oct 30 22:03:22.038459 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:22.038437    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-56njb\" (UniqueName: \"kubernetes.io/projected/bbd4c2ea-a56a-4f1f-9e37-a77b71326592-kube-api-access-56njb\") pod \"maas-default-gateway-openshift-default-85544d4cd7-gbtm2\" (UID: \"bbd4c2ea-a56a-4f1f-9e37-a77b71326592\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-gbtm2"
Oct 30 22:03:22.038496 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:22.038437    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6jw8w\" (UniqueName: \"kubernetes.io/projected/edc8b181-2f27-423e-96c9-8057cf0d1851-kube-api-access-6jw8w\") pod \"maas-default-gateway-openshift-default-85544d4cd7-fwfzh\" (UID: \"edc8b181-2f27-423e-96c9-8057cf0d1851\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-fwfzh"
Oct 30 22:03:22.050948 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:22.050909    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:03:22.090808 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:22.090776    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-fwfzh"
Oct 30 22:03:22.135868 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:22.135831    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-gbtm2"
Oct 30 22:03:22.507402 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:22.507322    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-fwfzh"]
Oct 30 22:03:22.543223 ip-10-0-136-210 kubenswrapper[5510]: W1030 22:03:22.543197    5510 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podedc8b181_2f27_423e_96c9_8057cf0d1851.slice/crio-9a4431fc0bee1bbbb52099c5fcb76a369b0cacb9b0ece04006be5ee0e6242d1b WatchSource:0}: Error finding container 9a4431fc0bee1bbbb52099c5fcb76a369b0cacb9b0ece04006be5ee0e6242d1b: Status 404 returned error can't find the container with id 9a4431fc0bee1bbbb52099c5fcb76a369b0cacb9b0ece04006be5ee0e6242d1b
Oct 30 22:03:22.545432 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:22.545394    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:03:22.545532 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:22.545467    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:03:22.545532 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:22.545496    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:03:22.559998 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:22.559977    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-gbtm2"]
Oct 30 22:03:22.564962 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:22.564938    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:03:22.565036 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:22.564995    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:03:22.565036 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:22.565022    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:03:22.746455 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:22.746419    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-gbtm2" event={"ID":"bbd4c2ea-a56a-4f1f-9e37-a77b71326592","Type":"ContainerStarted","Data":"64c20ea99cbed10d600e8716909cd88c2a1b548237e198e41ccdb42a5893a177"}
Oct 30 22:03:22.746637 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:22.746484    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-gbtm2" event={"ID":"bbd4c2ea-a56a-4f1f-9e37-a77b71326592","Type":"ContainerStarted","Data":"7196748ef622781fa141281059208545f43a3872f2305070c776455ce63debe5"}
Oct 30 22:03:22.748173 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:22.748141    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-fwfzh" event={"ID":"edc8b181-2f27-423e-96c9-8057cf0d1851","Type":"ContainerStarted","Data":"da0b4a000b09a3bc94f8d8890c9f3fafc8080fde6c9e24a051634312cd58141c"}
Oct 30 22:03:22.748301 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:22.748193    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-fwfzh" event={"ID":"edc8b181-2f27-423e-96c9-8057cf0d1851","Type":"ContainerStarted","Data":"9a4431fc0bee1bbbb52099c5fcb76a369b0cacb9b0ece04006be5ee0e6242d1b"}
Oct 30 22:03:22.862208 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:22.862144    5510 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-gbtm2" podStartSLOduration=1.862127673 podStartE2EDuration="1.862127673s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:03:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:03:22.861151416 +0000 UTC m=+783.421961403" watchObservedRunningTime="2026-10-30 22:03:22.862127673 +0000 UTC m=+783.422937676"
Oct 30 22:03:22.978789 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:22.978724    5510 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-fwfzh" podStartSLOduration=1.978697612 podStartE2EDuration="1.978697612s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:03:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:03:22.97714128 +0000 UTC m=+783.537951286" watchObservedRunningTime="2026-10-30 22:03:22.978697612 +0000 UTC m=+783.539507618"
Oct 30 22:03:23.095687 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:23.091571    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-fwfzh"
Oct 30 22:03:23.099890 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:23.099855    5510 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-85544d4cd7-fwfzh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body=
Oct 30 22:03:23.100144 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:23.100118    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-fwfzh" podUID="edc8b181-2f27-423e-96c9-8057cf0d1851" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused"
Oct 30 22:03:23.136017 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:23.135936    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-gbtm2"
Oct 30 22:03:23.137407 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:23.137384    5510 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-85544d4cd7-gbtm2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body=
Oct 30 22:03:23.137458 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:23.137436    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-gbtm2" podUID="bbd4c2ea-a56a-4f1f-9e37-a77b71326592" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused"
Oct 30 22:03:24.092599 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:24.092122    5510 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-85544d4cd7-fwfzh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body=
Oct 30 22:03:24.092599 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:24.092194    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-fwfzh" podUID="edc8b181-2f27-423e-96c9-8057cf0d1851" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused"
Oct 30 22:03:24.136533 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:24.136502    5510 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-85544d4cd7-gbtm2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body=
Oct 30 22:03:24.136743 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:24.136555    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-gbtm2" podUID="bbd4c2ea-a56a-4f1f-9e37-a77b71326592" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused"
Oct 30 22:03:25.093695 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:25.093076    5510 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-85544d4cd7-fwfzh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body=
Oct 30 22:03:25.093695 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:25.093186    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-fwfzh" podUID="edc8b181-2f27-423e-96c9-8057cf0d1851" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused"
Oct 30 22:03:25.137129 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:25.137093    5510 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-85544d4cd7-gbtm2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body=
Oct 30 22:03:25.137305 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:25.137150    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-gbtm2" podUID="bbd4c2ea-a56a-4f1f-9e37-a77b71326592" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused"
Oct 30 22:03:26.094127 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:26.094081    5510 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-85544d4cd7-fwfzh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body=
Oct 30 22:03:26.094635 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:26.094166    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-fwfzh" podUID="edc8b181-2f27-423e-96c9-8057cf0d1851" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused"
Oct 30 22:03:26.136947 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:26.136907    5510 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-85544d4cd7-gbtm2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body=
Oct 30 22:03:26.137103 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:26.136970    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-gbtm2" podUID="bbd4c2ea-a56a-4f1f-9e37-a77b71326592" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused"
Oct 30 22:03:27.092108 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:27.092066    5510 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-85544d4cd7-fwfzh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body=
Oct 30 22:03:27.092444 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:27.092413    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-fwfzh" podUID="edc8b181-2f27-423e-96c9-8057cf0d1851" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused"
Oct 30 22:03:27.136794 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:27.136759    5510 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-85544d4cd7-gbtm2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body=
Oct 30 22:03:27.137167 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:27.136814    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-gbtm2" podUID="bbd4c2ea-a56a-4f1f-9e37-a77b71326592" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused"
Oct 30 22:03:28.094130 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:28.094088    5510 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-85544d4cd7-fwfzh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body=
Oct 30 22:03:28.094346 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:28.094218    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-fwfzh" podUID="edc8b181-2f27-423e-96c9-8057cf0d1851" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused"
Oct 30 22:03:28.136618 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:28.136580    5510 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-85544d4cd7-gbtm2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body=
Oct 30 22:03:28.136805 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:28.136640    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-gbtm2" podUID="bbd4c2ea-a56a-4f1f-9e37-a77b71326592" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused"
Oct 30 22:03:28.495350 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:28.495268    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:03:29.091719 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:29.091683    5510 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-85544d4cd7-fwfzh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body=
Oct 30 22:03:29.091906 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:29.091742    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-fwfzh" podUID="edc8b181-2f27-423e-96c9-8057cf0d1851" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused"
Oct 30 22:03:29.137348 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:29.137309    5510 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-85544d4cd7-gbtm2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body=
Oct 30 22:03:29.137538 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:29.137370    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-gbtm2" podUID="bbd4c2ea-a56a-4f1f-9e37-a77b71326592" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused"
Oct 30 22:03:30.091744 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:30.091703    5510 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-85544d4cd7-fwfzh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body=
Oct 30 22:03:30.092225 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:30.091770    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-fwfzh" podUID="edc8b181-2f27-423e-96c9-8057cf0d1851" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused"
Oct 30 22:03:30.909116 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:30.909062    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-gbtm2"
Oct 30 22:03:30.910288 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:30.910261    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-gbtm2"
Oct 30 22:03:30.911031 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:30.911005    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-gbtm2"
Oct 30 22:03:31.095453 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:31.095422    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-fwfzh"
Oct 30 22:03:31.095839 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:31.095692    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-fwfzh"
Oct 30 22:03:31.096384 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:31.096367    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-fwfzh"
Oct 30 22:03:38.418984 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:03:38.418940    5510 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6gjbf" containerName="extractor"
Oct 30 22:03:41.748071 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:41.748035    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:03:41.967104 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:41.967075    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:03:42.442792 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:42.442753    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:03:42.606732 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:42.606689    5510 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-clqsv"]
Oct 30 22:03:42.609225 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:42.609203    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-clqsv"
Oct 30 22:03:42.658252 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:42.658216    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w5-c9dbe6-openshift-default-dockercfg-bpxxl\""
Oct 30 22:03:42.700160 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:42.700089    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-clqsv"]
Oct 30 22:03:42.705393 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:42.705369    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/21557bdf-26e4-4a0c-ad9f-50dcc391616b-workload-socket\") pod \"e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-clqsv\" (UID: \"21557bdf-26e4-4a0c-ad9f-50dcc391616b\") " pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-clqsv"
Oct 30 22:03:42.705516 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:42.705400    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/21557bdf-26e4-4a0c-ad9f-50dcc391616b-istiod-ca-cert\") pod \"e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-clqsv\" (UID: \"21557bdf-26e4-4a0c-ad9f-50dcc391616b\") " pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-clqsv"
Oct 30 22:03:42.705516 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:42.705418    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/21557bdf-26e4-4a0c-ad9f-50dcc391616b-istio-token\") pod \"e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-clqsv\" (UID: \"21557bdf-26e4-4a0c-ad9f-50dcc391616b\") " pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-clqsv"
Oct 30 22:03:42.705602 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:42.705566    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/21557bdf-26e4-4a0c-ad9f-50dcc391616b-workload-certs\") pod \"e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-clqsv\" (UID: \"21557bdf-26e4-4a0c-ad9f-50dcc391616b\") " pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-clqsv"
Oct 30 22:03:42.705727 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:42.705710    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/21557bdf-26e4-4a0c-ad9f-50dcc391616b-credential-socket\") pod \"e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-clqsv\" (UID: \"21557bdf-26e4-4a0c-ad9f-50dcc391616b\") " pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-clqsv"
Oct 30 22:03:42.705798 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:42.705743    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/21557bdf-26e4-4a0c-ad9f-50dcc391616b-istio-podinfo\") pod \"e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-clqsv\" (UID: \"21557bdf-26e4-4a0c-ad9f-50dcc391616b\") " pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-clqsv"
Oct 30 22:03:42.705849 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:42.705806    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kzxw8\" (UniqueName: \"kubernetes.io/projected/21557bdf-26e4-4a0c-ad9f-50dcc391616b-kube-api-access-kzxw8\") pod \"e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-clqsv\" (UID: \"21557bdf-26e4-4a0c-ad9f-50dcc391616b\") " pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-clqsv"
Oct 30 22:03:42.705849 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:42.705830    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/21557bdf-26e4-4a0c-ad9f-50dcc391616b-istio-data\") pod \"e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-clqsv\" (UID: \"21557bdf-26e4-4a0c-ad9f-50dcc391616b\") " pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-clqsv"
Oct 30 22:03:42.705963 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:42.705941    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/21557bdf-26e4-4a0c-ad9f-50dcc391616b-istio-envoy\") pod \"e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-clqsv\" (UID: \"21557bdf-26e4-4a0c-ad9f-50dcc391616b\") " pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-clqsv"
Oct 30 22:03:42.746538 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:42.746506    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:03:42.806816 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:42.806779    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/21557bdf-26e4-4a0c-ad9f-50dcc391616b-workload-socket\") pod \"e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-clqsv\" (UID: \"21557bdf-26e4-4a0c-ad9f-50dcc391616b\") " pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-clqsv"
Oct 30 22:03:42.806816 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:42.806816    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/21557bdf-26e4-4a0c-ad9f-50dcc391616b-istiod-ca-cert\") pod \"e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-clqsv\" (UID: \"21557bdf-26e4-4a0c-ad9f-50dcc391616b\") " pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-clqsv"
Oct 30 22:03:42.807349 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:42.806841    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/21557bdf-26e4-4a0c-ad9f-50dcc391616b-istio-token\") pod \"e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-clqsv\" (UID: \"21557bdf-26e4-4a0c-ad9f-50dcc391616b\") " pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-clqsv"
Oct 30 22:03:42.807349 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:42.806902    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/21557bdf-26e4-4a0c-ad9f-50dcc391616b-workload-certs\") pod \"e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-clqsv\" (UID: \"21557bdf-26e4-4a0c-ad9f-50dcc391616b\") " pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-clqsv"
Oct 30 22:03:42.807349 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:42.806932    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/21557bdf-26e4-4a0c-ad9f-50dcc391616b-credential-socket\") pod \"e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-clqsv\" (UID: \"21557bdf-26e4-4a0c-ad9f-50dcc391616b\") " pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-clqsv"
Oct 30 22:03:42.807349 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:42.806952    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/21557bdf-26e4-4a0c-ad9f-50dcc391616b-istio-podinfo\") pod \"e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-clqsv\" (UID: \"21557bdf-26e4-4a0c-ad9f-50dcc391616b\") " pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-clqsv"
Oct 30 22:03:42.807349 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:42.806980    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kzxw8\" (UniqueName: \"kubernetes.io/projected/21557bdf-26e4-4a0c-ad9f-50dcc391616b-kube-api-access-kzxw8\") pod \"e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-clqsv\" (UID: \"21557bdf-26e4-4a0c-ad9f-50dcc391616b\") " pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-clqsv"
Oct 30 22:03:42.807349 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:42.807004    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/21557bdf-26e4-4a0c-ad9f-50dcc391616b-istio-data\") pod \"e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-clqsv\" (UID: \"21557bdf-26e4-4a0c-ad9f-50dcc391616b\") " pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-clqsv"
Oct 30 22:03:42.807349 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:42.807081    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/21557bdf-26e4-4a0c-ad9f-50dcc391616b-istio-envoy\") pod \"e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-clqsv\" (UID: \"21557bdf-26e4-4a0c-ad9f-50dcc391616b\") " pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-clqsv"
Oct 30 22:03:42.807349 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:42.807332    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/21557bdf-26e4-4a0c-ad9f-50dcc391616b-credential-socket\") pod \"e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-clqsv\" (UID: \"21557bdf-26e4-4a0c-ad9f-50dcc391616b\") " pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-clqsv"
Oct 30 22:03:42.807698 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:42.807421    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/21557bdf-26e4-4a0c-ad9f-50dcc391616b-istio-data\") pod \"e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-clqsv\" (UID: \"21557bdf-26e4-4a0c-ad9f-50dcc391616b\") " pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-clqsv"
Oct 30 22:03:42.807698 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:42.807444    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/21557bdf-26e4-4a0c-ad9f-50dcc391616b-workload-certs\") pod \"e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-clqsv\" (UID: \"21557bdf-26e4-4a0c-ad9f-50dcc391616b\") " pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-clqsv"
Oct 30 22:03:42.807698 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:42.807511    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/21557bdf-26e4-4a0c-ad9f-50dcc391616b-workload-socket\") pod \"e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-clqsv\" (UID: \"21557bdf-26e4-4a0c-ad9f-50dcc391616b\") " pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-clqsv"
Oct 30 22:03:42.807807 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:42.807700    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/21557bdf-26e4-4a0c-ad9f-50dcc391616b-istiod-ca-cert\") pod \"e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-clqsv\" (UID: \"21557bdf-26e4-4a0c-ad9f-50dcc391616b\") " pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-clqsv"
Oct 30 22:03:42.809419 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:42.809393    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/21557bdf-26e4-4a0c-ad9f-50dcc391616b-istio-envoy\") pod \"e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-clqsv\" (UID: \"21557bdf-26e4-4a0c-ad9f-50dcc391616b\") " pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-clqsv"
Oct 30 22:03:42.809563 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:42.809544    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/21557bdf-26e4-4a0c-ad9f-50dcc391616b-istio-podinfo\") pod \"e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-clqsv\" (UID: \"21557bdf-26e4-4a0c-ad9f-50dcc391616b\") " pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-clqsv"
Oct 30 22:03:42.864989 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:42.864952    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/21557bdf-26e4-4a0c-ad9f-50dcc391616b-istio-token\") pod \"e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-clqsv\" (UID: \"21557bdf-26e4-4a0c-ad9f-50dcc391616b\") " pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-clqsv"
Oct 30 22:03:42.865163 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:42.865088    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kzxw8\" (UniqueName: \"kubernetes.io/projected/21557bdf-26e4-4a0c-ad9f-50dcc391616b-kube-api-access-kzxw8\") pod \"e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-clqsv\" (UID: \"21557bdf-26e4-4a0c-ad9f-50dcc391616b\") " pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-clqsv"
Oct 30 22:03:42.928107 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:42.928071    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-clqsv"
Oct 30 22:03:42.949866 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:42.949832    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:03:43.186533 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:43.186492    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:03:43.284124 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:43.284090    5510 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-ccsc2"]
Oct 30 22:03:43.287331 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:43.287307    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-ccsc2"
Oct 30 22:03:43.338243 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:43.338214    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w3-7bb1c2-openshift-default-dockercfg-9dlcb\""
Oct 30 22:03:43.346729 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:43.346694    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-clqsv"]
Oct 30 22:03:43.353465 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:43.353441    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-ccsc2"]
Oct 30 22:03:43.375745 ip-10-0-136-210 kubenswrapper[5510]: W1030 22:03:43.375716    5510 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod21557bdf_26e4_4a0c_ad9f_50dcc391616b.slice/crio-7388e36da4856c4db671cfdab01bd92ca1782277e174b908d17b435149bbdbc2 WatchSource:0}: Error finding container 7388e36da4856c4db671cfdab01bd92ca1782277e174b908d17b435149bbdbc2: Status 404 returned error can't find the container with id 7388e36da4856c4db671cfdab01bd92ca1782277e174b908d17b435149bbdbc2
Oct 30 22:03:43.378133 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:43.378098    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:03:43.378232 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:43.378163    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:03:43.378232 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:43.378211    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:03:43.383897 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:43.383872    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:03:43.416123 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:43.416054    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qmscm\" (UniqueName: \"kubernetes.io/projected/caef4e8d-7dcc-455a-94a2-4bb341662a88-kube-api-access-qmscm\") pod \"e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-ccsc2\" (UID: \"caef4e8d-7dcc-455a-94a2-4bb341662a88\") " pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-ccsc2"
Oct 30 22:03:43.416230 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:43.416210    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/caef4e8d-7dcc-455a-94a2-4bb341662a88-istio-podinfo\") pod \"e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-ccsc2\" (UID: \"caef4e8d-7dcc-455a-94a2-4bb341662a88\") " pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-ccsc2"
Oct 30 22:03:43.416304 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:43.416289    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/caef4e8d-7dcc-455a-94a2-4bb341662a88-istio-data\") pod \"e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-ccsc2\" (UID: \"caef4e8d-7dcc-455a-94a2-4bb341662a88\") " pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-ccsc2"
Oct 30 22:03:43.416383 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:43.416367    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/caef4e8d-7dcc-455a-94a2-4bb341662a88-workload-socket\") pod \"e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-ccsc2\" (UID: \"caef4e8d-7dcc-455a-94a2-4bb341662a88\") " pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-ccsc2"
Oct 30 22:03:43.416424 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:43.416411    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/caef4e8d-7dcc-455a-94a2-4bb341662a88-istio-envoy\") pod \"e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-ccsc2\" (UID: \"caef4e8d-7dcc-455a-94a2-4bb341662a88\") " pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-ccsc2"
Oct 30 22:03:43.416467 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:43.416436    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/caef4e8d-7dcc-455a-94a2-4bb341662a88-istiod-ca-cert\") pod \"e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-ccsc2\" (UID: \"caef4e8d-7dcc-455a-94a2-4bb341662a88\") " pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-ccsc2"
Oct 30 22:03:43.416503 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:43.416474    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/caef4e8d-7dcc-455a-94a2-4bb341662a88-credential-socket\") pod \"e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-ccsc2\" (UID: \"caef4e8d-7dcc-455a-94a2-4bb341662a88\") " pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-ccsc2"
Oct 30 22:03:43.416543 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:43.416503    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/caef4e8d-7dcc-455a-94a2-4bb341662a88-workload-certs\") pod \"e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-ccsc2\" (UID: \"caef4e8d-7dcc-455a-94a2-4bb341662a88\") " pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-ccsc2"
Oct 30 22:03:43.416543 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:43.416534    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/caef4e8d-7dcc-455a-94a2-4bb341662a88-istio-token\") pod \"e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-ccsc2\" (UID: \"caef4e8d-7dcc-455a-94a2-4bb341662a88\") " pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-ccsc2"
Oct 30 22:03:43.517187 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:43.517147    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/caef4e8d-7dcc-455a-94a2-4bb341662a88-istio-envoy\") pod \"e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-ccsc2\" (UID: \"caef4e8d-7dcc-455a-94a2-4bb341662a88\") " pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-ccsc2"
Oct 30 22:03:43.517187 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:43.517186    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/caef4e8d-7dcc-455a-94a2-4bb341662a88-istiod-ca-cert\") pod \"e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-ccsc2\" (UID: \"caef4e8d-7dcc-455a-94a2-4bb341662a88\") " pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-ccsc2"
Oct 30 22:03:43.517426 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:43.517226    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/caef4e8d-7dcc-455a-94a2-4bb341662a88-credential-socket\") pod \"e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-ccsc2\" (UID: \"caef4e8d-7dcc-455a-94a2-4bb341662a88\") " pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-ccsc2"
Oct 30 22:03:43.517426 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:43.517251    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/caef4e8d-7dcc-455a-94a2-4bb341662a88-workload-certs\") pod \"e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-ccsc2\" (UID: \"caef4e8d-7dcc-455a-94a2-4bb341662a88\") " pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-ccsc2"
Oct 30 22:03:43.517426 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:43.517281    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/caef4e8d-7dcc-455a-94a2-4bb341662a88-istio-token\") pod \"e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-ccsc2\" (UID: \"caef4e8d-7dcc-455a-94a2-4bb341662a88\") " pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-ccsc2"
Oct 30 22:03:43.517426 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:43.517311    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qmscm\" (UniqueName: \"kubernetes.io/projected/caef4e8d-7dcc-455a-94a2-4bb341662a88-kube-api-access-qmscm\") pod \"e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-ccsc2\" (UID: \"caef4e8d-7dcc-455a-94a2-4bb341662a88\") " pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-ccsc2"
Oct 30 22:03:43.517426 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:43.517370    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/caef4e8d-7dcc-455a-94a2-4bb341662a88-istio-podinfo\") pod \"e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-ccsc2\" (UID: \"caef4e8d-7dcc-455a-94a2-4bb341662a88\") " pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-ccsc2"
Oct 30 22:03:43.517702 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:43.517433    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/caef4e8d-7dcc-455a-94a2-4bb341662a88-istio-data\") pod \"e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-ccsc2\" (UID: \"caef4e8d-7dcc-455a-94a2-4bb341662a88\") " pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-ccsc2"
Oct 30 22:03:43.517702 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:43.517518    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/caef4e8d-7dcc-455a-94a2-4bb341662a88-workload-socket\") pod \"e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-ccsc2\" (UID: \"caef4e8d-7dcc-455a-94a2-4bb341662a88\") " pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-ccsc2"
Oct 30 22:03:43.517702 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:43.517671    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/caef4e8d-7dcc-455a-94a2-4bb341662a88-workload-certs\") pod \"e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-ccsc2\" (UID: \"caef4e8d-7dcc-455a-94a2-4bb341662a88\") " pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-ccsc2"
Oct 30 22:03:43.517878 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:43.517707    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/caef4e8d-7dcc-455a-94a2-4bb341662a88-credential-socket\") pod \"e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-ccsc2\" (UID: \"caef4e8d-7dcc-455a-94a2-4bb341662a88\") " pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-ccsc2"
Oct 30 22:03:43.517936 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:43.517875    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/caef4e8d-7dcc-455a-94a2-4bb341662a88-workload-socket\") pod \"e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-ccsc2\" (UID: \"caef4e8d-7dcc-455a-94a2-4bb341662a88\") " pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-ccsc2"
Oct 30 22:03:43.517974 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:43.517929    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/caef4e8d-7dcc-455a-94a2-4bb341662a88-istio-data\") pod \"e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-ccsc2\" (UID: \"caef4e8d-7dcc-455a-94a2-4bb341662a88\") " pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-ccsc2"
Oct 30 22:03:43.518141 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:43.518113    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/caef4e8d-7dcc-455a-94a2-4bb341662a88-istiod-ca-cert\") pod \"e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-ccsc2\" (UID: \"caef4e8d-7dcc-455a-94a2-4bb341662a88\") " pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-ccsc2"
Oct 30 22:03:43.519491 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:43.519467    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/caef4e8d-7dcc-455a-94a2-4bb341662a88-istio-envoy\") pod \"e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-ccsc2\" (UID: \"caef4e8d-7dcc-455a-94a2-4bb341662a88\") " pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-ccsc2"
Oct 30 22:03:43.519873 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:43.519857    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/caef4e8d-7dcc-455a-94a2-4bb341662a88-istio-podinfo\") pod \"e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-ccsc2\" (UID: \"caef4e8d-7dcc-455a-94a2-4bb341662a88\") " pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-ccsc2"
Oct 30 22:03:43.601605 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:43.601526    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/caef4e8d-7dcc-455a-94a2-4bb341662a88-istio-token\") pod \"e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-ccsc2\" (UID: \"caef4e8d-7dcc-455a-94a2-4bb341662a88\") " pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-ccsc2"
Oct 30 22:03:43.601763 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:43.601667    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qmscm\" (UniqueName: \"kubernetes.io/projected/caef4e8d-7dcc-455a-94a2-4bb341662a88-kube-api-access-qmscm\") pod \"e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-ccsc2\" (UID: \"caef4e8d-7dcc-455a-94a2-4bb341662a88\") " pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-ccsc2"
Oct 30 22:03:43.670890 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:43.670833    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:03:43.881869 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:43.881758    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:03:43.886164 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:43.886142    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-clqsv" event={"ID":"21557bdf-26e4-4a0c-ad9f-50dcc391616b","Type":"ContainerStarted","Data":"f3a4ecf277b974e0f83bff4dbe835374e4b4ebe28697c46ac83f94aba5fa0924"}
Oct 30 22:03:43.886287 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:43.886185    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-clqsv" event={"ID":"21557bdf-26e4-4a0c-ad9f-50dcc391616b","Type":"ContainerStarted","Data":"7388e36da4856c4db671cfdab01bd92ca1782277e174b908d17b435149bbdbc2"}
Oct 30 22:03:43.900570 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:43.900545    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-ccsc2"
Oct 30 22:03:43.928493 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:43.928469    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-clqsv"
Oct 30 22:03:44.005576 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:44.005525    5510 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-clqsv" podStartSLOduration=2.005512954 podStartE2EDuration="2.005512954s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:03:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:03:44.004334012 +0000 UTC m=+804.565144016" watchObservedRunningTime="2026-10-30 22:03:44.005512954 +0000 UTC m=+804.566322958"
Oct 30 22:03:44.030771 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:44.030729    5510 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6fvq6"]
Oct 30 22:03:44.034043 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:44.034017    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6fvq6"
Oct 30 22:03:44.086100 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:44.086070    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w4-65e59f-openshift-default-dockercfg-rvg5q\""
Oct 30 22:03:44.105111 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:44.105077    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6fvq6"]
Oct 30 22:03:44.112649 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:44.112619    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:03:44.124376 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:44.124344    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5ff054ea-b27c-4c64-be2d-c645c0ca3d24-istiod-ca-cert\") pod \"e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6fvq6\" (UID: \"5ff054ea-b27c-4c64-be2d-c645c0ca3d24\") " pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6fvq6"
Oct 30 22:03:44.124567 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:44.124390    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5ff054ea-b27c-4c64-be2d-c645c0ca3d24-istio-token\") pod \"e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6fvq6\" (UID: \"5ff054ea-b27c-4c64-be2d-c645c0ca3d24\") " pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6fvq6"
Oct 30 22:03:44.124567 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:44.124478    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5ff054ea-b27c-4c64-be2d-c645c0ca3d24-istio-envoy\") pod \"e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6fvq6\" (UID: \"5ff054ea-b27c-4c64-be2d-c645c0ca3d24\") " pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6fvq6"
Oct 30 22:03:44.124567 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:44.124537    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5ff054ea-b27c-4c64-be2d-c645c0ca3d24-workload-socket\") pod \"e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6fvq6\" (UID: \"5ff054ea-b27c-4c64-be2d-c645c0ca3d24\") " pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6fvq6"
Oct 30 22:03:44.124811 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:44.124593    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nx7zv\" (UniqueName: \"kubernetes.io/projected/5ff054ea-b27c-4c64-be2d-c645c0ca3d24-kube-api-access-nx7zv\") pod \"e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6fvq6\" (UID: \"5ff054ea-b27c-4c64-be2d-c645c0ca3d24\") " pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6fvq6"
Oct 30 22:03:44.124811 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:44.124642    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5ff054ea-b27c-4c64-be2d-c645c0ca3d24-credential-socket\") pod \"e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6fvq6\" (UID: \"5ff054ea-b27c-4c64-be2d-c645c0ca3d24\") " pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6fvq6"
Oct 30 22:03:44.124811 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:44.124711    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5ff054ea-b27c-4c64-be2d-c645c0ca3d24-istio-data\") pod \"e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6fvq6\" (UID: \"5ff054ea-b27c-4c64-be2d-c645c0ca3d24\") " pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6fvq6"
Oct 30 22:03:44.124811 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:44.124737    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5ff054ea-b27c-4c64-be2d-c645c0ca3d24-workload-certs\") pod \"e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6fvq6\" (UID: \"5ff054ea-b27c-4c64-be2d-c645c0ca3d24\") " pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6fvq6"
Oct 30 22:03:44.125034 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:44.124811    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5ff054ea-b27c-4c64-be2d-c645c0ca3d24-istio-podinfo\") pod \"e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6fvq6\" (UID: \"5ff054ea-b27c-4c64-be2d-c645c0ca3d24\") " pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6fvq6"
Oct 30 22:03:44.226257 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:44.226170    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5ff054ea-b27c-4c64-be2d-c645c0ca3d24-istio-envoy\") pod \"e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6fvq6\" (UID: \"5ff054ea-b27c-4c64-be2d-c645c0ca3d24\") " pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6fvq6"
Oct 30 22:03:44.226257 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:44.226221    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5ff054ea-b27c-4c64-be2d-c645c0ca3d24-workload-socket\") pod \"e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6fvq6\" (UID: \"5ff054ea-b27c-4c64-be2d-c645c0ca3d24\") " pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6fvq6"
Oct 30 22:03:44.226257 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:44.226247    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nx7zv\" (UniqueName: \"kubernetes.io/projected/5ff054ea-b27c-4c64-be2d-c645c0ca3d24-kube-api-access-nx7zv\") pod \"e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6fvq6\" (UID: \"5ff054ea-b27c-4c64-be2d-c645c0ca3d24\") " pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6fvq6"
Oct 30 22:03:44.226547 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:44.226280    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5ff054ea-b27c-4c64-be2d-c645c0ca3d24-credential-socket\") pod \"e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6fvq6\" (UID: \"5ff054ea-b27c-4c64-be2d-c645c0ca3d24\") " pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6fvq6"
Oct 30 22:03:44.226547 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:44.226301    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5ff054ea-b27c-4c64-be2d-c645c0ca3d24-istio-data\") pod \"e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6fvq6\" (UID: \"5ff054ea-b27c-4c64-be2d-c645c0ca3d24\") " pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6fvq6"
Oct 30 22:03:44.226547 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:44.226323    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5ff054ea-b27c-4c64-be2d-c645c0ca3d24-workload-certs\") pod \"e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6fvq6\" (UID: \"5ff054ea-b27c-4c64-be2d-c645c0ca3d24\") " pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6fvq6"
Oct 30 22:03:44.226547 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:44.226381    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5ff054ea-b27c-4c64-be2d-c645c0ca3d24-istio-podinfo\") pod \"e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6fvq6\" (UID: \"5ff054ea-b27c-4c64-be2d-c645c0ca3d24\") " pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6fvq6"
Oct 30 22:03:44.226547 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:44.226435    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5ff054ea-b27c-4c64-be2d-c645c0ca3d24-istiod-ca-cert\") pod \"e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6fvq6\" (UID: \"5ff054ea-b27c-4c64-be2d-c645c0ca3d24\") " pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6fvq6"
Oct 30 22:03:44.226547 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:44.226478    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5ff054ea-b27c-4c64-be2d-c645c0ca3d24-istio-token\") pod \"e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6fvq6\" (UID: \"5ff054ea-b27c-4c64-be2d-c645c0ca3d24\") " pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6fvq6"
Oct 30 22:03:44.226905 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:44.226734    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5ff054ea-b27c-4c64-be2d-c645c0ca3d24-workload-socket\") pod \"e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6fvq6\" (UID: \"5ff054ea-b27c-4c64-be2d-c645c0ca3d24\") " pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6fvq6"
Oct 30 22:03:44.226905 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:44.226766    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5ff054ea-b27c-4c64-be2d-c645c0ca3d24-credential-socket\") pod \"e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6fvq6\" (UID: \"5ff054ea-b27c-4c64-be2d-c645c0ca3d24\") " pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6fvq6"
Oct 30 22:03:44.227018 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:44.226969    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5ff054ea-b27c-4c64-be2d-c645c0ca3d24-workload-certs\") pod \"e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6fvq6\" (UID: \"5ff054ea-b27c-4c64-be2d-c645c0ca3d24\") " pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6fvq6"
Oct 30 22:03:44.227018 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:44.227003    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5ff054ea-b27c-4c64-be2d-c645c0ca3d24-istio-data\") pod \"e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6fvq6\" (UID: \"5ff054ea-b27c-4c64-be2d-c645c0ca3d24\") " pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6fvq6"
Oct 30 22:03:44.227314 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:44.227288    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5ff054ea-b27c-4c64-be2d-c645c0ca3d24-istiod-ca-cert\") pod \"e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6fvq6\" (UID: \"5ff054ea-b27c-4c64-be2d-c645c0ca3d24\") " pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6fvq6"
Oct 30 22:03:44.228587 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:44.228559    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5ff054ea-b27c-4c64-be2d-c645c0ca3d24-istio-envoy\") pod \"e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6fvq6\" (UID: \"5ff054ea-b27c-4c64-be2d-c645c0ca3d24\") " pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6fvq6"
Oct 30 22:03:44.228807 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:44.228790    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5ff054ea-b27c-4c64-be2d-c645c0ca3d24-istio-podinfo\") pod \"e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6fvq6\" (UID: \"5ff054ea-b27c-4c64-be2d-c645c0ca3d24\") " pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6fvq6"
Oct 30 22:03:44.285340 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:44.285302    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5ff054ea-b27c-4c64-be2d-c645c0ca3d24-istio-token\") pod \"e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6fvq6\" (UID: \"5ff054ea-b27c-4c64-be2d-c645c0ca3d24\") " pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6fvq6"
Oct 30 22:03:44.285494 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:44.285353    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nx7zv\" (UniqueName: \"kubernetes.io/projected/5ff054ea-b27c-4c64-be2d-c645c0ca3d24-kube-api-access-nx7zv\") pod \"e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6fvq6\" (UID: \"5ff054ea-b27c-4c64-be2d-c645c0ca3d24\") " pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6fvq6"
Oct 30 22:03:44.319622 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:44.319599    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-ccsc2"]
Oct 30 22:03:44.321744 ip-10-0-136-210 kubenswrapper[5510]: W1030 22:03:44.321712    5510 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcaef4e8d_7dcc_455a_94a2_4bb341662a88.slice/crio-d228450133636e3fce07ad40ff3ab39cb70c8888f1b7a9dbc5ef2ae073c4dcc5 WatchSource:0}: Error finding container d228450133636e3fce07ad40ff3ab39cb70c8888f1b7a9dbc5ef2ae073c4dcc5: Status 404 returned error can't find the container with id d228450133636e3fce07ad40ff3ab39cb70c8888f1b7a9dbc5ef2ae073c4dcc5
Oct 30 22:03:44.323052 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:44.323001    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:03:44.324225 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:44.324196    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:03:44.324309 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:44.324258    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:03:44.324309 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:44.324287    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:03:44.345769 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:44.345747    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6fvq6"
Oct 30 22:03:44.548564 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:44.548529    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:03:44.755374 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:44.755328    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:03:44.770028 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:44.769998    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6fvq6"]
Oct 30 22:03:44.772410 ip-10-0-136-210 kubenswrapper[5510]: W1030 22:03:44.772380    5510 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ff054ea_b27c_4c64_be2d_c645c0ca3d24.slice/crio-2f3048de9b7826761e5cafca5e011ff1506133ecd03d8331b178e8ea9a36e9c8 WatchSource:0}: Error finding container 2f3048de9b7826761e5cafca5e011ff1506133ecd03d8331b178e8ea9a36e9c8: Status 404 returned error can't find the container with id 2f3048de9b7826761e5cafca5e011ff1506133ecd03d8331b178e8ea9a36e9c8
Oct 30 22:03:44.774860 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:44.774824    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:03:44.774958 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:44.774905    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:03:44.774958 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:44.774950    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:03:44.893396 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:44.893358    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-ccsc2" event={"ID":"caef4e8d-7dcc-455a-94a2-4bb341662a88","Type":"ContainerStarted","Data":"0bae0294bc1a482d1e9171428212b20af595fa61ee28059daa35724bb102a910"}
Oct 30 22:03:44.893821 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:44.893423    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-ccsc2" event={"ID":"caef4e8d-7dcc-455a-94a2-4bb341662a88","Type":"ContainerStarted","Data":"d228450133636e3fce07ad40ff3ab39cb70c8888f1b7a9dbc5ef2ae073c4dcc5"}
Oct 30 22:03:44.895290 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:44.895260    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6fvq6" event={"ID":"5ff054ea-b27c-4c64-be2d-c645c0ca3d24","Type":"ContainerStarted","Data":"a24ba72543ab31aeb5b19f93d3fba849190ec169d677ec88fd48df2f4f237693"}
Oct 30 22:03:44.895411 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:44.895323    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6fvq6" event={"ID":"5ff054ea-b27c-4c64-be2d-c645c0ca3d24","Type":"ContainerStarted","Data":"2f3048de9b7826761e5cafca5e011ff1506133ecd03d8331b178e8ea9a36e9c8"}
Oct 30 22:03:44.901598 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:44.901577    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-ccsc2"
Oct 30 22:03:44.934812 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:44.934783    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-clqsv"
Oct 30 22:03:45.009575 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:45.009519    5510 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-ccsc2" podStartSLOduration=2.009504066 podStartE2EDuration="2.009504066s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:03:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:03:45.007726531 +0000 UTC m=+805.568536553" watchObservedRunningTime="2026-10-30 22:03:45.009504066 +0000 UTC m=+805.570314148"
Oct 30 22:03:45.243018 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:45.242916    5510 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6fvq6" podStartSLOduration=2.242902292 podStartE2EDuration="2.242902292s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:03:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:03:45.24073128 +0000 UTC m=+805.801541302" watchObservedRunningTime="2026-10-30 22:03:45.242902292 +0000 UTC m=+805.803712293"
Oct 30 22:03:45.346736 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:45.346693    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6fvq6"
Oct 30 22:03:45.351630 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:45.351601    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6fvq6"
Oct 30 22:03:45.900948 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:45.900917    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-clqsv"
Oct 30 22:03:45.901519 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:45.901491    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6fvq6"
Oct 30 22:03:45.901966 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:45.901944    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-clqsv"
Oct 30 22:03:45.902191 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:45.902177    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6fvq6"
Oct 30 22:03:45.906069 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:45.906046    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-ccsc2"
Oct 30 22:03:46.906129 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:46.906096    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-ccsc2"
Oct 30 22:03:46.906928 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:46.906910    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-ccsc2"
Oct 30 22:03:47.366550 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:47.366521    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:03:47.678470 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:47.678366    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:03:47.994605 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:47.994507    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:03:48.229666 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:48.229631    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:03:48.425615 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:48.425582    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:03:48.656218 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:48.656182    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:03:48.970125 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:48.970087    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:03:49.226381 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:49.226290    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:03:49.532204 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:49.532159    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:03:49.753999 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:49.753958    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:03:50.207315 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:50.207279    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:03:50.462605 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:50.462513    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:03:50.835197 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:50.835158    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:03:51.124939 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:51.124851    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:03:53.200429 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:53.200147    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:03:53.394360 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:53.394320    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:03:53.563386 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:53.563351    5510 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w5-c9dbe6-6986bd68b7-2kg6w"]
Oct 30 22:03:53.567774 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:53.567750    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-c9dbe6-6986bd68b7-2kg6w"
Oct 30 22:03:53.617090 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:53.617063    5510 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w5-c9dbe6\""
Oct 30 22:03:53.651429 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:53.651395    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e3ff7b9f-1fe3-4b03-848e-298ac1105742-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w5-c9dbe6-6986bd68b7-2kg6w\" (UID: \"e3ff7b9f-1fe3-4b03-848e-298ac1105742\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-c9dbe6-6986bd68b7-2kg6w"
Oct 30 22:03:53.651711 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:53.651439    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zjll7\" (UniqueName: \"kubernetes.io/projected/e3ff7b9f-1fe3-4b03-848e-298ac1105742-kube-api-access-zjll7\") pod \"payload-pre-processing-e2e-worker-w5-c9dbe6-6986bd68b7-2kg6w\" (UID: \"e3ff7b9f-1fe3-4b03-848e-298ac1105742\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-c9dbe6-6986bd68b7-2kg6w"
Oct 30 22:03:53.671279 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:53.671244    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w5-c9dbe6-dockercfg-6nbvn\""
Oct 30 22:03:53.714255 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:53.714217    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w5-c9dbe6-6986bd68b7-2kg6w"]
Oct 30 22:03:53.715965 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:53.715939    5510 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w5-c9dbe6-584f9bc9cf-z2ph8"]
Oct 30 22:03:53.719723 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:53.719702    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w5-c9dbe6-584f9bc9cf-z2ph8"
Oct 30 22:03:53.730249 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:53.730218    5510 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-bc714d-58cbbc478d-xsqpn"]
Oct 30 22:03:53.733948 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:53.733929    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-bc714d-58cbbc478d-xsqpn"
Oct 30 22:03:53.752527 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:53.752495    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/559d4d98-2fa9-41fe-b159-79a92477294a-plugins-config-volume\") pod \"payload-processing-e2e-worker-w5-c9dbe6-584f9bc9cf-z2ph8\" (UID: \"559d4d98-2fa9-41fe-b159-79a92477294a\") " pod="openshift-ingress/payload-processing-e2e-worker-w5-c9dbe6-584f9bc9cf-z2ph8"
Oct 30 22:03:53.752700 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:53.752609    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7q72b\" (UniqueName: \"kubernetes.io/projected/559d4d98-2fa9-41fe-b159-79a92477294a-kube-api-access-7q72b\") pod \"payload-processing-e2e-worker-w5-c9dbe6-584f9bc9cf-z2ph8\" (UID: \"559d4d98-2fa9-41fe-b159-79a92477294a\") " pod="openshift-ingress/payload-processing-e2e-worker-w5-c9dbe6-584f9bc9cf-z2ph8"
Oct 30 22:03:53.752761 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:53.752703    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e3ff7b9f-1fe3-4b03-848e-298ac1105742-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w5-c9dbe6-6986bd68b7-2kg6w\" (UID: \"e3ff7b9f-1fe3-4b03-848e-298ac1105742\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-c9dbe6-6986bd68b7-2kg6w"
Oct 30 22:03:53.752761 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:53.752734    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zjll7\" (UniqueName: \"kubernetes.io/projected/e3ff7b9f-1fe3-4b03-848e-298ac1105742-kube-api-access-zjll7\") pod \"payload-pre-processing-e2e-worker-w5-c9dbe6-6986bd68b7-2kg6w\" (UID: \"e3ff7b9f-1fe3-4b03-848e-298ac1105742\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-c9dbe6-6986bd68b7-2kg6w"
Oct 30 22:03:53.752859 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:53.752762    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/afc34d05-1306-4b04-8c81-f123a51f682c-metrics-certs\") pod \"maas-api-e2e-worker-w2-bc714d-58cbbc478d-xsqpn\" (UID: \"afc34d05-1306-4b04-8c81-f123a51f682c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-bc714d-58cbbc478d-xsqpn"
Oct 30 22:03:53.752859 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:53.752789    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tv88v\" (UniqueName: \"kubernetes.io/projected/afc34d05-1306-4b04-8c81-f123a51f682c-kube-api-access-tv88v\") pod \"maas-api-e2e-worker-w2-bc714d-58cbbc478d-xsqpn\" (UID: \"afc34d05-1306-4b04-8c81-f123a51f682c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-bc714d-58cbbc478d-xsqpn"
Oct 30 22:03:53.752991 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:53.752971    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/afc34d05-1306-4b04-8c81-f123a51f682c-openshift-service-ca\") pod \"maas-api-e2e-worker-w2-bc714d-58cbbc478d-xsqpn\" (UID: \"afc34d05-1306-4b04-8c81-f123a51f682c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-bc714d-58cbbc478d-xsqpn"
Oct 30 22:03:53.753046 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:53.753012    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/afc34d05-1306-4b04-8c81-f123a51f682c-maas-api-tls\") pod \"maas-api-e2e-worker-w2-bc714d-58cbbc478d-xsqpn\" (UID: \"afc34d05-1306-4b04-8c81-f123a51f682c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-bc714d-58cbbc478d-xsqpn"
Oct 30 22:03:53.753431 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:53.753410    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e3ff7b9f-1fe3-4b03-848e-298ac1105742-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w5-c9dbe6-6986bd68b7-2kg6w\" (UID: \"e3ff7b9f-1fe3-4b03-848e-298ac1105742\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-c9dbe6-6986bd68b7-2kg6w"
Oct 30 22:03:53.792884 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:53.792848    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w2-bc714d-serving-cert\""
Oct 30 22:03:53.816486 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:53.816391    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w5-c9dbe6-584f9bc9cf-z2ph8"]
Oct 30 22:03:53.830925 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:53.830883    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zjll7\" (UniqueName: \"kubernetes.io/projected/e3ff7b9f-1fe3-4b03-848e-298ac1105742-kube-api-access-zjll7\") pod \"payload-pre-processing-e2e-worker-w5-c9dbe6-6986bd68b7-2kg6w\" (UID: \"e3ff7b9f-1fe3-4b03-848e-298ac1105742\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-c9dbe6-6986bd68b7-2kg6w"
Oct 30 22:03:53.854316 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:53.854280    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/559d4d98-2fa9-41fe-b159-79a92477294a-plugins-config-volume\") pod \"payload-processing-e2e-worker-w5-c9dbe6-584f9bc9cf-z2ph8\" (UID: \"559d4d98-2fa9-41fe-b159-79a92477294a\") " pod="openshift-ingress/payload-processing-e2e-worker-w5-c9dbe6-584f9bc9cf-z2ph8"
Oct 30 22:03:53.854468 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:53.854338    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7q72b\" (UniqueName: \"kubernetes.io/projected/559d4d98-2fa9-41fe-b159-79a92477294a-kube-api-access-7q72b\") pod \"payload-processing-e2e-worker-w5-c9dbe6-584f9bc9cf-z2ph8\" (UID: \"559d4d98-2fa9-41fe-b159-79a92477294a\") " pod="openshift-ingress/payload-processing-e2e-worker-w5-c9dbe6-584f9bc9cf-z2ph8"
Oct 30 22:03:53.854527 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:53.854507    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/afc34d05-1306-4b04-8c81-f123a51f682c-metrics-certs\") pod \"maas-api-e2e-worker-w2-bc714d-58cbbc478d-xsqpn\" (UID: \"afc34d05-1306-4b04-8c81-f123a51f682c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-bc714d-58cbbc478d-xsqpn"
Oct 30 22:03:53.854573 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:53.854556    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tv88v\" (UniqueName: \"kubernetes.io/projected/afc34d05-1306-4b04-8c81-f123a51f682c-kube-api-access-tv88v\") pod \"maas-api-e2e-worker-w2-bc714d-58cbbc478d-xsqpn\" (UID: \"afc34d05-1306-4b04-8c81-f123a51f682c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-bc714d-58cbbc478d-xsqpn"
Oct 30 22:03:53.854616 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:53.854589    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/afc34d05-1306-4b04-8c81-f123a51f682c-openshift-service-ca\") pod \"maas-api-e2e-worker-w2-bc714d-58cbbc478d-xsqpn\" (UID: \"afc34d05-1306-4b04-8c81-f123a51f682c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-bc714d-58cbbc478d-xsqpn"
Oct 30 22:03:53.854652 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:53.854638    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/afc34d05-1306-4b04-8c81-f123a51f682c-maas-api-tls\") pod \"maas-api-e2e-worker-w2-bc714d-58cbbc478d-xsqpn\" (UID: \"afc34d05-1306-4b04-8c81-f123a51f682c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-bc714d-58cbbc478d-xsqpn"
Oct 30 22:03:53.854826 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:03:53.854804    5510 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w2-bc714d-serving-cert: secret "maas-api-e2e-worker-w2-bc714d-serving-cert" not found
Oct 30 22:03:53.854909 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:03:53.854883    5510 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/afc34d05-1306-4b04-8c81-f123a51f682c-maas-api-tls podName:afc34d05-1306-4b04-8c81-f123a51f682c nodeName:}" failed. No retries permitted until 2026-10-30 22:03:54.3548622 +0000 UTC m=+814.915672199 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/afc34d05-1306-4b04-8c81-f123a51f682c-maas-api-tls") pod "maas-api-e2e-worker-w2-bc714d-58cbbc478d-xsqpn" (UID: "afc34d05-1306-4b04-8c81-f123a51f682c") : secret "maas-api-e2e-worker-w2-bc714d-serving-cert" not found
Oct 30 22:03:53.855154 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:53.855129    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/559d4d98-2fa9-41fe-b159-79a92477294a-plugins-config-volume\") pod \"payload-processing-e2e-worker-w5-c9dbe6-584f9bc9cf-z2ph8\" (UID: \"559d4d98-2fa9-41fe-b159-79a92477294a\") " pod="openshift-ingress/payload-processing-e2e-worker-w5-c9dbe6-584f9bc9cf-z2ph8"
Oct 30 22:03:53.855268 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:53.855250    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/afc34d05-1306-4b04-8c81-f123a51f682c-openshift-service-ca\") pod \"maas-api-e2e-worker-w2-bc714d-58cbbc478d-xsqpn\" (UID: \"afc34d05-1306-4b04-8c81-f123a51f682c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-bc714d-58cbbc478d-xsqpn"
Oct 30 22:03:53.856959 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:53.856940    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/afc34d05-1306-4b04-8c81-f123a51f682c-metrics-certs\") pod \"maas-api-e2e-worker-w2-bc714d-58cbbc478d-xsqpn\" (UID: \"afc34d05-1306-4b04-8c81-f123a51f682c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-bc714d-58cbbc478d-xsqpn"
Oct 30 22:03:53.883521 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:53.883492    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-bc714d-58cbbc478d-xsqpn"]
Oct 30 22:03:53.900169 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:53.900139    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-c9dbe6-6986bd68b7-2kg6w"
Oct 30 22:03:53.913513 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:53.913425    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tv88v\" (UniqueName: \"kubernetes.io/projected/afc34d05-1306-4b04-8c81-f123a51f682c-kube-api-access-tv88v\") pod \"maas-api-e2e-worker-w2-bc714d-58cbbc478d-xsqpn\" (UID: \"afc34d05-1306-4b04-8c81-f123a51f682c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-bc714d-58cbbc478d-xsqpn"
Oct 30 22:03:53.914088 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:53.914052    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7q72b\" (UniqueName: \"kubernetes.io/projected/559d4d98-2fa9-41fe-b159-79a92477294a-kube-api-access-7q72b\") pod \"payload-processing-e2e-worker-w5-c9dbe6-584f9bc9cf-z2ph8\" (UID: \"559d4d98-2fa9-41fe-b159-79a92477294a\") " pod="openshift-ingress/payload-processing-e2e-worker-w5-c9dbe6-584f9bc9cf-z2ph8"
Oct 30 22:03:54.020692 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:54.020616    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:03:54.036432 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:54.036397    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w5-c9dbe6-584f9bc9cf-z2ph8"
Oct 30 22:03:54.259231 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:54.259191    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:03:54.289164 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:54.289122    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w5-c9dbe6-6986bd68b7-2kg6w"]
Oct 30 22:03:54.367105 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:54.367074    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/afc34d05-1306-4b04-8c81-f123a51f682c-maas-api-tls\") pod \"maas-api-e2e-worker-w2-bc714d-58cbbc478d-xsqpn\" (UID: \"afc34d05-1306-4b04-8c81-f123a51f682c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-bc714d-58cbbc478d-xsqpn"
Oct 30 22:03:54.369325 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:54.369306    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/afc34d05-1306-4b04-8c81-f123a51f682c-maas-api-tls\") pod \"maas-api-e2e-worker-w2-bc714d-58cbbc478d-xsqpn\" (UID: \"afc34d05-1306-4b04-8c81-f123a51f682c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-bc714d-58cbbc478d-xsqpn"
Oct 30 22:03:54.404822 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:54.404794    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w5-c9dbe6-584f9bc9cf-z2ph8"]
Oct 30 22:03:54.407346 ip-10-0-136-210 kubenswrapper[5510]: W1030 22:03:54.407321    5510 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod559d4d98_2fa9_41fe_b159_79a92477294a.slice/crio-4cd5c883fe296218a63ec6e6df97546d73aa08966f2c2d98e99d6b13506f4802 WatchSource:0}: Error finding container 4cd5c883fe296218a63ec6e6df97546d73aa08966f2c2d98e99d6b13506f4802: Status 404 returned error can't find the container with id 4cd5c883fe296218a63ec6e6df97546d73aa08966f2c2d98e99d6b13506f4802
Oct 30 22:03:54.646310 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:54.646227    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-bc714d-58cbbc478d-xsqpn"
Oct 30 22:03:54.923519 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:54.923438    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:03:54.948940 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:54.948905    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w5-c9dbe6-584f9bc9cf-z2ph8" event={"ID":"559d4d98-2fa9-41fe-b159-79a92477294a","Type":"ContainerStarted","Data":"cb168005c0cd05e7265faea26dced6f5dd75278fc17617555e35eed68e0b5f14"}
Oct 30 22:03:54.949120 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:54.948966    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w5-c9dbe6-584f9bc9cf-z2ph8" event={"ID":"559d4d98-2fa9-41fe-b159-79a92477294a","Type":"ContainerStarted","Data":"4cd5c883fe296218a63ec6e6df97546d73aa08966f2c2d98e99d6b13506f4802"}
Oct 30 22:03:54.949120 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:54.948997    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w5-c9dbe6-584f9bc9cf-z2ph8"
Oct 30 22:03:54.950499 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:54.950478    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-c9dbe6-6986bd68b7-2kg6w" event={"ID":"e3ff7b9f-1fe3-4b03-848e-298ac1105742","Type":"ContainerStarted","Data":"a196a415d9dbde8836727c9b005cccefefffd6c4b39b8e434225744bd08e2c32"}
Oct 30 22:03:54.950586 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:54.950517    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-c9dbe6-6986bd68b7-2kg6w" event={"ID":"e3ff7b9f-1fe3-4b03-848e-298ac1105742","Type":"ContainerStarted","Data":"4808189a81b1eb93da96bf71101c6a912e16940a52bc5edbaf992c48ab0b6f1a"}
Oct 30 22:03:54.950586 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:54.950551    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-c9dbe6-6986bd68b7-2kg6w"
Oct 30 22:03:55.060504 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:55.060449    5510 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w5-c9dbe6-584f9bc9cf-z2ph8" podStartSLOduration=2.060433583 podStartE2EDuration="2.060433583s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:03:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:03:55.059369008 +0000 UTC m=+815.620179012" watchObservedRunningTime="2026-10-30 22:03:55.060433583 +0000 UTC m=+815.621243588"
Oct 30 22:03:55.065773 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:55.065744    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-bc714d-58cbbc478d-xsqpn"]
Oct 30 22:03:55.100842 ip-10-0-136-210 kubenswrapper[5510]: W1030 22:03:55.100813    5510 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podafc34d05_1306_4b04_8c81_f123a51f682c.slice/crio-6bd84cb5a0ce22cd6d35c436f92ec19a37a02a299bf623d3c00cfd94b9d3ca05 WatchSource:0}: Error finding container 6bd84cb5a0ce22cd6d35c436f92ec19a37a02a299bf623d3c00cfd94b9d3ca05: Status 404 returned error can't find the container with id 6bd84cb5a0ce22cd6d35c436f92ec19a37a02a299bf623d3c00cfd94b9d3ca05
Oct 30 22:03:55.174585 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:55.174490    5510 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-c9dbe6-6986bd68b7-2kg6w" podStartSLOduration=2.174477306 podStartE2EDuration="2.174477306s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:03:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:03:55.17304235 +0000 UTC m=+815.733852356" watchObservedRunningTime="2026-10-30 22:03:55.174477306 +0000 UTC m=+815.735287373"
Oct 30 22:03:55.190970 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:55.190934    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:03:55.913171 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:55.913134    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:03:55.956733 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:55.956695    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-bc714d-58cbbc478d-xsqpn" event={"ID":"afc34d05-1306-4b04-8c81-f123a51f682c","Type":"ContainerStarted","Data":"6bd84cb5a0ce22cd6d35c436f92ec19a37a02a299bf623d3c00cfd94b9d3ca05"}
Oct 30 22:03:56.122934 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:56.122899    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:03:56.965559 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:56.965515    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-bc714d-58cbbc478d-xsqpn" event={"ID":"afc34d05-1306-4b04-8c81-f123a51f682c","Type":"ContainerStarted","Data":"7c5f6e92fc61553d960f5e6364d7393271237a87f58a0c3e0a1ceb00ec19b79e"}
Oct 30 22:03:56.966250 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:56.966217    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-bc714d-58cbbc478d-xsqpn"
Oct 30 22:03:57.080374 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:57.080311    5510 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-bc714d-58cbbc478d-xsqpn" podStartSLOduration=3.131759201 podStartE2EDuration="4.08029752s" totalImagesPullingTime="948.538319ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:03:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:03:57.078525356 +0000 UTC m=+817.639335370" watchObservedRunningTime="2026-10-30 22:03:57.08029752 +0000 UTC m=+817.641107563"
Oct 30 22:03:58.523950 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:58.523914    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:03:58.719049 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:58.719008    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:03:58.897681 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:58.897582    5510 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-779f32-67bbb7bf95-sjnl9"]
Oct 30 22:03:58.901676 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:58.901635    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-779f32-67bbb7bf95-sjnl9"
Oct 30 22:03:59.009808 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:59.009762    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w0-779f32-dockercfg-d6dj7\""
Oct 30 22:03:59.010370 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:59.010345    5510 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w0-779f32\""
Oct 30 22:03:59.027273 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:59.027230    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4fd056ae-943a-43c1-8682-8edb3fe3b5ed-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-779f32-67bbb7bf95-sjnl9\" (UID: \"4fd056ae-943a-43c1-8682-8edb3fe3b5ed\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-779f32-67bbb7bf95-sjnl9"
Oct 30 22:03:59.027273 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:59.027417    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4wmpg\" (UniqueName: \"kubernetes.io/projected/4fd056ae-943a-43c1-8682-8edb3fe3b5ed-kube-api-access-4wmpg\") pod \"payload-pre-processing-e2e-worker-w0-779f32-67bbb7bf95-sjnl9\" (UID: \"4fd056ae-943a-43c1-8682-8edb3fe3b5ed\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-779f32-67bbb7bf95-sjnl9"
Oct 30 22:03:59.034517 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:59.034484    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-779f32-67bbb7bf95-sjnl9"]
Oct 30 22:03:59.128201 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:59.128169    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4wmpg\" (UniqueName: \"kubernetes.io/projected/4fd056ae-943a-43c1-8682-8edb3fe3b5ed-kube-api-access-4wmpg\") pod \"payload-pre-processing-e2e-worker-w0-779f32-67bbb7bf95-sjnl9\" (UID: \"4fd056ae-943a-43c1-8682-8edb3fe3b5ed\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-779f32-67bbb7bf95-sjnl9"
Oct 30 22:03:59.128348 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:59.128242    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4fd056ae-943a-43c1-8682-8edb3fe3b5ed-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-779f32-67bbb7bf95-sjnl9\" (UID: \"4fd056ae-943a-43c1-8682-8edb3fe3b5ed\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-779f32-67bbb7bf95-sjnl9"
Oct 30 22:03:59.129001 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:59.128981    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4fd056ae-943a-43c1-8682-8edb3fe3b5ed-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-779f32-67bbb7bf95-sjnl9\" (UID: \"4fd056ae-943a-43c1-8682-8edb3fe3b5ed\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-779f32-67bbb7bf95-sjnl9"
Oct 30 22:03:59.187230 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:59.187159    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4wmpg\" (UniqueName: \"kubernetes.io/projected/4fd056ae-943a-43c1-8682-8edb3fe3b5ed-kube-api-access-4wmpg\") pod \"payload-pre-processing-e2e-worker-w0-779f32-67bbb7bf95-sjnl9\" (UID: \"4fd056ae-943a-43c1-8682-8edb3fe3b5ed\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-779f32-67bbb7bf95-sjnl9"
Oct 30 22:03:59.215140 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:59.215108    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-779f32-67bbb7bf95-sjnl9"
Oct 30 22:03:59.237132 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:59.237104    5510 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-779f32-76b4d9d8c7-fxknc"]
Oct 30 22:03:59.241880 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:59.241854    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-779f32-76b4d9d8c7-fxknc"
Oct 30 22:03:59.291209 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:59.291177    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w0-779f32-serving-cert\""
Oct 30 22:03:59.307381 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:59.307353    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-779f32-76b4d9d8c7-fxknc"]
Oct 30 22:03:59.331219 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:59.331179    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4a00e5f4-7118-41ce-b4c7-eb0f74da8237-maas-api-tls\") pod \"maas-api-e2e-worker-w0-779f32-76b4d9d8c7-fxknc\" (UID: \"4a00e5f4-7118-41ce-b4c7-eb0f74da8237\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-779f32-76b4d9d8c7-fxknc"
Oct 30 22:03:59.331397 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:59.331245    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4a00e5f4-7118-41ce-b4c7-eb0f74da8237-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-779f32-76b4d9d8c7-fxknc\" (UID: \"4a00e5f4-7118-41ce-b4c7-eb0f74da8237\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-779f32-76b4d9d8c7-fxknc"
Oct 30 22:03:59.331397 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:59.331369    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-42qrm\" (UniqueName: \"kubernetes.io/projected/4a00e5f4-7118-41ce-b4c7-eb0f74da8237-kube-api-access-42qrm\") pod \"maas-api-e2e-worker-w0-779f32-76b4d9d8c7-fxknc\" (UID: \"4a00e5f4-7118-41ce-b4c7-eb0f74da8237\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-779f32-76b4d9d8c7-fxknc"
Oct 30 22:03:59.331540 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:59.331521    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4a00e5f4-7118-41ce-b4c7-eb0f74da8237-metrics-certs\") pod \"maas-api-e2e-worker-w0-779f32-76b4d9d8c7-fxknc\" (UID: \"4a00e5f4-7118-41ce-b4c7-eb0f74da8237\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-779f32-76b4d9d8c7-fxknc"
Oct 30 22:03:59.433016 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:59.432973    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4a00e5f4-7118-41ce-b4c7-eb0f74da8237-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-779f32-76b4d9d8c7-fxknc\" (UID: \"4a00e5f4-7118-41ce-b4c7-eb0f74da8237\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-779f32-76b4d9d8c7-fxknc"
Oct 30 22:03:59.433016 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:59.433026    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-42qrm\" (UniqueName: \"kubernetes.io/projected/4a00e5f4-7118-41ce-b4c7-eb0f74da8237-kube-api-access-42qrm\") pod \"maas-api-e2e-worker-w0-779f32-76b4d9d8c7-fxknc\" (UID: \"4a00e5f4-7118-41ce-b4c7-eb0f74da8237\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-779f32-76b4d9d8c7-fxknc"
Oct 30 22:03:59.433248 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:59.433076    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4a00e5f4-7118-41ce-b4c7-eb0f74da8237-metrics-certs\") pod \"maas-api-e2e-worker-w0-779f32-76b4d9d8c7-fxknc\" (UID: \"4a00e5f4-7118-41ce-b4c7-eb0f74da8237\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-779f32-76b4d9d8c7-fxknc"
Oct 30 22:03:59.433248 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:59.433147    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4a00e5f4-7118-41ce-b4c7-eb0f74da8237-maas-api-tls\") pod \"maas-api-e2e-worker-w0-779f32-76b4d9d8c7-fxknc\" (UID: \"4a00e5f4-7118-41ce-b4c7-eb0f74da8237\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-779f32-76b4d9d8c7-fxknc"
Oct 30 22:03:59.433350 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:03:59.433265    5510 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w0-779f32-serving-cert: secret "maas-api-e2e-worker-w0-779f32-serving-cert" not found
Oct 30 22:03:59.433350 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:03:59.433331    5510 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a00e5f4-7118-41ce-b4c7-eb0f74da8237-maas-api-tls podName:4a00e5f4-7118-41ce-b4c7-eb0f74da8237 nodeName:}" failed. No retries permitted until 2026-10-30 22:03:59.933310337 +0000 UTC m=+820.494120334 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/4a00e5f4-7118-41ce-b4c7-eb0f74da8237-maas-api-tls") pod "maas-api-e2e-worker-w0-779f32-76b4d9d8c7-fxknc" (UID: "4a00e5f4-7118-41ce-b4c7-eb0f74da8237") : secret "maas-api-e2e-worker-w0-779f32-serving-cert" not found
Oct 30 22:03:59.433712 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:59.433652    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4a00e5f4-7118-41ce-b4c7-eb0f74da8237-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-779f32-76b4d9d8c7-fxknc\" (UID: \"4a00e5f4-7118-41ce-b4c7-eb0f74da8237\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-779f32-76b4d9d8c7-fxknc"
Oct 30 22:03:59.435516 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:59.435494    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4a00e5f4-7118-41ce-b4c7-eb0f74da8237-metrics-certs\") pod \"maas-api-e2e-worker-w0-779f32-76b4d9d8c7-fxknc\" (UID: \"4a00e5f4-7118-41ce-b4c7-eb0f74da8237\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-779f32-76b4d9d8c7-fxknc"
Oct 30 22:03:59.481525 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:59.481452    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:03:59.521169 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:59.518022    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-42qrm\" (UniqueName: \"kubernetes.io/projected/4a00e5f4-7118-41ce-b4c7-eb0f74da8237-kube-api-access-42qrm\") pod \"maas-api-e2e-worker-w0-779f32-76b4d9d8c7-fxknc\" (UID: \"4a00e5f4-7118-41ce-b4c7-eb0f74da8237\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-779f32-76b4d9d8c7-fxknc"
Oct 30 22:03:59.583321 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:59.583296    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-779f32-67bbb7bf95-sjnl9"]
Oct 30 22:03:59.585874 ip-10-0-136-210 kubenswrapper[5510]: W1030 22:03:59.585847    5510 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4fd056ae_943a_43c1_8682_8edb3fe3b5ed.slice/crio-8b2914f5436eca3bb7b62c78423554d4d4d8457ed9de72b5b2c45f774202c2b0 WatchSource:0}: Error finding container 8b2914f5436eca3bb7b62c78423554d4d4d8457ed9de72b5b2c45f774202c2b0: Status 404 returned error can't find the container with id 8b2914f5436eca3bb7b62c78423554d4d4d8457ed9de72b5b2c45f774202c2b0
Oct 30 22:03:59.696099 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:59.696067    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:03:59.891410 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:59.891376    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:03:59.946129 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:59.946055    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4a00e5f4-7118-41ce-b4c7-eb0f74da8237-maas-api-tls\") pod \"maas-api-e2e-worker-w0-779f32-76b4d9d8c7-fxknc\" (UID: \"4a00e5f4-7118-41ce-b4c7-eb0f74da8237\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-779f32-76b4d9d8c7-fxknc"
Oct 30 22:03:59.948510 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:59.948487    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4a00e5f4-7118-41ce-b4c7-eb0f74da8237-maas-api-tls\") pod \"maas-api-e2e-worker-w0-779f32-76b4d9d8c7-fxknc\" (UID: \"4a00e5f4-7118-41ce-b4c7-eb0f74da8237\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-779f32-76b4d9d8c7-fxknc"
Oct 30 22:03:59.984289 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:59.984257    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-779f32-67bbb7bf95-sjnl9" event={"ID":"4fd056ae-943a-43c1-8682-8edb3fe3b5ed","Type":"ContainerStarted","Data":"a46ab953c0354b4a66225f2364b7f14057e5153277721e749c2479e31b77ee2f"}
Oct 30 22:03:59.984532 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:59.984330    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-779f32-67bbb7bf95-sjnl9" event={"ID":"4fd056ae-943a-43c1-8682-8edb3fe3b5ed","Type":"ContainerStarted","Data":"8b2914f5436eca3bb7b62c78423554d4d4d8457ed9de72b5b2c45f774202c2b0"}
Oct 30 22:03:59.984532 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:03:59.984411    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-779f32-67bbb7bf95-sjnl9"
Oct 30 22:04:00.095137 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:00.095071    5510 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-779f32-67bbb7bf95-sjnl9" podStartSLOduration=2.095055182 podStartE2EDuration="2.095055182s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:03:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:04:00.094532911 +0000 UTC m=+820.655342916" watchObservedRunningTime="2026-10-30 22:04:00.095055182 +0000 UTC m=+820.655865188"
Oct 30 22:04:00.156040 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:00.155955    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-779f32-76b4d9d8c7-fxknc"
Oct 30 22:04:00.542076 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:00.541997    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-779f32-76b4d9d8c7-fxknc"]
Oct 30 22:04:00.544916 ip-10-0-136-210 kubenswrapper[5510]: W1030 22:04:00.544879    5510 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a00e5f4_7118_41ce_b4c7_eb0f74da8237.slice/crio-1f023c1e4a91ad636f7f96833905aec1d86ce21e3e7384695862888de03ecd97 WatchSource:0}: Error finding container 1f023c1e4a91ad636f7f96833905aec1d86ce21e3e7384695862888de03ecd97: Status 404 returned error can't find the container with id 1f023c1e4a91ad636f7f96833905aec1d86ce21e3e7384695862888de03ecd97
Oct 30 22:04:00.991054 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:00.990973    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-779f32-76b4d9d8c7-fxknc" event={"ID":"4a00e5f4-7118-41ce-b4c7-eb0f74da8237","Type":"ContainerStarted","Data":"1f023c1e4a91ad636f7f96833905aec1d86ce21e3e7384695862888de03ecd97"}
Oct 30 22:04:01.207969 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:01.207935    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:04:01.411284 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:01.411255    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:04:01.998538 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:01.998503    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-779f32-76b4d9d8c7-fxknc" event={"ID":"4a00e5f4-7118-41ce-b4c7-eb0f74da8237","Type":"ContainerStarted","Data":"aac40e09210b083d154b781f485361fc2c7c33a4a4a0d1179f43cee494f0ed4c"}
Oct 30 22:04:01.998942 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:01.998595    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-779f32-76b4d9d8c7-fxknc"
Oct 30 22:04:02.114363 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:02.114312    5510 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-779f32-76b4d9d8c7-fxknc" podStartSLOduration=2.281629661 podStartE2EDuration="3.114298056s" totalImagesPullingTime="832.668395ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:03:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:04:02.112880536 +0000 UTC m=+822.673690542" watchObservedRunningTime="2026-10-30 22:04:02.114298056 +0000 UTC m=+822.675108060"
Oct 30 22:04:02.976229 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:02.976201    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-bc714d-58cbbc478d-xsqpn"
Oct 30 22:04:03.884154 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:03.884112    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:04:04.083046 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:04.082992    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:04:04.738723 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:04.738680    5510 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-fb0b93-65f446cc49-89bdv"]
Oct 30 22:04:04.742515 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:04.742494    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-fb0b93-65f446cc49-89bdv"
Oct 30 22:04:04.792305 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:04.792273    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w6-fb0b93-serving-cert\""
Oct 30 22:04:04.814506 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:04.814477    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/50ad0ebb-47d3-476a-8b19-33d875cd0b1f-maas-api-tls\") pod \"maas-api-e2e-worker-w6-fb0b93-65f446cc49-89bdv\" (UID: \"50ad0ebb-47d3-476a-8b19-33d875cd0b1f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-fb0b93-65f446cc49-89bdv"
Oct 30 22:04:04.814691 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:04.814599    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/50ad0ebb-47d3-476a-8b19-33d875cd0b1f-openshift-service-ca\") pod \"maas-api-e2e-worker-w6-fb0b93-65f446cc49-89bdv\" (UID: \"50ad0ebb-47d3-476a-8b19-33d875cd0b1f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-fb0b93-65f446cc49-89bdv"
Oct 30 22:04:04.814691 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:04.814667    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/50ad0ebb-47d3-476a-8b19-33d875cd0b1f-metrics-certs\") pod \"maas-api-e2e-worker-w6-fb0b93-65f446cc49-89bdv\" (UID: \"50ad0ebb-47d3-476a-8b19-33d875cd0b1f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-fb0b93-65f446cc49-89bdv"
Oct 30 22:04:04.814691 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:04.814685    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jnmzx\" (UniqueName: \"kubernetes.io/projected/50ad0ebb-47d3-476a-8b19-33d875cd0b1f-kube-api-access-jnmzx\") pod \"maas-api-e2e-worker-w6-fb0b93-65f446cc49-89bdv\" (UID: \"50ad0ebb-47d3-476a-8b19-33d875cd0b1f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-fb0b93-65f446cc49-89bdv"
Oct 30 22:04:04.840943 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:04.840911    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-fb0b93-65f446cc49-89bdv"]
Oct 30 22:04:04.915617 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:04.915583    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/50ad0ebb-47d3-476a-8b19-33d875cd0b1f-openshift-service-ca\") pod \"maas-api-e2e-worker-w6-fb0b93-65f446cc49-89bdv\" (UID: \"50ad0ebb-47d3-476a-8b19-33d875cd0b1f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-fb0b93-65f446cc49-89bdv"
Oct 30 22:04:04.916031 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:04.915647    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/50ad0ebb-47d3-476a-8b19-33d875cd0b1f-metrics-certs\") pod \"maas-api-e2e-worker-w6-fb0b93-65f446cc49-89bdv\" (UID: \"50ad0ebb-47d3-476a-8b19-33d875cd0b1f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-fb0b93-65f446cc49-89bdv"
Oct 30 22:04:04.916031 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:04.915761    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jnmzx\" (UniqueName: \"kubernetes.io/projected/50ad0ebb-47d3-476a-8b19-33d875cd0b1f-kube-api-access-jnmzx\") pod \"maas-api-e2e-worker-w6-fb0b93-65f446cc49-89bdv\" (UID: \"50ad0ebb-47d3-476a-8b19-33d875cd0b1f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-fb0b93-65f446cc49-89bdv"
Oct 30 22:04:04.916031 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:04.915858    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/50ad0ebb-47d3-476a-8b19-33d875cd0b1f-maas-api-tls\") pod \"maas-api-e2e-worker-w6-fb0b93-65f446cc49-89bdv\" (UID: \"50ad0ebb-47d3-476a-8b19-33d875cd0b1f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-fb0b93-65f446cc49-89bdv"
Oct 30 22:04:04.916031 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:04:04.916000    5510 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w6-fb0b93-serving-cert: secret "maas-api-e2e-worker-w6-fb0b93-serving-cert" not found
Oct 30 22:04:04.916178 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:04:04.916072    5510 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/50ad0ebb-47d3-476a-8b19-33d875cd0b1f-maas-api-tls podName:50ad0ebb-47d3-476a-8b19-33d875cd0b1f nodeName:}" failed. No retries permitted until 2026-10-30 22:04:05.416049576 +0000 UTC m=+825.976859576 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/50ad0ebb-47d3-476a-8b19-33d875cd0b1f-maas-api-tls") pod "maas-api-e2e-worker-w6-fb0b93-65f446cc49-89bdv" (UID: "50ad0ebb-47d3-476a-8b19-33d875cd0b1f") : secret "maas-api-e2e-worker-w6-fb0b93-serving-cert" not found
Oct 30 22:04:04.916433 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:04.916417    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/50ad0ebb-47d3-476a-8b19-33d875cd0b1f-openshift-service-ca\") pod \"maas-api-e2e-worker-w6-fb0b93-65f446cc49-89bdv\" (UID: \"50ad0ebb-47d3-476a-8b19-33d875cd0b1f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-fb0b93-65f446cc49-89bdv"
Oct 30 22:04:04.918293 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:04.918265    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/50ad0ebb-47d3-476a-8b19-33d875cd0b1f-metrics-certs\") pod \"maas-api-e2e-worker-w6-fb0b93-65f446cc49-89bdv\" (UID: \"50ad0ebb-47d3-476a-8b19-33d875cd0b1f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-fb0b93-65f446cc49-89bdv"
Oct 30 22:04:04.976419 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:04.976387    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jnmzx\" (UniqueName: \"kubernetes.io/projected/50ad0ebb-47d3-476a-8b19-33d875cd0b1f-kube-api-access-jnmzx\") pod \"maas-api-e2e-worker-w6-fb0b93-65f446cc49-89bdv\" (UID: \"50ad0ebb-47d3-476a-8b19-33d875cd0b1f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-fb0b93-65f446cc49-89bdv"
Oct 30 22:04:04.997056 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:04.996977    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:04:05.205130 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:05.205098    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:04:05.426310 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:05.426273    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/50ad0ebb-47d3-476a-8b19-33d875cd0b1f-maas-api-tls\") pod \"maas-api-e2e-worker-w6-fb0b93-65f446cc49-89bdv\" (UID: \"50ad0ebb-47d3-476a-8b19-33d875cd0b1f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-fb0b93-65f446cc49-89bdv"
Oct 30 22:04:05.429178 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:05.429147    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/50ad0ebb-47d3-476a-8b19-33d875cd0b1f-maas-api-tls\") pod \"maas-api-e2e-worker-w6-fb0b93-65f446cc49-89bdv\" (UID: \"50ad0ebb-47d3-476a-8b19-33d875cd0b1f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-fb0b93-65f446cc49-89bdv"
Oct 30 22:04:05.429178 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:05.429151    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:04:05.654760 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:05.654725    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-fb0b93-65f446cc49-89bdv"
Oct 30 22:04:05.960077 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:05.959981    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-c9dbe6-6986bd68b7-2kg6w"
Oct 30 22:04:05.960506 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:05.960157    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w5-c9dbe6-584f9bc9cf-z2ph8"
Oct 30 22:04:06.026104 ip-10-0-136-210 kubenswrapper[5510]: W1030 22:04:06.026071    5510 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50ad0ebb_47d3_476a_8b19_33d875cd0b1f.slice/crio-9787158922bf01d3fc06c6654822709e5fc99aa24052453e20f96299d858b68c WatchSource:0}: Error finding container 9787158922bf01d3fc06c6654822709e5fc99aa24052453e20f96299d858b68c: Status 404 returned error can't find the container with id 9787158922bf01d3fc06c6654822709e5fc99aa24052453e20f96299d858b68c
Oct 30 22:04:06.059014 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:06.058983    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-fb0b93-65f446cc49-89bdv"]
Oct 30 22:04:06.627198 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:06.627164    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:04:06.881953 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:06.881893    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:04:07.034952 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:07.034917    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-fb0b93-65f446cc49-89bdv" event={"ID":"50ad0ebb-47d3-476a-8b19-33d875cd0b1f","Type":"ContainerStarted","Data":"35e95cc60a10e1a0e206f56a33188e2abf680825405c912866a191799285fe67"}
Oct 30 22:04:07.035386 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:07.034971    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-fb0b93-65f446cc49-89bdv" event={"ID":"50ad0ebb-47d3-476a-8b19-33d875cd0b1f","Type":"ContainerStarted","Data":"9787158922bf01d3fc06c6654822709e5fc99aa24052453e20f96299d858b68c"}
Oct 30 22:04:07.035386 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:07.035063    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-fb0b93-65f446cc49-89bdv"
Oct 30 22:04:08.009055 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:08.009027    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-779f32-76b4d9d8c7-fxknc"
Oct 30 22:04:08.124684 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:08.124595    5510 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-fb0b93-65f446cc49-89bdv" podStartSLOduration=3.29937947 podStartE2EDuration="4.124574009s" totalImagesPullingTime="825.194539ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:04:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:04:07.159783726 +0000 UTC m=+827.720593745" watchObservedRunningTime="2026-10-30 22:04:08.124574009 +0000 UTC m=+828.685384016"
Oct 30 22:04:08.419512 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:04:08.419477    5510 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6gjbf" containerName="extractor"
Oct 30 22:04:09.282210 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:09.282175    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:04:09.561731 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:09.553300    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:04:10.138358 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:10.138318    5510 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-a00d9b-d9c558db-ltxtj"]
Oct 30 22:04:10.142409 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:10.142384    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-a00d9b-d9c558db-ltxtj"
Oct 30 22:04:10.192216 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:10.192187    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w1-a00d9b-serving-cert\""
Oct 30 22:04:10.240999 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:10.240964    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-a00d9b-d9c558db-ltxtj"]
Oct 30 22:04:10.298718 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:10.298678    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/32dcff45-2112-479a-847a-43c0c9b26de4-maas-api-tls\") pod \"maas-api-e2e-worker-w1-a00d9b-d9c558db-ltxtj\" (UID: \"32dcff45-2112-479a-847a-43c0c9b26de4\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-a00d9b-d9c558db-ltxtj"
Oct 30 22:04:10.300894 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:10.298843    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/32dcff45-2112-479a-847a-43c0c9b26de4-openshift-service-ca\") pod \"maas-api-e2e-worker-w1-a00d9b-d9c558db-ltxtj\" (UID: \"32dcff45-2112-479a-847a-43c0c9b26de4\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-a00d9b-d9c558db-ltxtj"
Oct 30 22:04:10.300894 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:10.298904    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-985mk\" (UniqueName: \"kubernetes.io/projected/32dcff45-2112-479a-847a-43c0c9b26de4-kube-api-access-985mk\") pod \"maas-api-e2e-worker-w1-a00d9b-d9c558db-ltxtj\" (UID: \"32dcff45-2112-479a-847a-43c0c9b26de4\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-a00d9b-d9c558db-ltxtj"
Oct 30 22:04:10.300894 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:10.298962    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/32dcff45-2112-479a-847a-43c0c9b26de4-metrics-certs\") pod \"maas-api-e2e-worker-w1-a00d9b-d9c558db-ltxtj\" (UID: \"32dcff45-2112-479a-847a-43c0c9b26de4\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-a00d9b-d9c558db-ltxtj"
Oct 30 22:04:10.379564 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:10.379529    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:04:10.400211 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:10.400123    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-985mk\" (UniqueName: \"kubernetes.io/projected/32dcff45-2112-479a-847a-43c0c9b26de4-kube-api-access-985mk\") pod \"maas-api-e2e-worker-w1-a00d9b-d9c558db-ltxtj\" (UID: \"32dcff45-2112-479a-847a-43c0c9b26de4\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-a00d9b-d9c558db-ltxtj"
Oct 30 22:04:10.400211 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:10.400183    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/32dcff45-2112-479a-847a-43c0c9b26de4-metrics-certs\") pod \"maas-api-e2e-worker-w1-a00d9b-d9c558db-ltxtj\" (UID: \"32dcff45-2112-479a-847a-43c0c9b26de4\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-a00d9b-d9c558db-ltxtj"
Oct 30 22:04:10.400438 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:10.400260    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/32dcff45-2112-479a-847a-43c0c9b26de4-maas-api-tls\") pod \"maas-api-e2e-worker-w1-a00d9b-d9c558db-ltxtj\" (UID: \"32dcff45-2112-479a-847a-43c0c9b26de4\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-a00d9b-d9c558db-ltxtj"
Oct 30 22:04:10.400438 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:10.400327    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/32dcff45-2112-479a-847a-43c0c9b26de4-openshift-service-ca\") pod \"maas-api-e2e-worker-w1-a00d9b-d9c558db-ltxtj\" (UID: \"32dcff45-2112-479a-847a-43c0c9b26de4\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-a00d9b-d9c558db-ltxtj"
Oct 30 22:04:10.401044 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:10.401024    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/32dcff45-2112-479a-847a-43c0c9b26de4-openshift-service-ca\") pod \"maas-api-e2e-worker-w1-a00d9b-d9c558db-ltxtj\" (UID: \"32dcff45-2112-479a-847a-43c0c9b26de4\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-a00d9b-d9c558db-ltxtj"
Oct 30 22:04:10.402828 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:10.402804    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/32dcff45-2112-479a-847a-43c0c9b26de4-maas-api-tls\") pod \"maas-api-e2e-worker-w1-a00d9b-d9c558db-ltxtj\" (UID: \"32dcff45-2112-479a-847a-43c0c9b26de4\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-a00d9b-d9c558db-ltxtj"
Oct 30 22:04:10.402948 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:10.402848    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/32dcff45-2112-479a-847a-43c0c9b26de4-metrics-certs\") pod \"maas-api-e2e-worker-w1-a00d9b-d9c558db-ltxtj\" (UID: \"32dcff45-2112-479a-847a-43c0c9b26de4\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-a00d9b-d9c558db-ltxtj"
Oct 30 22:04:10.458779 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:10.458744    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-985mk\" (UniqueName: \"kubernetes.io/projected/32dcff45-2112-479a-847a-43c0c9b26de4-kube-api-access-985mk\") pod \"maas-api-e2e-worker-w1-a00d9b-d9c558db-ltxtj\" (UID: \"32dcff45-2112-479a-847a-43c0c9b26de4\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-a00d9b-d9c558db-ltxtj"
Oct 30 22:04:10.577862 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:10.577829    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:04:10.755760 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:10.755652    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-a00d9b-d9c558db-ltxtj"
Oct 30 22:04:10.771861 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:10.771826    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:04:10.998455 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:10.998422    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-779f32-67bbb7bf95-sjnl9"
Oct 30 22:04:11.144102 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:11.144066    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-a00d9b-d9c558db-ltxtj"]
Oct 30 22:04:12.061239 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:12.061206    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-a00d9b-d9c558db-ltxtj" event={"ID":"32dcff45-2112-479a-847a-43c0c9b26de4","Type":"ContainerStarted","Data":"a194793a5514d201d346de0af5437b6f0fc7031fc5c0eee55985d3513bbaeb9d"}
Oct 30 22:04:13.045402 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:13.045373    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-fb0b93-65f446cc49-89bdv"
Oct 30 22:04:13.068012 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:13.067978    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-a00d9b-d9c558db-ltxtj" event={"ID":"32dcff45-2112-479a-847a-43c0c9b26de4","Type":"ContainerStarted","Data":"8c64fe4357270d89ad89d124f56e1e7d4be311492acb8ebc8a1f4e7967bb6f76"}
Oct 30 22:04:13.068405 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:13.068074    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-a00d9b-d9c558db-ltxtj"
Oct 30 22:04:13.270451 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:13.270401    5510 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-a00d9b-d9c558db-ltxtj" podStartSLOduration=2.334586941 podStartE2EDuration="3.270386731s" totalImagesPullingTime="935.79979ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:04:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:04:13.269120994 +0000 UTC m=+833.829930999" watchObservedRunningTime="2026-10-30 22:04:13.270386731 +0000 UTC m=+833.831196808"
Oct 30 22:04:15.908503 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:15.908470    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:04:19.078519 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:19.078489    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-a00d9b-d9c558db-ltxtj"
Oct 30 22:04:20.930587 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:20.930556    5510 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-7bb1c2-6746d48b7f-8bqqt"]
Oct 30 22:04:20.935624 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:20.935602    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-7bb1c2-6746d48b7f-8bqqt"
Oct 30 22:04:20.985481 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:20.985450    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w3-7bb1c2-serving-cert\""
Oct 30 22:04:20.994012 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:20.993987    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-7bb1c2-6746d48b7f-8bqqt"]
Oct 30 22:04:21.036569 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:21.036532    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2d9nr\" (UniqueName: \"kubernetes.io/projected/3266de62-2593-4496-b3e6-7600ebb710ce-kube-api-access-2d9nr\") pod \"maas-api-e2e-worker-w3-7bb1c2-6746d48b7f-8bqqt\" (UID: \"3266de62-2593-4496-b3e6-7600ebb710ce\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-7bb1c2-6746d48b7f-8bqqt"
Oct 30 22:04:21.036742 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:21.036576    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3266de62-2593-4496-b3e6-7600ebb710ce-openshift-service-ca\") pod \"maas-api-e2e-worker-w3-7bb1c2-6746d48b7f-8bqqt\" (UID: \"3266de62-2593-4496-b3e6-7600ebb710ce\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-7bb1c2-6746d48b7f-8bqqt"
Oct 30 22:04:21.036796 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:21.036751    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3266de62-2593-4496-b3e6-7600ebb710ce-metrics-certs\") pod \"maas-api-e2e-worker-w3-7bb1c2-6746d48b7f-8bqqt\" (UID: \"3266de62-2593-4496-b3e6-7600ebb710ce\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-7bb1c2-6746d48b7f-8bqqt"
Oct 30 22:04:21.036852 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:21.036811    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3266de62-2593-4496-b3e6-7600ebb710ce-maas-api-tls\") pod \"maas-api-e2e-worker-w3-7bb1c2-6746d48b7f-8bqqt\" (UID: \"3266de62-2593-4496-b3e6-7600ebb710ce\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-7bb1c2-6746d48b7f-8bqqt"
Oct 30 22:04:21.137586 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:21.137549    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2d9nr\" (UniqueName: \"kubernetes.io/projected/3266de62-2593-4496-b3e6-7600ebb710ce-kube-api-access-2d9nr\") pod \"maas-api-e2e-worker-w3-7bb1c2-6746d48b7f-8bqqt\" (UID: \"3266de62-2593-4496-b3e6-7600ebb710ce\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-7bb1c2-6746d48b7f-8bqqt"
Oct 30 22:04:21.137799 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:21.137601    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3266de62-2593-4496-b3e6-7600ebb710ce-openshift-service-ca\") pod \"maas-api-e2e-worker-w3-7bb1c2-6746d48b7f-8bqqt\" (UID: \"3266de62-2593-4496-b3e6-7600ebb710ce\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-7bb1c2-6746d48b7f-8bqqt"
Oct 30 22:04:21.137799 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:21.137734    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3266de62-2593-4496-b3e6-7600ebb710ce-metrics-certs\") pod \"maas-api-e2e-worker-w3-7bb1c2-6746d48b7f-8bqqt\" (UID: \"3266de62-2593-4496-b3e6-7600ebb710ce\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-7bb1c2-6746d48b7f-8bqqt"
Oct 30 22:04:21.137917 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:21.137798    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3266de62-2593-4496-b3e6-7600ebb710ce-maas-api-tls\") pod \"maas-api-e2e-worker-w3-7bb1c2-6746d48b7f-8bqqt\" (UID: \"3266de62-2593-4496-b3e6-7600ebb710ce\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-7bb1c2-6746d48b7f-8bqqt"
Oct 30 22:04:21.137959 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:04:21.137943    5510 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w3-7bb1c2-serving-cert: secret "maas-api-e2e-worker-w3-7bb1c2-serving-cert" not found
Oct 30 22:04:21.138021 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:04:21.138010    5510 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3266de62-2593-4496-b3e6-7600ebb710ce-maas-api-tls podName:3266de62-2593-4496-b3e6-7600ebb710ce nodeName:}" failed. No retries permitted until 2026-10-30 22:04:21.637993758 +0000 UTC m=+842.198803746 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/3266de62-2593-4496-b3e6-7600ebb710ce-maas-api-tls") pod "maas-api-e2e-worker-w3-7bb1c2-6746d48b7f-8bqqt" (UID: "3266de62-2593-4496-b3e6-7600ebb710ce") : secret "maas-api-e2e-worker-w3-7bb1c2-serving-cert" not found
Oct 30 22:04:21.138288 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:21.138267    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3266de62-2593-4496-b3e6-7600ebb710ce-openshift-service-ca\") pod \"maas-api-e2e-worker-w3-7bb1c2-6746d48b7f-8bqqt\" (UID: \"3266de62-2593-4496-b3e6-7600ebb710ce\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-7bb1c2-6746d48b7f-8bqqt"
Oct 30 22:04:21.140160 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:21.140138    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3266de62-2593-4496-b3e6-7600ebb710ce-metrics-certs\") pod \"maas-api-e2e-worker-w3-7bb1c2-6746d48b7f-8bqqt\" (UID: \"3266de62-2593-4496-b3e6-7600ebb710ce\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-7bb1c2-6746d48b7f-8bqqt"
Oct 30 22:04:21.163840 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:21.163812    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:04:21.196201 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:21.196137    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2d9nr\" (UniqueName: \"kubernetes.io/projected/3266de62-2593-4496-b3e6-7600ebb710ce-kube-api-access-2d9nr\") pod \"maas-api-e2e-worker-w3-7bb1c2-6746d48b7f-8bqqt\" (UID: \"3266de62-2593-4496-b3e6-7600ebb710ce\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-7bb1c2-6746d48b7f-8bqqt"
Oct 30 22:04:21.372438 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:21.372406    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:04:21.569017 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:21.568975    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:04:21.652706 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:21.652649    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3266de62-2593-4496-b3e6-7600ebb710ce-maas-api-tls\") pod \"maas-api-e2e-worker-w3-7bb1c2-6746d48b7f-8bqqt\" (UID: \"3266de62-2593-4496-b3e6-7600ebb710ce\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-7bb1c2-6746d48b7f-8bqqt"
Oct 30 22:04:21.655091 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:21.655065    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3266de62-2593-4496-b3e6-7600ebb710ce-maas-api-tls\") pod \"maas-api-e2e-worker-w3-7bb1c2-6746d48b7f-8bqqt\" (UID: \"3266de62-2593-4496-b3e6-7600ebb710ce\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-7bb1c2-6746d48b7f-8bqqt"
Oct 30 22:04:21.848757 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:21.848676    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-7bb1c2-6746d48b7f-8bqqt"
Oct 30 22:04:22.226445 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:22.226419    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-7bb1c2-6746d48b7f-8bqqt"]
Oct 30 22:04:22.229157 ip-10-0-136-210 kubenswrapper[5510]: W1030 22:04:22.229129    5510 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3266de62_2593_4496_b3e6_7600ebb710ce.slice/crio-a03783270fec980133b005b05c3f832b2ad717a95a20f4b2419b214b6c5f466e WatchSource:0}: Error finding container a03783270fec980133b005b05c3f832b2ad717a95a20f4b2419b214b6c5f466e: Status 404 returned error can't find the container with id a03783270fec980133b005b05c3f832b2ad717a95a20f4b2419b214b6c5f466e
Oct 30 22:04:22.395960 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:22.395923    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:04:23.121486 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:23.121457    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-7bb1c2-6746d48b7f-8bqqt" event={"ID":"3266de62-2593-4496-b3e6-7600ebb710ce","Type":"ContainerStarted","Data":"a03783270fec980133b005b05c3f832b2ad717a95a20f4b2419b214b6c5f466e"}
Oct 30 22:04:24.128546 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:24.128506    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-7bb1c2-6746d48b7f-8bqqt" event={"ID":"3266de62-2593-4496-b3e6-7600ebb710ce","Type":"ContainerStarted","Data":"6410f2ac44e9b1c8e76dcf326d66cd9b1d2bcf9b7d51e52dc499695b93a5df4f"}
Oct 30 22:04:24.128978 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:24.128618    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-7bb1c2-6746d48b7f-8bqqt"
Oct 30 22:04:24.241159 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:24.241092    5510 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-7bb1c2-6746d48b7f-8bqqt" podStartSLOduration=3.394150594 podStartE2EDuration="4.241072516s" totalImagesPullingTime="846.921922ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:04:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:04:24.240806062 +0000 UTC m=+844.801616067" watchObservedRunningTime="2026-10-30 22:04:24.241072516 +0000 UTC m=+844.801882524"
Oct 30 22:04:26.381771 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:26.381727    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:04:26.579413 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:26.579372    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:04:26.855874 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:26.855839    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:04:29.203262 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:29.203225    5510 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-lggs2"]
Oct 30 22:04:29.208884 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:29.208861    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-lggs2"
Oct 30 22:04:29.322422 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:29.322386    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-lggs2"]
Oct 30 22:04:29.342258 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:29.342215    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a3a8e626-b7fa-4fe1-8d74-0d8bef87fb63-istiod-ca-cert\") pod \"e2e-worker-w1-a00d9b-openshift-default-847df8fc59-lggs2\" (UID: \"a3a8e626-b7fa-4fe1-8d74-0d8bef87fb63\") " pod="openshift-ingress/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-lggs2"
Oct 30 22:04:29.342258 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:29.342262    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a3a8e626-b7fa-4fe1-8d74-0d8bef87fb63-workload-certs\") pod \"e2e-worker-w1-a00d9b-openshift-default-847df8fc59-lggs2\" (UID: \"a3a8e626-b7fa-4fe1-8d74-0d8bef87fb63\") " pod="openshift-ingress/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-lggs2"
Oct 30 22:04:29.342476 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:29.342434    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3a8e626-b7fa-4fe1-8d74-0d8bef87fb63-credential-socket\") pod \"e2e-worker-w1-a00d9b-openshift-default-847df8fc59-lggs2\" (UID: \"a3a8e626-b7fa-4fe1-8d74-0d8bef87fb63\") " pod="openshift-ingress/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-lggs2"
Oct 30 22:04:29.342537 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:29.342525    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a3a8e626-b7fa-4fe1-8d74-0d8bef87fb63-istio-podinfo\") pod \"e2e-worker-w1-a00d9b-openshift-default-847df8fc59-lggs2\" (UID: \"a3a8e626-b7fa-4fe1-8d74-0d8bef87fb63\") " pod="openshift-ingress/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-lggs2"
Oct 30 22:04:29.342584 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:29.342565    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3a8e626-b7fa-4fe1-8d74-0d8bef87fb63-workload-socket\") pod \"e2e-worker-w1-a00d9b-openshift-default-847df8fc59-lggs2\" (UID: \"a3a8e626-b7fa-4fe1-8d74-0d8bef87fb63\") " pod="openshift-ingress/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-lggs2"
Oct 30 22:04:29.342715 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:29.342649    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lsgdn\" (UniqueName: \"kubernetes.io/projected/a3a8e626-b7fa-4fe1-8d74-0d8bef87fb63-kube-api-access-lsgdn\") pod \"e2e-worker-w1-a00d9b-openshift-default-847df8fc59-lggs2\" (UID: \"a3a8e626-b7fa-4fe1-8d74-0d8bef87fb63\") " pod="openshift-ingress/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-lggs2"
Oct 30 22:04:29.342787 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:29.342770    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a3a8e626-b7fa-4fe1-8d74-0d8bef87fb63-istio-envoy\") pod \"e2e-worker-w1-a00d9b-openshift-default-847df8fc59-lggs2\" (UID: \"a3a8e626-b7fa-4fe1-8d74-0d8bef87fb63\") " pod="openshift-ingress/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-lggs2"
Oct 30 22:04:29.342852 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:29.342836    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a3a8e626-b7fa-4fe1-8d74-0d8bef87fb63-istio-data\") pod \"e2e-worker-w1-a00d9b-openshift-default-847df8fc59-lggs2\" (UID: \"a3a8e626-b7fa-4fe1-8d74-0d8bef87fb63\") " pod="openshift-ingress/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-lggs2"
Oct 30 22:04:29.342905 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:29.342877    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a3a8e626-b7fa-4fe1-8d74-0d8bef87fb63-istio-token\") pod \"e2e-worker-w1-a00d9b-openshift-default-847df8fc59-lggs2\" (UID: \"a3a8e626-b7fa-4fe1-8d74-0d8bef87fb63\") " pod="openshift-ingress/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-lggs2"
Oct 30 22:04:29.370119 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:29.370086    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w1-a00d9b-openshift-default-dockercfg-w7t4n\""
Oct 30 22:04:29.466747 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:29.464033    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3a8e626-b7fa-4fe1-8d74-0d8bef87fb63-credential-socket\") pod \"e2e-worker-w1-a00d9b-openshift-default-847df8fc59-lggs2\" (UID: \"a3a8e626-b7fa-4fe1-8d74-0d8bef87fb63\") " pod="openshift-ingress/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-lggs2"
Oct 30 22:04:29.466747 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:29.464162    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a3a8e626-b7fa-4fe1-8d74-0d8bef87fb63-istio-podinfo\") pod \"e2e-worker-w1-a00d9b-openshift-default-847df8fc59-lggs2\" (UID: \"a3a8e626-b7fa-4fe1-8d74-0d8bef87fb63\") " pod="openshift-ingress/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-lggs2"
Oct 30 22:04:29.467176 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:29.467142    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3a8e626-b7fa-4fe1-8d74-0d8bef87fb63-credential-socket\") pod \"e2e-worker-w1-a00d9b-openshift-default-847df8fc59-lggs2\" (UID: \"a3a8e626-b7fa-4fe1-8d74-0d8bef87fb63\") " pod="openshift-ingress/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-lggs2"
Oct 30 22:04:29.468244 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:29.468217    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3a8e626-b7fa-4fe1-8d74-0d8bef87fb63-workload-socket\") pod \"e2e-worker-w1-a00d9b-openshift-default-847df8fc59-lggs2\" (UID: \"a3a8e626-b7fa-4fe1-8d74-0d8bef87fb63\") " pod="openshift-ingress/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-lggs2"
Oct 30 22:04:29.468729 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:29.468700    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3a8e626-b7fa-4fe1-8d74-0d8bef87fb63-workload-socket\") pod \"e2e-worker-w1-a00d9b-openshift-default-847df8fc59-lggs2\" (UID: \"a3a8e626-b7fa-4fe1-8d74-0d8bef87fb63\") " pod="openshift-ingress/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-lggs2"
Oct 30 22:04:29.468834 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:29.468774    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lsgdn\" (UniqueName: \"kubernetes.io/projected/a3a8e626-b7fa-4fe1-8d74-0d8bef87fb63-kube-api-access-lsgdn\") pod \"e2e-worker-w1-a00d9b-openshift-default-847df8fc59-lggs2\" (UID: \"a3a8e626-b7fa-4fe1-8d74-0d8bef87fb63\") " pod="openshift-ingress/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-lggs2"
Oct 30 22:04:29.468834 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:29.468809    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a3a8e626-b7fa-4fe1-8d74-0d8bef87fb63-istio-envoy\") pod \"e2e-worker-w1-a00d9b-openshift-default-847df8fc59-lggs2\" (UID: \"a3a8e626-b7fa-4fe1-8d74-0d8bef87fb63\") " pod="openshift-ingress/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-lggs2"
Oct 30 22:04:29.468997 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:29.468970    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a3a8e626-b7fa-4fe1-8d74-0d8bef87fb63-istio-data\") pod \"e2e-worker-w1-a00d9b-openshift-default-847df8fc59-lggs2\" (UID: \"a3a8e626-b7fa-4fe1-8d74-0d8bef87fb63\") " pod="openshift-ingress/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-lggs2"
Oct 30 22:04:29.469062 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:29.469040    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a3a8e626-b7fa-4fe1-8d74-0d8bef87fb63-istio-token\") pod \"e2e-worker-w1-a00d9b-openshift-default-847df8fc59-lggs2\" (UID: \"a3a8e626-b7fa-4fe1-8d74-0d8bef87fb63\") " pod="openshift-ingress/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-lggs2"
Oct 30 22:04:29.469121 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:29.469090    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a3a8e626-b7fa-4fe1-8d74-0d8bef87fb63-istiod-ca-cert\") pod \"e2e-worker-w1-a00d9b-openshift-default-847df8fc59-lggs2\" (UID: \"a3a8e626-b7fa-4fe1-8d74-0d8bef87fb63\") " pod="openshift-ingress/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-lggs2"
Oct 30 22:04:29.469184 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:29.469137    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a3a8e626-b7fa-4fe1-8d74-0d8bef87fb63-workload-certs\") pod \"e2e-worker-w1-a00d9b-openshift-default-847df8fc59-lggs2\" (UID: \"a3a8e626-b7fa-4fe1-8d74-0d8bef87fb63\") " pod="openshift-ingress/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-lggs2"
Oct 30 22:04:29.469263 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:29.469242    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a3a8e626-b7fa-4fe1-8d74-0d8bef87fb63-istio-data\") pod \"e2e-worker-w1-a00d9b-openshift-default-847df8fc59-lggs2\" (UID: \"a3a8e626-b7fa-4fe1-8d74-0d8bef87fb63\") " pod="openshift-ingress/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-lggs2"
Oct 30 22:04:29.469500 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:29.469477    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a3a8e626-b7fa-4fe1-8d74-0d8bef87fb63-workload-certs\") pod \"e2e-worker-w1-a00d9b-openshift-default-847df8fc59-lggs2\" (UID: \"a3a8e626-b7fa-4fe1-8d74-0d8bef87fb63\") " pod="openshift-ingress/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-lggs2"
Oct 30 22:04:29.469834 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:29.469814    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a3a8e626-b7fa-4fe1-8d74-0d8bef87fb63-istiod-ca-cert\") pod \"e2e-worker-w1-a00d9b-openshift-default-847df8fc59-lggs2\" (UID: \"a3a8e626-b7fa-4fe1-8d74-0d8bef87fb63\") " pod="openshift-ingress/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-lggs2"
Oct 30 22:04:29.470688 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:29.470638    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a3a8e626-b7fa-4fe1-8d74-0d8bef87fb63-istio-podinfo\") pod \"e2e-worker-w1-a00d9b-openshift-default-847df8fc59-lggs2\" (UID: \"a3a8e626-b7fa-4fe1-8d74-0d8bef87fb63\") " pod="openshift-ingress/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-lggs2"
Oct 30 22:04:29.471542 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:29.471522    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a3a8e626-b7fa-4fe1-8d74-0d8bef87fb63-istio-envoy\") pod \"e2e-worker-w1-a00d9b-openshift-default-847df8fc59-lggs2\" (UID: \"a3a8e626-b7fa-4fe1-8d74-0d8bef87fb63\") " pod="openshift-ingress/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-lggs2"
Oct 30 22:04:29.527650 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:29.527615    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a3a8e626-b7fa-4fe1-8d74-0d8bef87fb63-istio-token\") pod \"e2e-worker-w1-a00d9b-openshift-default-847df8fc59-lggs2\" (UID: \"a3a8e626-b7fa-4fe1-8d74-0d8bef87fb63\") " pod="openshift-ingress/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-lggs2"
Oct 30 22:04:29.527857 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:29.527833    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lsgdn\" (UniqueName: \"kubernetes.io/projected/a3a8e626-b7fa-4fe1-8d74-0d8bef87fb63-kube-api-access-lsgdn\") pod \"e2e-worker-w1-a00d9b-openshift-default-847df8fc59-lggs2\" (UID: \"a3a8e626-b7fa-4fe1-8d74-0d8bef87fb63\") " pod="openshift-ingress/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-lggs2"
Oct 30 22:04:29.821851 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:29.821805    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-lggs2"
Oct 30 22:04:30.140023 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:30.139954    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-7bb1c2-6746d48b7f-8bqqt"
Oct 30 22:04:30.303114 ip-10-0-136-210 kubenswrapper[5510]: W1030 22:04:30.303082    5510 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3a8e626_b7fa_4fe1_8d74_0d8bef87fb63.slice/crio-e1f005c2683ab35f2e1955016772aa195b783b6212395c735d1cda5f832bdb4b WatchSource:0}: Error finding container e1f005c2683ab35f2e1955016772aa195b783b6212395c735d1cda5f832bdb4b: Status 404 returned error can't find the container with id e1f005c2683ab35f2e1955016772aa195b783b6212395c735d1cda5f832bdb4b
Oct 30 22:04:30.305538 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:30.305504    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:04:30.305604 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:30.305576    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:04:30.305643 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:30.305609    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:04:30.306315 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:30.306291    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-lggs2"]
Oct 30 22:04:31.196031 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:31.195711    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-lggs2" event={"ID":"a3a8e626-b7fa-4fe1-8d74-0d8bef87fb63","Type":"ContainerStarted","Data":"5540c53ae933ff5b3eb69e9f0fb8a7ac5ed9a36279ae4079ea91a6f84f93b83c"}
Oct 30 22:04:31.196031 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:31.195786    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-lggs2" event={"ID":"a3a8e626-b7fa-4fe1-8d74-0d8bef87fb63","Type":"ContainerStarted","Data":"e1f005c2683ab35f2e1955016772aa195b783b6212395c735d1cda5f832bdb4b"}
Oct 30 22:04:31.434495 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:31.430683    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:04:31.629105 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:31.629069    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:04:31.822833 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:31.822799    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-lggs2"
Oct 30 22:04:31.824113 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:31.824087    5510 patch_prober.go:29] interesting pod/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-lggs2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body=
Oct 30 22:04:31.824256 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:31.824133    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-lggs2" podUID="a3a8e626-b7fa-4fe1-8d74-0d8bef87fb63" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused"
Oct 30 22:04:31.824547 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:31.824526    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:04:32.822448 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:32.822410    5510 patch_prober.go:29] interesting pod/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-lggs2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body=
Oct 30 22:04:32.823059 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:32.822466    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-lggs2" podUID="a3a8e626-b7fa-4fe1-8d74-0d8bef87fb63" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused"
Oct 30 22:04:33.823131 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:33.822778    5510 patch_prober.go:29] interesting pod/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-lggs2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body=
Oct 30 22:04:33.823131 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:33.822844    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-lggs2" podUID="a3a8e626-b7fa-4fe1-8d74-0d8bef87fb63" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused"
Oct 30 22:04:34.822950 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:34.822916    5510 patch_prober.go:29] interesting pod/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-lggs2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body=
Oct 30 22:04:34.823042 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:34.822988    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-lggs2" podUID="a3a8e626-b7fa-4fe1-8d74-0d8bef87fb63" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused"
Oct 30 22:04:35.822376 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:35.822339    5510 patch_prober.go:29] interesting pod/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-lggs2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body=
Oct 30 22:04:35.822801 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:35.822400    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-lggs2" podUID="a3a8e626-b7fa-4fe1-8d74-0d8bef87fb63" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused"
Oct 30 22:04:36.827611 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:36.827577    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-lggs2"
Oct 30 22:04:36.828094 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:36.828038    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-lggs2"
Oct 30 22:04:36.828829 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:36.828809    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-lggs2"
Oct 30 22:04:36.859812 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:36.859781    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:04:37.004852 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:37.004789    5510 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-lggs2" podStartSLOduration=8.004771088 podStartE2EDuration="8.004771088s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:04:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:04:31.395958626 +0000 UTC m=+851.956768652" watchObservedRunningTime="2026-10-30 22:04:37.004771088 +0000 UTC m=+857.565581094"
Oct 30 22:04:38.419417 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:04:38.419382    5510 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6gjbf" containerName="extractor"
Oct 30 22:04:39.239446 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:39.239417    5510 generic.go:356] "Generic (PLEG): container finished" podID="caef4e8d-7dcc-455a-94a2-4bb341662a88" containerID="0bae0294bc1a482d1e9171428212b20af595fa61ee28059daa35724bb102a910" exitCode=0
Oct 30 22:04:39.239621 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:39.239496    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-ccsc2" event={"ID":"caef4e8d-7dcc-455a-94a2-4bb341662a88","Type":"ContainerDied","Data":"0bae0294bc1a482d1e9171428212b20af595fa61ee28059daa35724bb102a910"}
Oct 30 22:04:39.239938 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:39.239923    5510 scope.go:122] "RemoveContainer" containerID="0bae0294bc1a482d1e9171428212b20af595fa61ee28059daa35724bb102a910"
Oct 30 22:04:39.241369 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:39.241341    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:04:39.241430 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:39.241410    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:04:39.241467 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:39.241441    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:04:40.262682 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:40.260176    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-ccsc2" event={"ID":"caef4e8d-7dcc-455a-94a2-4bb341662a88","Type":"ContainerStarted","Data":"6bcb261945056982cd70d737d4fd06cd141d27c20163e0534990323500d8654d"}
Oct 30 22:04:40.901029 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:40.900982    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-ccsc2"
Oct 30 22:04:40.901252 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:40.901196    5510 patch_prober.go:29] interesting pod/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-ccsc2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" start-of-body=
Oct 30 22:04:40.901344 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:40.901249    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-ccsc2" podUID="caef4e8d-7dcc-455a-94a2-4bb341662a88" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused"
Oct 30 22:04:41.901436 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:41.901395    5510 patch_prober.go:29] interesting pod/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-ccsc2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" start-of-body=
Oct 30 22:04:41.901948 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:41.901453    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-ccsc2" podUID="caef4e8d-7dcc-455a-94a2-4bb341662a88" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused"
Oct 30 22:04:42.020356 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:42.020318    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:04:42.901931 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:42.901897    5510 patch_prober.go:29] interesting pod/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-ccsc2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" start-of-body=
Oct 30 22:04:42.902345 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:42.901963    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-ccsc2" podUID="caef4e8d-7dcc-455a-94a2-4bb341662a88" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused"
Oct 30 22:04:43.037817 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:43.037741    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:04:43.698277 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:43.698238    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:04:43.902406 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:43.902368    5510 patch_prober.go:29] interesting pod/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-ccsc2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" start-of-body=
Oct 30 22:04:43.902843 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:43.902428    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-ccsc2" podUID="caef4e8d-7dcc-455a-94a2-4bb341662a88" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused"
Oct 30 22:04:43.918890 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:43.918862    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:04:44.901943 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:44.901904    5510 patch_prober.go:29] interesting pod/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-ccsc2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" start-of-body=
Oct 30 22:04:44.902130 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:44.901960    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-ccsc2" podUID="caef4e8d-7dcc-455a-94a2-4bb341662a88" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused"
Oct 30 22:04:45.905926 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:45.905890    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-ccsc2"
Oct 30 22:04:45.906386 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:45.906332    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-ccsc2"
Oct 30 22:04:45.907135 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:45.907114    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-ccsc2"
Oct 30 22:04:46.941871 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:46.941830    5510 patch_prober.go:29] interesting pod/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-clqsv container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 22:04:46.942304 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:46.941907    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-clqsv" podUID="21557bdf-26e4-4a0c-ad9f-50dcc391616b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:04:47.696678 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:47.692326    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:04:48.243466 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:48.243413    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:04:49.346156 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:49.346084    5510 generic.go:356] "Generic (PLEG): container finished" podID="21557bdf-26e4-4a0c-ad9f-50dcc391616b" containerID="f3a4ecf277b974e0f83bff4dbe835374e4b4ebe28697c46ac83f94aba5fa0924" exitCode=0
Oct 30 22:04:49.346575 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:49.346160    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-clqsv" event={"ID":"21557bdf-26e4-4a0c-ad9f-50dcc391616b","Type":"ContainerDied","Data":"f3a4ecf277b974e0f83bff4dbe835374e4b4ebe28697c46ac83f94aba5fa0924"}
Oct 30 22:04:49.346575 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:49.346557    5510 scope.go:122] "RemoveContainer" containerID="f3a4ecf277b974e0f83bff4dbe835374e4b4ebe28697c46ac83f94aba5fa0924"
Oct 30 22:04:49.347942 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:49.347913    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:04:49.348008 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:49.347978    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:04:49.348047 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:49.348009    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:04:50.374859 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:50.374826    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-clqsv" event={"ID":"21557bdf-26e4-4a0c-ad9f-50dcc391616b","Type":"ContainerStarted","Data":"283c8194c84b50b132cb851bab4cd1a702243d948d2414f149b1ae16ce4ffbed"}
Oct 30 22:04:50.928900 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:50.928863    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-clqsv"
Oct 30 22:04:50.929122 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:50.929094    5510 patch_prober.go:29] interesting pod/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-clqsv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body=
Oct 30 22:04:50.929209 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:50.929159    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-clqsv" podUID="21557bdf-26e4-4a0c-ad9f-50dcc391616b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused"
Oct 30 22:04:51.929073 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:51.929037    5510 patch_prober.go:29] interesting pod/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-clqsv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body=
Oct 30 22:04:51.929455 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:51.929092    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-clqsv" podUID="21557bdf-26e4-4a0c-ad9f-50dcc391616b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused"
Oct 30 22:04:52.929252 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:52.929217    5510 patch_prober.go:29] interesting pod/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-clqsv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body=
Oct 30 22:04:52.929748 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:52.929272    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-clqsv" podUID="21557bdf-26e4-4a0c-ad9f-50dcc391616b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused"
Oct 30 22:04:53.177898 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:53.177859    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:04:53.467687 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:53.461244    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:04:53.928780 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:53.928748    5510 patch_prober.go:29] interesting pod/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-clqsv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body=
Oct 30 22:04:53.928956 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:53.928815    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-clqsv" podUID="21557bdf-26e4-4a0c-ad9f-50dcc391616b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused"
Oct 30 22:04:54.169402 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:54.169363    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:04:54.388042 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:54.388007    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:04:54.928440 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:54.928401    5510 patch_prober.go:29] interesting pod/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-clqsv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body=
Oct 30 22:04:54.928639 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:54.928459    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-clqsv" podUID="21557bdf-26e4-4a0c-ad9f-50dcc391616b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused"
Oct 30 22:04:55.240248 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:55.240162    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:04:55.932638 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:55.932607    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-clqsv"
Oct 30 22:04:55.933072 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:55.933054    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-clqsv"
Oct 30 22:04:55.933731 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:55.933712    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-clqsv"
Oct 30 22:04:56.138297 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:56.138262    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:04:56.375871 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:56.375841    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:04:59.027703 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:04:59.022205    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:05:04.152539 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:05:04.152484    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:05:04.574589 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:05:04.574537    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:05:05.217362 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:05:05.217307    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:05:05.431255 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:05:05.431171    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:05:08.476542 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:05:08.476449    5510 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6gjbf" containerName="extractor"
Oct 30 22:05:09.268454 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:05:09.268414    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:05:09.488955 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:05:09.488912    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:05:14.476599 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:05:14.476559    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:05:14.817527 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:05:14.817491    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:05:15.492952 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:05:15.492911    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:05:15.768164 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:05:15.768123    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:05:20.045171 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:05:20.045132    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:05:20.254530 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:05:20.254468    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:05:20.882392 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:05:20.882346    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:05:21.122004 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:05:21.121958    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:05:22.724015 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:05:22.723978    5510 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/system.slice/system-systemd\\\\x2dcoredump.slice/systemd-coredump@2-14516-0.service\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ff054ea_b27c_4c64_be2d_c645c0ca3d24.slice/crio-a24ba72543ab31aeb5b19f93d3fba849190ec169d677ec88fd48df2f4f237693.scope\": RecentStats: unable to find data in memory cache]"
Oct 30 22:05:22.745715 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:05:22.745680    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:05:23.649085 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:05:23.649052    5510 generic.go:356] "Generic (PLEG): container finished" podID="5ff054ea-b27c-4c64-be2d-c645c0ca3d24" containerID="a24ba72543ab31aeb5b19f93d3fba849190ec169d677ec88fd48df2f4f237693" exitCode=0
Oct 30 22:05:23.649264 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:05:23.649128    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6fvq6" event={"ID":"5ff054ea-b27c-4c64-be2d-c645c0ca3d24","Type":"ContainerDied","Data":"a24ba72543ab31aeb5b19f93d3fba849190ec169d677ec88fd48df2f4f237693"}
Oct 30 22:05:23.649510 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:05:23.649497    5510 scope.go:122] "RemoveContainer" containerID="a24ba72543ab31aeb5b19f93d3fba849190ec169d677ec88fd48df2f4f237693"
Oct 30 22:05:23.650976 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:05:23.650953    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:05:23.651069 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:05:23.651014    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:05:23.651069 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:05:23.651043    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:05:24.279187 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:05:24.279014    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:05:24.686749 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:05:24.683243    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6fvq6" event={"ID":"5ff054ea-b27c-4c64-be2d-c645c0ca3d24","Type":"ContainerStarted","Data":"32d2dbdde860ac1cc4f5ac866be960f485032f283e065535f10556e82ee3d659"}
Oct 30 22:05:25.013141 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:05:25.013040    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:05:25.270336 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:05:25.270298    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:05:25.352683 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:05:25.350121    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6fvq6"
Oct 30 22:05:25.352683 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:05:25.350281    5510 patch_prober.go:29] interesting pod/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6fvq6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body=
Oct 30 22:05:25.352683 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:05:25.350343    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6fvq6" podUID="5ff054ea-b27c-4c64-be2d-c645c0ca3d24" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused"
Oct 30 22:05:25.473586 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:05:25.473551    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:05:26.352683 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:05:26.348014    5510 patch_prober.go:29] interesting pod/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6fvq6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body=
Oct 30 22:05:26.352683 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:05:26.348089    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6fvq6" podUID="5ff054ea-b27c-4c64-be2d-c645c0ca3d24" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused"
Oct 30 22:05:27.350096 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:05:27.350055    5510 patch_prober.go:29] interesting pod/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6fvq6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body=
Oct 30 22:05:27.350393 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:05:27.350363    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6fvq6" podUID="5ff054ea-b27c-4c64-be2d-c645c0ca3d24" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused"
Oct 30 22:05:27.592167 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:05:27.592065    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:05:28.357688 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:05:28.357069    5510 patch_prober.go:29] interesting pod/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6fvq6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body=
Oct 30 22:05:28.357688 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:05:28.357143    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6fvq6" podUID="5ff054ea-b27c-4c64-be2d-c645c0ca3d24" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused"
Oct 30 22:05:29.350105 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:05:29.350068    5510 patch_prober.go:29] interesting pod/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6fvq6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body=
Oct 30 22:05:29.356784 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:05:29.356749    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6fvq6" podUID="5ff054ea-b27c-4c64-be2d-c645c0ca3d24" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused"
Oct 30 22:05:30.351706 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:05:30.351609    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6fvq6"
Oct 30 22:05:30.352223 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:05:30.351927    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6fvq6"
Oct 30 22:05:30.352591 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:05:30.352574    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6fvq6"
Oct 30 22:05:30.420758 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:05:30.420721    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:05:35.333328 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:05:35.333288    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:05:38.418762 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:05:38.418721    5510 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6gjbf" containerName="extractor"
Oct 30 22:05:40.229896 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:05:40.229860    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:05:41.644349 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:05:41.644309    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:05:45.224564 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:05:45.224525    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:05:48.572164 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:05:48.572119    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:05:48.788648 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:05:48.788611    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:05:50.106536 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:05:50.106500    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:05:54.398300 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:05:54.398259    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:05:55.084462 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:05:55.084422    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:05:59.934101 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:05:59.934065    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:06:01.160252 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:06:01.160213    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:06:04.929539 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:06:04.929499    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:06:07.076292 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:06:07.076248    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:06:08.419935 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:06:08.419897    5510 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6gjbf" containerName="extractor"
Oct 30 22:06:09.908257 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:06:09.908220    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:06:11.340723 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:06:11.340680    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:06:14.815688 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:06:14.814889    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:06:17.199331 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:06:17.199283    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:06:19.833319 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:06:19.833274    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:06:23.068059 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:06:23.068027    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:06:24.739537 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:06:24.739502    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:06:29.924538 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:06:29.924315    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:06:34.822617 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:06:34.821284    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:06:36.602800 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:06:36.602767    5510 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w6-fb0b93/premium-w6-fb0b93-kserve-5cb9948c54-w7pnk"]
Oct 30 22:06:36.605806 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:06:36.605779    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-fb0b93/premium-w6-fb0b93-kserve-5cb9948c54-w7pnk"
Oct 30 22:06:36.656167 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:06:36.656132    5510 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w6-fb0b93\"/\"openshift-service-ca.crt\""
Oct 30 22:06:36.669693 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:06:36.669644    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/290ae7a2-de37-4af3-82ae-a8e855b83bb9-tmp-dir\") pod \"premium-w6-fb0b93-kserve-5cb9948c54-w7pnk\" (UID: \"290ae7a2-de37-4af3-82ae-a8e855b83bb9\") " pod="e2e-models-e2e-worker-w6-fb0b93/premium-w6-fb0b93-kserve-5cb9948c54-w7pnk"
Oct 30 22:06:36.669859 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:06:36.669733    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jlzzj\" (UniqueName: \"kubernetes.io/projected/290ae7a2-de37-4af3-82ae-a8e855b83bb9-kube-api-access-jlzzj\") pod \"premium-w6-fb0b93-kserve-5cb9948c54-w7pnk\" (UID: \"290ae7a2-de37-4af3-82ae-a8e855b83bb9\") " pod="e2e-models-e2e-worker-w6-fb0b93/premium-w6-fb0b93-kserve-5cb9948c54-w7pnk"
Oct 30 22:06:36.669859 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:06:36.669776    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/290ae7a2-de37-4af3-82ae-a8e855b83bb9-model-cache\") pod \"premium-w6-fb0b93-kserve-5cb9948c54-w7pnk\" (UID: \"290ae7a2-de37-4af3-82ae-a8e855b83bb9\") " pod="e2e-models-e2e-worker-w6-fb0b93/premium-w6-fb0b93-kserve-5cb9948c54-w7pnk"
Oct 30 22:06:36.669980 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:06:36.669895    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/290ae7a2-de37-4af3-82ae-a8e855b83bb9-tls-certs\") pod \"premium-w6-fb0b93-kserve-5cb9948c54-w7pnk\" (UID: \"290ae7a2-de37-4af3-82ae-a8e855b83bb9\") " pod="e2e-models-e2e-worker-w6-fb0b93/premium-w6-fb0b93-kserve-5cb9948c54-w7pnk"
Oct 30 22:06:36.670027 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:06:36.669982    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/290ae7a2-de37-4af3-82ae-a8e855b83bb9-home\") pod \"premium-w6-fb0b93-kserve-5cb9948c54-w7pnk\" (UID: \"290ae7a2-de37-4af3-82ae-a8e855b83bb9\") " pod="e2e-models-e2e-worker-w6-fb0b93/premium-w6-fb0b93-kserve-5cb9948c54-w7pnk"
Oct 30 22:06:36.670160 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:06:36.670132    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/290ae7a2-de37-4af3-82ae-a8e855b83bb9-dshm\") pod \"premium-w6-fb0b93-kserve-5cb9948c54-w7pnk\" (UID: \"290ae7a2-de37-4af3-82ae-a8e855b83bb9\") " pod="e2e-models-e2e-worker-w6-fb0b93/premium-w6-fb0b93-kserve-5cb9948c54-w7pnk"
Oct 30 22:06:36.702897 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:06:36.702866    5510 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w6-fb0b93\"/\"kube-root-ca.crt\""
Oct 30 22:06:36.703059 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:06:36.702926    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w6-fb0b93\"/\"premium-w6-fb0b93-kserve-self-signed-certs\""
Oct 30 22:06:36.703059 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:06:36.702976    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w6-fb0b93\"/\"default-dockercfg-vssbl\""
Oct 30 22:06:36.705149 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:06:36.705129    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w6-fb0b93/premium-w6-fb0b93-kserve-5cb9948c54-w7pnk"]
Oct 30 22:06:36.766028 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:06:36.762457    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:06:36.774928 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:06:36.774879    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jlzzj\" (UniqueName: \"kubernetes.io/projected/290ae7a2-de37-4af3-82ae-a8e855b83bb9-kube-api-access-jlzzj\") pod \"premium-w6-fb0b93-kserve-5cb9948c54-w7pnk\" (UID: \"290ae7a2-de37-4af3-82ae-a8e855b83bb9\") " pod="e2e-models-e2e-worker-w6-fb0b93/premium-w6-fb0b93-kserve-5cb9948c54-w7pnk"
Oct 30 22:06:36.775132 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:06:36.774947    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/290ae7a2-de37-4af3-82ae-a8e855b83bb9-model-cache\") pod \"premium-w6-fb0b93-kserve-5cb9948c54-w7pnk\" (UID: \"290ae7a2-de37-4af3-82ae-a8e855b83bb9\") " pod="e2e-models-e2e-worker-w6-fb0b93/premium-w6-fb0b93-kserve-5cb9948c54-w7pnk"
Oct 30 22:06:36.775132 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:06:36.775087    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/290ae7a2-de37-4af3-82ae-a8e855b83bb9-tls-certs\") pod \"premium-w6-fb0b93-kserve-5cb9948c54-w7pnk\" (UID: \"290ae7a2-de37-4af3-82ae-a8e855b83bb9\") " pod="e2e-models-e2e-worker-w6-fb0b93/premium-w6-fb0b93-kserve-5cb9948c54-w7pnk"
Oct 30 22:06:36.775257 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:06:36.775168    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/290ae7a2-de37-4af3-82ae-a8e855b83bb9-home\") pod \"premium-w6-fb0b93-kserve-5cb9948c54-w7pnk\" (UID: \"290ae7a2-de37-4af3-82ae-a8e855b83bb9\") " pod="e2e-models-e2e-worker-w6-fb0b93/premium-w6-fb0b93-kserve-5cb9948c54-w7pnk"
Oct 30 22:06:36.775257 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:06:36.775206    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/290ae7a2-de37-4af3-82ae-a8e855b83bb9-dshm\") pod \"premium-w6-fb0b93-kserve-5cb9948c54-w7pnk\" (UID: \"290ae7a2-de37-4af3-82ae-a8e855b83bb9\") " pod="e2e-models-e2e-worker-w6-fb0b93/premium-w6-fb0b93-kserve-5cb9948c54-w7pnk"
Oct 30 22:06:36.775257 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:06:36.775242    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/290ae7a2-de37-4af3-82ae-a8e855b83bb9-tmp-dir\") pod \"premium-w6-fb0b93-kserve-5cb9948c54-w7pnk\" (UID: \"290ae7a2-de37-4af3-82ae-a8e855b83bb9\") " pod="e2e-models-e2e-worker-w6-fb0b93/premium-w6-fb0b93-kserve-5cb9948c54-w7pnk"
Oct 30 22:06:36.775735 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:06:36.775708    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/290ae7a2-de37-4af3-82ae-a8e855b83bb9-tmp-dir\") pod \"premium-w6-fb0b93-kserve-5cb9948c54-w7pnk\" (UID: \"290ae7a2-de37-4af3-82ae-a8e855b83bb9\") " pod="e2e-models-e2e-worker-w6-fb0b93/premium-w6-fb0b93-kserve-5cb9948c54-w7pnk"
Oct 30 22:06:36.776035 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:06:36.776014    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/290ae7a2-de37-4af3-82ae-a8e855b83bb9-home\") pod \"premium-w6-fb0b93-kserve-5cb9948c54-w7pnk\" (UID: \"290ae7a2-de37-4af3-82ae-a8e855b83bb9\") " pod="e2e-models-e2e-worker-w6-fb0b93/premium-w6-fb0b93-kserve-5cb9948c54-w7pnk"
Oct 30 22:06:36.776399 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:06:36.776382    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/290ae7a2-de37-4af3-82ae-a8e855b83bb9-model-cache\") pod \"premium-w6-fb0b93-kserve-5cb9948c54-w7pnk\" (UID: \"290ae7a2-de37-4af3-82ae-a8e855b83bb9\") " pod="e2e-models-e2e-worker-w6-fb0b93/premium-w6-fb0b93-kserve-5cb9948c54-w7pnk"
Oct 30 22:06:36.779463 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:06:36.778935    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/290ae7a2-de37-4af3-82ae-a8e855b83bb9-dshm\") pod \"premium-w6-fb0b93-kserve-5cb9948c54-w7pnk\" (UID: \"290ae7a2-de37-4af3-82ae-a8e855b83bb9\") " pod="e2e-models-e2e-worker-w6-fb0b93/premium-w6-fb0b93-kserve-5cb9948c54-w7pnk"
Oct 30 22:06:36.779463 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:06:36.779076    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/290ae7a2-de37-4af3-82ae-a8e855b83bb9-tls-certs\") pod \"premium-w6-fb0b93-kserve-5cb9948c54-w7pnk\" (UID: \"290ae7a2-de37-4af3-82ae-a8e855b83bb9\") " pod="e2e-models-e2e-worker-w6-fb0b93/premium-w6-fb0b93-kserve-5cb9948c54-w7pnk"
Oct 30 22:06:36.833706 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:06:36.833644    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jlzzj\" (UniqueName: \"kubernetes.io/projected/290ae7a2-de37-4af3-82ae-a8e855b83bb9-kube-api-access-jlzzj\") pod \"premium-w6-fb0b93-kserve-5cb9948c54-w7pnk\" (UID: \"290ae7a2-de37-4af3-82ae-a8e855b83bb9\") " pod="e2e-models-e2e-worker-w6-fb0b93/premium-w6-fb0b93-kserve-5cb9948c54-w7pnk"
Oct 30 22:06:36.916807 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:06:36.916725    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-fb0b93/premium-w6-fb0b93-kserve-5cb9948c54-w7pnk"
Oct 30 22:06:37.340392 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:06:37.340352    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w6-fb0b93/premium-w6-fb0b93-kserve-5cb9948c54-w7pnk"]
Oct 30 22:06:37.342994 ip-10-0-136-210 kubenswrapper[5510]: W1030 22:06:37.342965    5510 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod290ae7a2_de37_4af3_82ae_a8e855b83bb9.slice/crio-3e22d6082f796674e2c31844b9107580ee5df38cf95021ec51f88cefde8ebe72 WatchSource:0}: Error finding container 3e22d6082f796674e2c31844b9107580ee5df38cf95021ec51f88cefde8ebe72: Status 404 returned error can't find the container with id 3e22d6082f796674e2c31844b9107580ee5df38cf95021ec51f88cefde8ebe72
Oct 30 22:06:38.107700 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:06:38.107644    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-fb0b93/premium-w6-fb0b93-kserve-5cb9948c54-w7pnk" event={"ID":"290ae7a2-de37-4af3-82ae-a8e855b83bb9","Type":"ContainerStarted","Data":"9f802ceb1f913781c756a6ea49cc3aef5e94b3a6b6e72248b31cc0a33d3e9691"}
Oct 30 22:06:38.108114 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:06:38.107713    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-fb0b93/premium-w6-fb0b93-kserve-5cb9948c54-w7pnk" event={"ID":"290ae7a2-de37-4af3-82ae-a8e855b83bb9","Type":"ContainerStarted","Data":"3e22d6082f796674e2c31844b9107580ee5df38cf95021ec51f88cefde8ebe72"}
Oct 30 22:06:38.257039 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:06:38.256988    5510 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w6-fb0b93/premium-w6-fb0b93-kserve-5cb9948c54-w7pnk" podStartSLOduration=2.256975585 podStartE2EDuration="2.256975585s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:06:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:06:38.255816947 +0000 UTC m=+978.816626951" watchObservedRunningTime="2026-10-30 22:06:38.256975585 +0000 UTC m=+978.817785590"
Oct 30 22:06:38.436454 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:06:38.436359    5510 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6gjbf" containerName="extractor"
Oct 30 22:06:38.917880 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:06:38.917850    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-fb0b93/premium-w6-fb0b93-kserve-5cb9948c54-w7pnk"
Oct 30 22:06:39.734295 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:06:39.734028    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:06:44.738308 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:06:44.738269    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:06:46.917295 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:06:46.917260    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w6-fb0b93/premium-w6-fb0b93-kserve-5cb9948c54-w7pnk"
Oct 30 22:06:46.929835 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:06:46.929807    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w6-fb0b93/premium-w6-fb0b93-kserve-5cb9948c54-w7pnk"
Oct 30 22:06:47.165044 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:06:47.165015    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-fb0b93/premium-w6-fb0b93-kserve-5cb9948c54-w7pnk"
Oct 30 22:06:49.617625 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:06:49.617564    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:06:53.709207 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:06:53.709167    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:06:54.445184 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:06:54.445145    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:06:55.315946 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:06:55.315162    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:06:59.346684 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:06:59.346252    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:07:04.224992 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:07:04.224903    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:07:08.420439 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:07:08.420398    5510 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6gjbf" containerName="extractor"
Oct 30 22:07:08.907733 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:07:08.906478    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:07:09.162398 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:07:09.162309    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:07:14.132231 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:07:14.132194    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:07:14.765927 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:07:14.765891    5510 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-rd56v"]
Oct 30 22:07:14.769834 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:07:14.769813    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-rd56v"
Oct 30 22:07:14.819325 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:07:14.819289    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w2-bc714d-openshift-default-dockercfg-gs4pf\""
Oct 30 22:07:14.871541 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:07:14.871506    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7e6ab5c7-cadf-4bb4-9dda-68b17a855729-istio-podinfo\") pod \"e2e-worker-w2-bc714d-openshift-default-67558b95cf-rd56v\" (UID: \"7e6ab5c7-cadf-4bb4-9dda-68b17a855729\") " pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-rd56v"
Oct 30 22:07:14.871775 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:07:14.871573    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7e6ab5c7-cadf-4bb4-9dda-68b17a855729-istiod-ca-cert\") pod \"e2e-worker-w2-bc714d-openshift-default-67558b95cf-rd56v\" (UID: \"7e6ab5c7-cadf-4bb4-9dda-68b17a855729\") " pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-rd56v"
Oct 30 22:07:14.871775 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:07:14.871677    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7e6ab5c7-cadf-4bb4-9dda-68b17a855729-workload-certs\") pod \"e2e-worker-w2-bc714d-openshift-default-67558b95cf-rd56v\" (UID: \"7e6ab5c7-cadf-4bb4-9dda-68b17a855729\") " pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-rd56v"
Oct 30 22:07:14.871775 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:07:14.871763    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7e6ab5c7-cadf-4bb4-9dda-68b17a855729-istio-envoy\") pod \"e2e-worker-w2-bc714d-openshift-default-67558b95cf-rd56v\" (UID: \"7e6ab5c7-cadf-4bb4-9dda-68b17a855729\") " pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-rd56v"
Oct 30 22:07:14.871918 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:07:14.871816    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e6ab5c7-cadf-4bb4-9dda-68b17a855729-workload-socket\") pod \"e2e-worker-w2-bc714d-openshift-default-67558b95cf-rd56v\" (UID: \"7e6ab5c7-cadf-4bb4-9dda-68b17a855729\") " pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-rd56v"
Oct 30 22:07:14.871918 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:07:14.871868    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e6ab5c7-cadf-4bb4-9dda-68b17a855729-credential-socket\") pod \"e2e-worker-w2-bc714d-openshift-default-67558b95cf-rd56v\" (UID: \"7e6ab5c7-cadf-4bb4-9dda-68b17a855729\") " pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-rd56v"
Oct 30 22:07:14.871918 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:07:14.871895    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7e6ab5c7-cadf-4bb4-9dda-68b17a855729-istio-data\") pod \"e2e-worker-w2-bc714d-openshift-default-67558b95cf-rd56v\" (UID: \"7e6ab5c7-cadf-4bb4-9dda-68b17a855729\") " pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-rd56v"
Oct 30 22:07:14.872067 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:07:14.871924    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dxqkw\" (UniqueName: \"kubernetes.io/projected/7e6ab5c7-cadf-4bb4-9dda-68b17a855729-kube-api-access-dxqkw\") pod \"e2e-worker-w2-bc714d-openshift-default-67558b95cf-rd56v\" (UID: \"7e6ab5c7-cadf-4bb4-9dda-68b17a855729\") " pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-rd56v"
Oct 30 22:07:14.872121 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:07:14.872082    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7e6ab5c7-cadf-4bb4-9dda-68b17a855729-istio-token\") pod \"e2e-worker-w2-bc714d-openshift-default-67558b95cf-rd56v\" (UID: \"7e6ab5c7-cadf-4bb4-9dda-68b17a855729\") " pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-rd56v"
Oct 30 22:07:14.876700 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:07:14.876673    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-rd56v"]
Oct 30 22:07:14.972957 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:07:14.972919    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7e6ab5c7-cadf-4bb4-9dda-68b17a855729-istio-envoy\") pod \"e2e-worker-w2-bc714d-openshift-default-67558b95cf-rd56v\" (UID: \"7e6ab5c7-cadf-4bb4-9dda-68b17a855729\") " pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-rd56v"
Oct 30 22:07:14.973146 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:07:14.972973    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e6ab5c7-cadf-4bb4-9dda-68b17a855729-workload-socket\") pod \"e2e-worker-w2-bc714d-openshift-default-67558b95cf-rd56v\" (UID: \"7e6ab5c7-cadf-4bb4-9dda-68b17a855729\") " pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-rd56v"
Oct 30 22:07:14.973146 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:07:14.972998    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e6ab5c7-cadf-4bb4-9dda-68b17a855729-credential-socket\") pod \"e2e-worker-w2-bc714d-openshift-default-67558b95cf-rd56v\" (UID: \"7e6ab5c7-cadf-4bb4-9dda-68b17a855729\") " pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-rd56v"
Oct 30 22:07:14.973146 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:07:14.973024    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7e6ab5c7-cadf-4bb4-9dda-68b17a855729-istio-data\") pod \"e2e-worker-w2-bc714d-openshift-default-67558b95cf-rd56v\" (UID: \"7e6ab5c7-cadf-4bb4-9dda-68b17a855729\") " pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-rd56v"
Oct 30 22:07:14.973146 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:07:14.973040    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dxqkw\" (UniqueName: \"kubernetes.io/projected/7e6ab5c7-cadf-4bb4-9dda-68b17a855729-kube-api-access-dxqkw\") pod \"e2e-worker-w2-bc714d-openshift-default-67558b95cf-rd56v\" (UID: \"7e6ab5c7-cadf-4bb4-9dda-68b17a855729\") " pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-rd56v"
Oct 30 22:07:14.973146 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:07:14.973144    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7e6ab5c7-cadf-4bb4-9dda-68b17a855729-istio-token\") pod \"e2e-worker-w2-bc714d-openshift-default-67558b95cf-rd56v\" (UID: \"7e6ab5c7-cadf-4bb4-9dda-68b17a855729\") " pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-rd56v"
Oct 30 22:07:14.973402 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:07:14.973321    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7e6ab5c7-cadf-4bb4-9dda-68b17a855729-istio-podinfo\") pod \"e2e-worker-w2-bc714d-openshift-default-67558b95cf-rd56v\" (UID: \"7e6ab5c7-cadf-4bb4-9dda-68b17a855729\") " pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-rd56v"
Oct 30 22:07:14.973402 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:07:14.973397    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7e6ab5c7-cadf-4bb4-9dda-68b17a855729-istiod-ca-cert\") pod \"e2e-worker-w2-bc714d-openshift-default-67558b95cf-rd56v\" (UID: \"7e6ab5c7-cadf-4bb4-9dda-68b17a855729\") " pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-rd56v"
Oct 30 22:07:14.973522 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:07:14.973441    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e6ab5c7-cadf-4bb4-9dda-68b17a855729-credential-socket\") pod \"e2e-worker-w2-bc714d-openshift-default-67558b95cf-rd56v\" (UID: \"7e6ab5c7-cadf-4bb4-9dda-68b17a855729\") " pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-rd56v"
Oct 30 22:07:14.973522 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:07:14.973445    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7e6ab5c7-cadf-4bb4-9dda-68b17a855729-workload-certs\") pod \"e2e-worker-w2-bc714d-openshift-default-67558b95cf-rd56v\" (UID: \"7e6ab5c7-cadf-4bb4-9dda-68b17a855729\") " pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-rd56v"
Oct 30 22:07:14.973522 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:07:14.973499    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e6ab5c7-cadf-4bb4-9dda-68b17a855729-workload-socket\") pod \"e2e-worker-w2-bc714d-openshift-default-67558b95cf-rd56v\" (UID: \"7e6ab5c7-cadf-4bb4-9dda-68b17a855729\") " pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-rd56v"
Oct 30 22:07:14.973743 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:07:14.973718    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7e6ab5c7-cadf-4bb4-9dda-68b17a855729-workload-certs\") pod \"e2e-worker-w2-bc714d-openshift-default-67558b95cf-rd56v\" (UID: \"7e6ab5c7-cadf-4bb4-9dda-68b17a855729\") " pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-rd56v"
Oct 30 22:07:14.973924 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:07:14.973898    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7e6ab5c7-cadf-4bb4-9dda-68b17a855729-istio-data\") pod \"e2e-worker-w2-bc714d-openshift-default-67558b95cf-rd56v\" (UID: \"7e6ab5c7-cadf-4bb4-9dda-68b17a855729\") " pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-rd56v"
Oct 30 22:07:14.974125 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:07:14.974102    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7e6ab5c7-cadf-4bb4-9dda-68b17a855729-istiod-ca-cert\") pod \"e2e-worker-w2-bc714d-openshift-default-67558b95cf-rd56v\" (UID: \"7e6ab5c7-cadf-4bb4-9dda-68b17a855729\") " pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-rd56v"
Oct 30 22:07:14.975333 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:07:14.975315    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7e6ab5c7-cadf-4bb4-9dda-68b17a855729-istio-envoy\") pod \"e2e-worker-w2-bc714d-openshift-default-67558b95cf-rd56v\" (UID: \"7e6ab5c7-cadf-4bb4-9dda-68b17a855729\") " pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-rd56v"
Oct 30 22:07:14.975764 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:07:14.975743    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7e6ab5c7-cadf-4bb4-9dda-68b17a855729-istio-podinfo\") pod \"e2e-worker-w2-bc714d-openshift-default-67558b95cf-rd56v\" (UID: \"7e6ab5c7-cadf-4bb4-9dda-68b17a855729\") " pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-rd56v"
Oct 30 22:07:15.034323 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:07:15.034289    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7e6ab5c7-cadf-4bb4-9dda-68b17a855729-istio-token\") pod \"e2e-worker-w2-bc714d-openshift-default-67558b95cf-rd56v\" (UID: \"7e6ab5c7-cadf-4bb4-9dda-68b17a855729\") " pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-rd56v"
Oct 30 22:07:15.035498 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:07:15.035475    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dxqkw\" (UniqueName: \"kubernetes.io/projected/7e6ab5c7-cadf-4bb4-9dda-68b17a855729-kube-api-access-dxqkw\") pod \"e2e-worker-w2-bc714d-openshift-default-67558b95cf-rd56v\" (UID: \"7e6ab5c7-cadf-4bb4-9dda-68b17a855729\") " pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-rd56v"
Oct 30 22:07:15.084148 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:07:15.084115    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-rd56v"
Oct 30 22:07:15.514235 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:07:15.514207    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-rd56v"]
Oct 30 22:07:15.516947 ip-10-0-136-210 kubenswrapper[5510]: W1030 22:07:15.516920    5510 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e6ab5c7_cadf_4bb4_9dda_68b17a855729.slice/crio-bab49d066a60965f765409d587bce7c2868d6a057479a235a7c04a5b284ec2a6 WatchSource:0}: Error finding container bab49d066a60965f765409d587bce7c2868d6a057479a235a7c04a5b284ec2a6: Status 404 returned error can't find the container with id bab49d066a60965f765409d587bce7c2868d6a057479a235a7c04a5b284ec2a6
Oct 30 22:07:15.519256 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:07:15.519224    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:07:15.519326 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:07:15.519296    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:07:15.519362 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:07:15.519324    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:07:16.318949 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:07:16.318904    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-rd56v" event={"ID":"7e6ab5c7-cadf-4bb4-9dda-68b17a855729","Type":"ContainerStarted","Data":"16d6941c50b495e8850bd41acef908fbe8e0aea410be304a0a0dbddf41f1fa55"}
Oct 30 22:07:16.319122 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:07:16.318968    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-rd56v" event={"ID":"7e6ab5c7-cadf-4bb4-9dda-68b17a855729","Type":"ContainerStarted","Data":"bab49d066a60965f765409d587bce7c2868d6a057479a235a7c04a5b284ec2a6"}
Oct 30 22:07:16.442295 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:07:16.442247    5510 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-rd56v" podStartSLOduration=2.442232862 podStartE2EDuration="2.442232862s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:07:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:07:16.440645153 +0000 UTC m=+1017.001455178" watchObservedRunningTime="2026-10-30 22:07:16.442232862 +0000 UTC m=+1017.003042866"
Oct 30 22:07:17.084827 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:07:17.084788    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-rd56v"
Oct 30 22:07:17.086763 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:07:17.086736    5510 patch_prober.go:29] interesting pod/e2e-worker-w2-bc714d-openshift-default-67558b95cf-rd56v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body=
Oct 30 22:07:17.086898 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:07:17.086784    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-rd56v" podUID="7e6ab5c7-cadf-4bb4-9dda-68b17a855729" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused"
Oct 30 22:07:18.085280 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:07:18.085243    5510 patch_prober.go:29] interesting pod/e2e-worker-w2-bc714d-openshift-default-67558b95cf-rd56v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body=
Oct 30 22:07:18.085721 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:07:18.085309    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-rd56v" podUID="7e6ab5c7-cadf-4bb4-9dda-68b17a855729" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused"
Oct 30 22:07:19.064694 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:07:19.063623    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:07:19.088406 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:07:19.088370    5510 patch_prober.go:29] interesting pod/e2e-worker-w2-bc714d-openshift-default-67558b95cf-rd56v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body=
Oct 30 22:07:19.089088 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:07:19.089056    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-rd56v" podUID="7e6ab5c7-cadf-4bb4-9dda-68b17a855729" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused"
Oct 30 22:07:20.094273 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:07:20.091832    5510 patch_prober.go:29] interesting pod/e2e-worker-w2-bc714d-openshift-default-67558b95cf-rd56v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body=
Oct 30 22:07:20.094273 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:07:20.091934    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-rd56v" podUID="7e6ab5c7-cadf-4bb4-9dda-68b17a855729" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused"
Oct 30 22:07:20.121086 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:07:20.121051    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:07:21.085236 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:07:21.085196    5510 patch_prober.go:29] interesting pod/e2e-worker-w2-bc714d-openshift-default-67558b95cf-rd56v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body=
Oct 30 22:07:21.085429 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:07:21.085251    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-rd56v" podUID="7e6ab5c7-cadf-4bb4-9dda-68b17a855729" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused"
Oct 30 22:07:22.085597 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:07:22.085555    5510 patch_prober.go:29] interesting pod/e2e-worker-w2-bc714d-openshift-default-67558b95cf-rd56v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body=
Oct 30 22:07:22.086039 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:07:22.085625    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-rd56v" podUID="7e6ab5c7-cadf-4bb4-9dda-68b17a855729" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused"
Oct 30 22:07:23.089067 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:07:23.089034    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-rd56v"
Oct 30 22:07:23.089454 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:07:23.089308    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-rd56v"
Oct 30 22:07:23.089915 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:07:23.089894    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-rd56v"
Oct 30 22:07:23.413399 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:07:23.413309    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:07:24.250989 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:07:24.250953    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:07:29.144993 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:07:29.144951    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:07:34.144682 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:07:34.141280    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:07:36.925991 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:07:36.925950    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:07:38.498503 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:07:38.498397    5510 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6gjbf" containerName="extractor"
Oct 30 22:07:39.062552 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:07:39.062517    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:07:44.041156 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:07:44.041118    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:07:48.957115 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:07:48.957072    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:07:53.854155 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:07:53.854075    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:07:58.754854 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:07:58.754812    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:08:03.750046 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:03.749752    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:08:08.420400 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:08:08.420364    5510 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6gjbf" containerName="extractor"
Oct 30 22:08:08.636087 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:08.636050    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:08:13.652684 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:13.651376    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:08:18.539685 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:18.538318    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:08:23.530456 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:23.530392    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:08:23.761916 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:23.761875    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:08:24.061197 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:24.061151    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:08:24.338493 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:24.338403    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-gbtm2"]
Oct 30 22:08:24.338704 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:24.338617    5510 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-gbtm2" podUID="bbd4c2ea-a56a-4f1f-9e37-a77b71326592" containerName="istio-proxy" containerID="cri-o://64c20ea99cbed10d600e8716909cd88c2a1b548237e198e41ccdb42a5893a177" gracePeriod=30
Oct 30 22:08:28.537386 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:28.537345    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:08:29.608471 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:29.608404    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-gbtm2"
Oct 30 22:08:29.707324 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:29.707290    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bbd4c2ea-a56a-4f1f-9e37-a77b71326592-kube-api-access-56njb\" (UniqueName: \"kubernetes.io/projected/bbd4c2ea-a56a-4f1f-9e37-a77b71326592-kube-api-access-56njb\") pod \"bbd4c2ea-a56a-4f1f-9e37-a77b71326592\" (UID: \"bbd4c2ea-a56a-4f1f-9e37-a77b71326592\") "
Oct 30 22:08:29.707502 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:29.707341    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bbd4c2ea-a56a-4f1f-9e37-a77b71326592-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bbd4c2ea-a56a-4f1f-9e37-a77b71326592-istio-envoy\") pod \"bbd4c2ea-a56a-4f1f-9e37-a77b71326592\" (UID: \"bbd4c2ea-a56a-4f1f-9e37-a77b71326592\") "
Oct 30 22:08:29.707502 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:29.707371    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bbd4c2ea-a56a-4f1f-9e37-a77b71326592-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bbd4c2ea-a56a-4f1f-9e37-a77b71326592-workload-socket\") pod \"bbd4c2ea-a56a-4f1f-9e37-a77b71326592\" (UID: \"bbd4c2ea-a56a-4f1f-9e37-a77b71326592\") "
Oct 30 22:08:29.707502 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:29.707415    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/bbd4c2ea-a56a-4f1f-9e37-a77b71326592-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bbd4c2ea-a56a-4f1f-9e37-a77b71326592-istio-podinfo\") pod \"bbd4c2ea-a56a-4f1f-9e37-a77b71326592\" (UID: \"bbd4c2ea-a56a-4f1f-9e37-a77b71326592\") "
Oct 30 22:08:29.707502 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:29.707431    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bbd4c2ea-a56a-4f1f-9e37-a77b71326592-istio-token\" (UniqueName: \"kubernetes.io/projected/bbd4c2ea-a56a-4f1f-9e37-a77b71326592-istio-token\") pod \"bbd4c2ea-a56a-4f1f-9e37-a77b71326592\" (UID: \"bbd4c2ea-a56a-4f1f-9e37-a77b71326592\") "
Oct 30 22:08:29.707721 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:29.707502    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bbd4c2ea-a56a-4f1f-9e37-a77b71326592-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bbd4c2ea-a56a-4f1f-9e37-a77b71326592-workload-certs\") pod \"bbd4c2ea-a56a-4f1f-9e37-a77b71326592\" (UID: \"bbd4c2ea-a56a-4f1f-9e37-a77b71326592\") "
Oct 30 22:08:29.707721 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:29.707536    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bbd4c2ea-a56a-4f1f-9e37-a77b71326592-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bbd4c2ea-a56a-4f1f-9e37-a77b71326592-istio-data\") pod \"bbd4c2ea-a56a-4f1f-9e37-a77b71326592\" (UID: \"bbd4c2ea-a56a-4f1f-9e37-a77b71326592\") "
Oct 30 22:08:29.707721 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:29.707651    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/bbd4c2ea-a56a-4f1f-9e37-a77b71326592-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bbd4c2ea-a56a-4f1f-9e37-a77b71326592-istiod-ca-cert\") pod \"bbd4c2ea-a56a-4f1f-9e37-a77b71326592\" (UID: \"bbd4c2ea-a56a-4f1f-9e37-a77b71326592\") "
Oct 30 22:08:29.707889 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:29.707784    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bbd4c2ea-a56a-4f1f-9e37-a77b71326592-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bbd4c2ea-a56a-4f1f-9e37-a77b71326592-credential-socket\") pod \"bbd4c2ea-a56a-4f1f-9e37-a77b71326592\" (UID: \"bbd4c2ea-a56a-4f1f-9e37-a77b71326592\") "
Oct 30 22:08:29.707889 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:29.707805    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bbd4c2ea-a56a-4f1f-9e37-a77b71326592-workload-socket" pod "bbd4c2ea-a56a-4f1f-9e37-a77b71326592" (UID: "bbd4c2ea-a56a-4f1f-9e37-a77b71326592"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:08:29.707889 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:29.707825    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bbd4c2ea-a56a-4f1f-9e37-a77b71326592-workload-certs" pod "bbd4c2ea-a56a-4f1f-9e37-a77b71326592" (UID: "bbd4c2ea-a56a-4f1f-9e37-a77b71326592"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:08:29.708050 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:29.707982    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bbd4c2ea-a56a-4f1f-9e37-a77b71326592-istio-data" pod "bbd4c2ea-a56a-4f1f-9e37-a77b71326592" (UID: "bbd4c2ea-a56a-4f1f-9e37-a77b71326592"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:08:29.708165 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:29.708109    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bbd4c2ea-a56a-4f1f-9e37-a77b71326592-credential-socket" pod "bbd4c2ea-a56a-4f1f-9e37-a77b71326592" (UID: "bbd4c2ea-a56a-4f1f-9e37-a77b71326592"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:08:29.708301 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:29.708220    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bbd4c2ea-a56a-4f1f-9e37-a77b71326592-istiod-ca-cert" pod "bbd4c2ea-a56a-4f1f-9e37-a77b71326592" (UID: "bbd4c2ea-a56a-4f1f-9e37-a77b71326592"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:08:29.708735 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:29.708715    5510 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bbd4c2ea-a56a-4f1f-9e37-a77b71326592-workload-socket\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:08:29.708847 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:29.708831    5510 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bbd4c2ea-a56a-4f1f-9e37-a77b71326592-workload-certs\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:08:29.708929 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:29.708850    5510 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bbd4c2ea-a56a-4f1f-9e37-a77b71326592-istio-data\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:08:29.708929 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:29.708864    5510 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bbd4c2ea-a56a-4f1f-9e37-a77b71326592-istiod-ca-cert\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:08:29.708929 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:29.708878    5510 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bbd4c2ea-a56a-4f1f-9e37-a77b71326592-credential-socket\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:08:29.710091 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:29.710063    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bbd4c2ea-a56a-4f1f-9e37-a77b71326592-istio-token" pod "bbd4c2ea-a56a-4f1f-9e37-a77b71326592" (UID: "bbd4c2ea-a56a-4f1f-9e37-a77b71326592"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:08:29.710190 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:29.710149    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bbd4c2ea-a56a-4f1f-9e37-a77b71326592-istio-envoy" pod "bbd4c2ea-a56a-4f1f-9e37-a77b71326592" (UID: "bbd4c2ea-a56a-4f1f-9e37-a77b71326592"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:08:29.710190 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:29.710171    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bbd4c2ea-a56a-4f1f-9e37-a77b71326592-kube-api-access-56njb" pod "bbd4c2ea-a56a-4f1f-9e37-a77b71326592" (UID: "bbd4c2ea-a56a-4f1f-9e37-a77b71326592"). InnerVolumeSpecName "kube-api-access-56njb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:08:29.710676 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:29.710629    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/bbd4c2ea-a56a-4f1f-9e37-a77b71326592-istio-podinfo" pod "bbd4c2ea-a56a-4f1f-9e37-a77b71326592" (UID: "bbd4c2ea-a56a-4f1f-9e37-a77b71326592"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:08:29.712088 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:29.712065    5510 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-vtwfv"]
Oct 30 22:08:29.712648 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:29.712630    5510 cpu_manager.go:414] "containerMap: removing container" podUID="bbd4c2ea-a56a-4f1f-9e37-a77b71326592" containerName="istio-proxy"
Oct 30 22:08:29.712742 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:29.712649    5510 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bbd4c2ea-a56a-4f1f-9e37-a77b71326592" containerName="istio-proxy"
Oct 30 22:08:29.712803 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:29.712793    5510 memory_manager.go:365] "RemoveStaleState removing state" podUID="bbd4c2ea-a56a-4f1f-9e37-a77b71326592" containerName="istio-proxy"
Oct 30 22:08:29.716178 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:29.716160    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-vtwfv"
Oct 30 22:08:29.768590 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:29.768561    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w0-779f32-openshift-default-dockercfg-lp8gt\""
Oct 30 22:08:29.782905 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:29.782871    5510 generic.go:356] "Generic (PLEG): container finished" podID="bbd4c2ea-a56a-4f1f-9e37-a77b71326592" containerID="64c20ea99cbed10d600e8716909cd88c2a1b548237e198e41ccdb42a5893a177" exitCode=0
Oct 30 22:08:29.783056 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:29.782927    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-gbtm2"
Oct 30 22:08:29.783056 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:29.782957    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-gbtm2" event={"ID":"bbd4c2ea-a56a-4f1f-9e37-a77b71326592","Type":"ContainerDied","Data":"64c20ea99cbed10d600e8716909cd88c2a1b548237e198e41ccdb42a5893a177"}
Oct 30 22:08:29.783056 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:29.783012    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-gbtm2" event={"ID":"bbd4c2ea-a56a-4f1f-9e37-a77b71326592","Type":"ContainerDied","Data":"7196748ef622781fa141281059208545f43a3872f2305070c776455ce63debe5"}
Oct 30 22:08:29.783056 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:29.783033    5510 scope.go:122] "RemoveContainer" containerID="64c20ea99cbed10d600e8716909cd88c2a1b548237e198e41ccdb42a5893a177"
Oct 30 22:08:29.793098 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:29.793075    5510 scope.go:122] "RemoveContainer" containerID="64c20ea99cbed10d600e8716909cd88c2a1b548237e198e41ccdb42a5893a177"
Oct 30 22:08:29.793364 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:08:29.793346    5510 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"64c20ea99cbed10d600e8716909cd88c2a1b548237e198e41ccdb42a5893a177\": container with ID starting with 64c20ea99cbed10d600e8716909cd88c2a1b548237e198e41ccdb42a5893a177 not found: ID does not exist" containerID="64c20ea99cbed10d600e8716909cd88c2a1b548237e198e41ccdb42a5893a177"
Oct 30 22:08:29.793416 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:29.793376    5510 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"64c20ea99cbed10d600e8716909cd88c2a1b548237e198e41ccdb42a5893a177"} err="failed to get container status \"64c20ea99cbed10d600e8716909cd88c2a1b548237e198e41ccdb42a5893a177\": rpc error: code = NotFound desc = could not find container \"64c20ea99cbed10d600e8716909cd88c2a1b548237e198e41ccdb42a5893a177\": container with ID starting with 64c20ea99cbed10d600e8716909cd88c2a1b548237e198e41ccdb42a5893a177 not found: ID does not exist"
Oct 30 22:08:29.809576 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:29.809552    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/00517b45-26ad-41d2-b86e-99d9092c2403-istio-token\") pod \"e2e-worker-w0-779f32-openshift-default-75ddfd96d9-vtwfv\" (UID: \"00517b45-26ad-41d2-b86e-99d9092c2403\") " pod="openshift-ingress/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-vtwfv"
Oct 30 22:08:29.809731 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:29.809587    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/00517b45-26ad-41d2-b86e-99d9092c2403-istio-podinfo\") pod \"e2e-worker-w0-779f32-openshift-default-75ddfd96d9-vtwfv\" (UID: \"00517b45-26ad-41d2-b86e-99d9092c2403\") " pod="openshift-ingress/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-vtwfv"
Oct 30 22:08:29.809784 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:29.809761    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/00517b45-26ad-41d2-b86e-99d9092c2403-workload-socket\") pod \"e2e-worker-w0-779f32-openshift-default-75ddfd96d9-vtwfv\" (UID: \"00517b45-26ad-41d2-b86e-99d9092c2403\") " pod="openshift-ingress/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-vtwfv"
Oct 30 22:08:29.809839 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:29.809804    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/00517b45-26ad-41d2-b86e-99d9092c2403-istiod-ca-cert\") pod \"e2e-worker-w0-779f32-openshift-default-75ddfd96d9-vtwfv\" (UID: \"00517b45-26ad-41d2-b86e-99d9092c2403\") " pod="openshift-ingress/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-vtwfv"
Oct 30 22:08:29.809905 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:29.809888    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/00517b45-26ad-41d2-b86e-99d9092c2403-credential-socket\") pod \"e2e-worker-w0-779f32-openshift-default-75ddfd96d9-vtwfv\" (UID: \"00517b45-26ad-41d2-b86e-99d9092c2403\") " pod="openshift-ingress/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-vtwfv"
Oct 30 22:08:29.809954 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:29.809939    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/00517b45-26ad-41d2-b86e-99d9092c2403-istio-envoy\") pod \"e2e-worker-w0-779f32-openshift-default-75ddfd96d9-vtwfv\" (UID: \"00517b45-26ad-41d2-b86e-99d9092c2403\") " pod="openshift-ingress/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-vtwfv"
Oct 30 22:08:29.809990 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:29.809979    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xwkh8\" (UniqueName: \"kubernetes.io/projected/00517b45-26ad-41d2-b86e-99d9092c2403-kube-api-access-xwkh8\") pod \"e2e-worker-w0-779f32-openshift-default-75ddfd96d9-vtwfv\" (UID: \"00517b45-26ad-41d2-b86e-99d9092c2403\") " pod="openshift-ingress/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-vtwfv"
Oct 30 22:08:29.810032 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:29.810018    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/00517b45-26ad-41d2-b86e-99d9092c2403-workload-certs\") pod \"e2e-worker-w0-779f32-openshift-default-75ddfd96d9-vtwfv\" (UID: \"00517b45-26ad-41d2-b86e-99d9092c2403\") " pod="openshift-ingress/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-vtwfv"
Oct 30 22:08:29.810074 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:29.810041    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/00517b45-26ad-41d2-b86e-99d9092c2403-istio-data\") pod \"e2e-worker-w0-779f32-openshift-default-75ddfd96d9-vtwfv\" (UID: \"00517b45-26ad-41d2-b86e-99d9092c2403\") " pod="openshift-ingress/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-vtwfv"
Oct 30 22:08:29.810117 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:29.810100    5510 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-56njb\" (UniqueName: \"kubernetes.io/projected/bbd4c2ea-a56a-4f1f-9e37-a77b71326592-kube-api-access-56njb\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:08:29.810150 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:29.810115    5510 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bbd4c2ea-a56a-4f1f-9e37-a77b71326592-istio-envoy\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:08:29.810150 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:29.810128    5510 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bbd4c2ea-a56a-4f1f-9e37-a77b71326592-istio-podinfo\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:08:29.810150 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:29.810140    5510 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bbd4c2ea-a56a-4f1f-9e37-a77b71326592-istio-token\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:08:29.837559 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:29.837526    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-vtwfv"]
Oct 30 22:08:29.910874 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:29.910788    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xwkh8\" (UniqueName: \"kubernetes.io/projected/00517b45-26ad-41d2-b86e-99d9092c2403-kube-api-access-xwkh8\") pod \"e2e-worker-w0-779f32-openshift-default-75ddfd96d9-vtwfv\" (UID: \"00517b45-26ad-41d2-b86e-99d9092c2403\") " pod="openshift-ingress/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-vtwfv"
Oct 30 22:08:29.910874 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:29.910833    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/00517b45-26ad-41d2-b86e-99d9092c2403-workload-certs\") pod \"e2e-worker-w0-779f32-openshift-default-75ddfd96d9-vtwfv\" (UID: \"00517b45-26ad-41d2-b86e-99d9092c2403\") " pod="openshift-ingress/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-vtwfv"
Oct 30 22:08:29.911054 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:29.910949    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/00517b45-26ad-41d2-b86e-99d9092c2403-istio-data\") pod \"e2e-worker-w0-779f32-openshift-default-75ddfd96d9-vtwfv\" (UID: \"00517b45-26ad-41d2-b86e-99d9092c2403\") " pod="openshift-ingress/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-vtwfv"
Oct 30 22:08:29.911054 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:29.911029    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/00517b45-26ad-41d2-b86e-99d9092c2403-istio-token\") pod \"e2e-worker-w0-779f32-openshift-default-75ddfd96d9-vtwfv\" (UID: \"00517b45-26ad-41d2-b86e-99d9092c2403\") " pod="openshift-ingress/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-vtwfv"
Oct 30 22:08:29.911138 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:29.911060    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/00517b45-26ad-41d2-b86e-99d9092c2403-istio-podinfo\") pod \"e2e-worker-w0-779f32-openshift-default-75ddfd96d9-vtwfv\" (UID: \"00517b45-26ad-41d2-b86e-99d9092c2403\") " pod="openshift-ingress/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-vtwfv"
Oct 30 22:08:29.911181 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:29.911147    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/00517b45-26ad-41d2-b86e-99d9092c2403-workload-socket\") pod \"e2e-worker-w0-779f32-openshift-default-75ddfd96d9-vtwfv\" (UID: \"00517b45-26ad-41d2-b86e-99d9092c2403\") " pod="openshift-ingress/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-vtwfv"
Oct 30 22:08:29.911181 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:29.911151    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/00517b45-26ad-41d2-b86e-99d9092c2403-workload-certs\") pod \"e2e-worker-w0-779f32-openshift-default-75ddfd96d9-vtwfv\" (UID: \"00517b45-26ad-41d2-b86e-99d9092c2403\") " pod="openshift-ingress/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-vtwfv"
Oct 30 22:08:29.911181 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:29.911172    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/00517b45-26ad-41d2-b86e-99d9092c2403-istiod-ca-cert\") pod \"e2e-worker-w0-779f32-openshift-default-75ddfd96d9-vtwfv\" (UID: \"00517b45-26ad-41d2-b86e-99d9092c2403\") " pod="openshift-ingress/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-vtwfv"
Oct 30 22:08:29.911348 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:29.911238    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/00517b45-26ad-41d2-b86e-99d9092c2403-credential-socket\") pod \"e2e-worker-w0-779f32-openshift-default-75ddfd96d9-vtwfv\" (UID: \"00517b45-26ad-41d2-b86e-99d9092c2403\") " pod="openshift-ingress/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-vtwfv"
Oct 30 22:08:29.911348 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:29.911288    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/00517b45-26ad-41d2-b86e-99d9092c2403-istio-envoy\") pod \"e2e-worker-w0-779f32-openshift-default-75ddfd96d9-vtwfv\" (UID: \"00517b45-26ad-41d2-b86e-99d9092c2403\") " pod="openshift-ingress/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-vtwfv"
Oct 30 22:08:29.911348 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:29.911290    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/00517b45-26ad-41d2-b86e-99d9092c2403-istio-data\") pod \"e2e-worker-w0-779f32-openshift-default-75ddfd96d9-vtwfv\" (UID: \"00517b45-26ad-41d2-b86e-99d9092c2403\") " pod="openshift-ingress/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-vtwfv"
Oct 30 22:08:29.911520 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:29.911505    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/00517b45-26ad-41d2-b86e-99d9092c2403-workload-socket\") pod \"e2e-worker-w0-779f32-openshift-default-75ddfd96d9-vtwfv\" (UID: \"00517b45-26ad-41d2-b86e-99d9092c2403\") " pod="openshift-ingress/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-vtwfv"
Oct 30 22:08:29.911634 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:29.911613    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/00517b45-26ad-41d2-b86e-99d9092c2403-credential-socket\") pod \"e2e-worker-w0-779f32-openshift-default-75ddfd96d9-vtwfv\" (UID: \"00517b45-26ad-41d2-b86e-99d9092c2403\") " pod="openshift-ingress/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-vtwfv"
Oct 30 22:08:29.911876 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:29.911860    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/00517b45-26ad-41d2-b86e-99d9092c2403-istiod-ca-cert\") pod \"e2e-worker-w0-779f32-openshift-default-75ddfd96d9-vtwfv\" (UID: \"00517b45-26ad-41d2-b86e-99d9092c2403\") " pod="openshift-ingress/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-vtwfv"
Oct 30 22:08:29.913418 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:29.913398    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/00517b45-26ad-41d2-b86e-99d9092c2403-istio-envoy\") pod \"e2e-worker-w0-779f32-openshift-default-75ddfd96d9-vtwfv\" (UID: \"00517b45-26ad-41d2-b86e-99d9092c2403\") " pod="openshift-ingress/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-vtwfv"
Oct 30 22:08:29.913690 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:29.913651    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/00517b45-26ad-41d2-b86e-99d9092c2403-istio-podinfo\") pod \"e2e-worker-w0-779f32-openshift-default-75ddfd96d9-vtwfv\" (UID: \"00517b45-26ad-41d2-b86e-99d9092c2403\") " pod="openshift-ingress/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-vtwfv"
Oct 30 22:08:29.968854 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:29.968777    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/00517b45-26ad-41d2-b86e-99d9092c2403-istio-token\") pod \"e2e-worker-w0-779f32-openshift-default-75ddfd96d9-vtwfv\" (UID: \"00517b45-26ad-41d2-b86e-99d9092c2403\") " pod="openshift-ingress/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-vtwfv"
Oct 30 22:08:29.969000 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:29.968928    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xwkh8\" (UniqueName: \"kubernetes.io/projected/00517b45-26ad-41d2-b86e-99d9092c2403-kube-api-access-xwkh8\") pod \"e2e-worker-w0-779f32-openshift-default-75ddfd96d9-vtwfv\" (UID: \"00517b45-26ad-41d2-b86e-99d9092c2403\") " pod="openshift-ingress/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-vtwfv"
Oct 30 22:08:30.031772 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:30.031744    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-gbtm2"]
Oct 30 22:08:30.036422 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:30.036397    5510 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-gbtm2"]
Oct 30 22:08:30.040538 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:30.040513    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-vtwfv"
Oct 30 22:08:30.458940 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:30.458918    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-vtwfv"]
Oct 30 22:08:30.461591 ip-10-0-136-210 kubenswrapper[5510]: W1030 22:08:30.461561    5510 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod00517b45_26ad_41d2_b86e_99d9092c2403.slice/crio-7a04639681263207122fc121fcfc8ed37ad4cd9c19d36748c6870185bee7705f WatchSource:0}: Error finding container 7a04639681263207122fc121fcfc8ed37ad4cd9c19d36748c6870185bee7705f: Status 404 returned error can't find the container with id 7a04639681263207122fc121fcfc8ed37ad4cd9c19d36748c6870185bee7705f
Oct 30 22:08:30.464172 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:30.464139    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:08:30.464250 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:30.464207    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:08:30.464250 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:30.464236    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:08:30.817030 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:30.816991    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-vtwfv" event={"ID":"00517b45-26ad-41d2-b86e-99d9092c2403","Type":"ContainerStarted","Data":"a0e12056ee7c8d043a262bdc5684d0d0f797a7ac2e31c4f8f3d351f84bc90018"}
Oct 30 22:08:30.817614 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:30.817587    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-vtwfv" event={"ID":"00517b45-26ad-41d2-b86e-99d9092c2403","Type":"ContainerStarted","Data":"7a04639681263207122fc121fcfc8ed37ad4cd9c19d36748c6870185bee7705f"}
Oct 30 22:08:30.934279 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:30.934229    5510 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-vtwfv" podStartSLOduration=1.934216616 podStartE2EDuration="1.934216616s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:08:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:08:30.933629664 +0000 UTC m=+1091.494439682" watchObservedRunningTime="2026-10-30 22:08:30.934216616 +0000 UTC m=+1091.495026621"
Oct 30 22:08:31.041438 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:31.041400    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-vtwfv"
Oct 30 22:08:31.042955 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:31.042925    5510 patch_prober.go:29] interesting pod/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-vtwfv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body=
Oct 30 22:08:31.043090 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:31.042969    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-vtwfv" podUID="00517b45-26ad-41d2-b86e-99d9092c2403" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused"
Oct 30 22:08:31.586358 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:31.586300    5510 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="bbd4c2ea-a56a-4f1f-9e37-a77b71326592" path="/var/lib/kubelet/pods/bbd4c2ea-a56a-4f1f-9e37-a77b71326592/volumes"
Oct 30 22:08:32.041082 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:32.041044    5510 patch_prober.go:29] interesting pod/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-vtwfv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body=
Oct 30 22:08:32.041610 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:32.041113    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-vtwfv" podUID="00517b45-26ad-41d2-b86e-99d9092c2403" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused"
Oct 30 22:08:33.041544 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:33.041501    5510 patch_prober.go:29] interesting pod/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-vtwfv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body=
Oct 30 22:08:33.042031 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:33.041554    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-vtwfv" podUID="00517b45-26ad-41d2-b86e-99d9092c2403" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused"
Oct 30 22:08:33.732677 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:33.729250    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:08:34.041378 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:34.041344    5510 patch_prober.go:29] interesting pod/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-vtwfv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body=
Oct 30 22:08:34.041571 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:34.041403    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-vtwfv" podUID="00517b45-26ad-41d2-b86e-99d9092c2403" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused"
Oct 30 22:08:35.040955 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:35.040896    5510 patch_prober.go:29] interesting pod/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-vtwfv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body=
Oct 30 22:08:35.041113 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:35.040956    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-vtwfv" podUID="00517b45-26ad-41d2-b86e-99d9092c2403" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused"
Oct 30 22:08:36.041081 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:36.041044    5510 patch_prober.go:29] interesting pod/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-vtwfv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body=
Oct 30 22:08:36.041497 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:36.041100    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-vtwfv" podUID="00517b45-26ad-41d2-b86e-99d9092c2403" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused"
Oct 30 22:08:37.292839 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:37.292804    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-vtwfv"
Oct 30 22:08:37.293607 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:37.293587    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-vtwfv"
Oct 30 22:08:37.295921 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:37.295895    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-vtwfv"
Oct 30 22:08:38.419775 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:08:38.419736    5510 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6gjbf" containerName="extractor"
Oct 30 22:08:38.736001 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:38.735776    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:08:43.582940 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:43.582905    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:08:48.537182 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:48.537143    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:08:53.413239 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:53.413190    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:08:54.613672 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:54.613610    5510 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-94hzz"]
Oct 30 22:08:54.625900 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:54.625870    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-94hzz"
Oct 30 22:08:54.650549 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:54.650515    5510 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-cmqdj"]
Oct 30 22:08:54.663551 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:54.662217    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-cmqdj"
Oct 30 22:08:54.753516 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:54.753455    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fcded500-59d4-493e-bce3-fcf44d59b277-istio-data\") pod \"maas-default-gateway-openshift-default-85544d4cd7-94hzz\" (UID: \"fcded500-59d4-493e-bce3-fcf44d59b277\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-94hzz"
Oct 30 22:08:54.753838 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:54.753803    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f4952ed4-c979-43db-b64e-a5666ff61bed-istio-token\") pod \"maas-default-gateway-openshift-default-85544d4cd7-cmqdj\" (UID: \"f4952ed4-c979-43db-b64e-a5666ff61bed\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-cmqdj"
Oct 30 22:08:54.754015 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:54.753995    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f4952ed4-c979-43db-b64e-a5666ff61bed-istio-data\") pod \"maas-default-gateway-openshift-default-85544d4cd7-cmqdj\" (UID: \"f4952ed4-c979-43db-b64e-a5666ff61bed\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-cmqdj"
Oct 30 22:08:54.754158 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:54.754141    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fcded500-59d4-493e-bce3-fcf44d59b277-credential-socket\") pod \"maas-default-gateway-openshift-default-85544d4cd7-94hzz\" (UID: \"fcded500-59d4-493e-bce3-fcf44d59b277\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-94hzz"
Oct 30 22:08:54.754358 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:54.754342    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rr5lz\" (UniqueName: \"kubernetes.io/projected/f4952ed4-c979-43db-b64e-a5666ff61bed-kube-api-access-rr5lz\") pod \"maas-default-gateway-openshift-default-85544d4cd7-cmqdj\" (UID: \"f4952ed4-c979-43db-b64e-a5666ff61bed\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-cmqdj"
Oct 30 22:08:54.754477 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:54.754463    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fcded500-59d4-493e-bce3-fcf44d59b277-istio-podinfo\") pod \"maas-default-gateway-openshift-default-85544d4cd7-94hzz\" (UID: \"fcded500-59d4-493e-bce3-fcf44d59b277\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-94hzz"
Oct 30 22:08:54.754603 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:54.754589    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kfskv\" (UniqueName: \"kubernetes.io/projected/fcded500-59d4-493e-bce3-fcf44d59b277-kube-api-access-kfskv\") pod \"maas-default-gateway-openshift-default-85544d4cd7-94hzz\" (UID: \"fcded500-59d4-493e-bce3-fcf44d59b277\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-94hzz"
Oct 30 22:08:54.754742 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:54.754728    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fcded500-59d4-493e-bce3-fcf44d59b277-workload-certs\") pod \"maas-default-gateway-openshift-default-85544d4cd7-94hzz\" (UID: \"fcded500-59d4-493e-bce3-fcf44d59b277\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-94hzz"
Oct 30 22:08:54.754858 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:54.754845    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4952ed4-c979-43db-b64e-a5666ff61bed-workload-socket\") pod \"maas-default-gateway-openshift-default-85544d4cd7-cmqdj\" (UID: \"f4952ed4-c979-43db-b64e-a5666ff61bed\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-cmqdj"
Oct 30 22:08:54.754988 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:54.754975    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fcded500-59d4-493e-bce3-fcf44d59b277-istio-envoy\") pod \"maas-default-gateway-openshift-default-85544d4cd7-94hzz\" (UID: \"fcded500-59d4-493e-bce3-fcf44d59b277\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-94hzz"
Oct 30 22:08:54.755116 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:54.755101    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f4952ed4-c979-43db-b64e-a5666ff61bed-istio-podinfo\") pod \"maas-default-gateway-openshift-default-85544d4cd7-cmqdj\" (UID: \"f4952ed4-c979-43db-b64e-a5666ff61bed\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-cmqdj"
Oct 30 22:08:54.755213 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:54.755201    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fcded500-59d4-493e-bce3-fcf44d59b277-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-85544d4cd7-94hzz\" (UID: \"fcded500-59d4-493e-bce3-fcf44d59b277\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-94hzz"
Oct 30 22:08:54.755311 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:54.755300    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fcded500-59d4-493e-bce3-fcf44d59b277-istio-token\") pod \"maas-default-gateway-openshift-default-85544d4cd7-94hzz\" (UID: \"fcded500-59d4-493e-bce3-fcf44d59b277\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-94hzz"
Oct 30 22:08:54.755400 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:54.755389    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fcded500-59d4-493e-bce3-fcf44d59b277-workload-socket\") pod \"maas-default-gateway-openshift-default-85544d4cd7-94hzz\" (UID: \"fcded500-59d4-493e-bce3-fcf44d59b277\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-94hzz"
Oct 30 22:08:54.755536 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:54.755505    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f4952ed4-c979-43db-b64e-a5666ff61bed-istio-envoy\") pod \"maas-default-gateway-openshift-default-85544d4cd7-cmqdj\" (UID: \"f4952ed4-c979-43db-b64e-a5666ff61bed\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-cmqdj"
Oct 30 22:08:54.755807 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:54.755778    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f4952ed4-c979-43db-b64e-a5666ff61bed-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-85544d4cd7-cmqdj\" (UID: \"f4952ed4-c979-43db-b64e-a5666ff61bed\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-cmqdj"
Oct 30 22:08:54.755988 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:54.755970    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4952ed4-c979-43db-b64e-a5666ff61bed-credential-socket\") pod \"maas-default-gateway-openshift-default-85544d4cd7-cmqdj\" (UID: \"f4952ed4-c979-43db-b64e-a5666ff61bed\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-cmqdj"
Oct 30 22:08:54.756178 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:54.756156    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f4952ed4-c979-43db-b64e-a5666ff61bed-workload-certs\") pod \"maas-default-gateway-openshift-default-85544d4cd7-cmqdj\" (UID: \"f4952ed4-c979-43db-b64e-a5666ff61bed\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-cmqdj"
Oct 30 22:08:54.772561 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:54.772518    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-94hzz"]
Oct 30 22:08:54.772885 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:54.772862    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-cmqdj"]
Oct 30 22:08:54.861842 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:54.861804    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4952ed4-c979-43db-b64e-a5666ff61bed-workload-socket\") pod \"maas-default-gateway-openshift-default-85544d4cd7-cmqdj\" (UID: \"f4952ed4-c979-43db-b64e-a5666ff61bed\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-cmqdj"
Oct 30 22:08:54.865186 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:54.863426    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fcded500-59d4-493e-bce3-fcf44d59b277-istio-envoy\") pod \"maas-default-gateway-openshift-default-85544d4cd7-94hzz\" (UID: \"fcded500-59d4-493e-bce3-fcf44d59b277\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-94hzz"
Oct 30 22:08:54.865819 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:54.865785    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f4952ed4-c979-43db-b64e-a5666ff61bed-istio-podinfo\") pod \"maas-default-gateway-openshift-default-85544d4cd7-cmqdj\" (UID: \"f4952ed4-c979-43db-b64e-a5666ff61bed\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-cmqdj"
Oct 30 22:08:54.866077 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:54.866053    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fcded500-59d4-493e-bce3-fcf44d59b277-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-85544d4cd7-94hzz\" (UID: \"fcded500-59d4-493e-bce3-fcf44d59b277\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-94hzz"
Oct 30 22:08:54.866886 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:54.862635    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4952ed4-c979-43db-b64e-a5666ff61bed-workload-socket\") pod \"maas-default-gateway-openshift-default-85544d4cd7-cmqdj\" (UID: \"f4952ed4-c979-43db-b64e-a5666ff61bed\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-cmqdj"
Oct 30 22:08:54.867673 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:54.867627    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fcded500-59d4-493e-bce3-fcf44d59b277-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-85544d4cd7-94hzz\" (UID: \"fcded500-59d4-493e-bce3-fcf44d59b277\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-94hzz"
Oct 30 22:08:54.867842 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:54.867789    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fcded500-59d4-493e-bce3-fcf44d59b277-istio-token\") pod \"maas-default-gateway-openshift-default-85544d4cd7-94hzz\" (UID: \"fcded500-59d4-493e-bce3-fcf44d59b277\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-94hzz"
Oct 30 22:08:54.867997 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:54.867981    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fcded500-59d4-493e-bce3-fcf44d59b277-workload-socket\") pod \"maas-default-gateway-openshift-default-85544d4cd7-94hzz\" (UID: \"fcded500-59d4-493e-bce3-fcf44d59b277\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-94hzz"
Oct 30 22:08:54.868727 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:54.868707    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f4952ed4-c979-43db-b64e-a5666ff61bed-istio-envoy\") pod \"maas-default-gateway-openshift-default-85544d4cd7-cmqdj\" (UID: \"f4952ed4-c979-43db-b64e-a5666ff61bed\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-cmqdj"
Oct 30 22:08:54.869178 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:54.869157    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f4952ed4-c979-43db-b64e-a5666ff61bed-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-85544d4cd7-cmqdj\" (UID: \"f4952ed4-c979-43db-b64e-a5666ff61bed\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-cmqdj"
Oct 30 22:08:54.870032 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:54.870013    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4952ed4-c979-43db-b64e-a5666ff61bed-credential-socket\") pod \"maas-default-gateway-openshift-default-85544d4cd7-cmqdj\" (UID: \"f4952ed4-c979-43db-b64e-a5666ff61bed\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-cmqdj"
Oct 30 22:08:54.870495 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:54.870477    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f4952ed4-c979-43db-b64e-a5666ff61bed-workload-certs\") pod \"maas-default-gateway-openshift-default-85544d4cd7-cmqdj\" (UID: \"f4952ed4-c979-43db-b64e-a5666ff61bed\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-cmqdj"
Oct 30 22:08:54.871024 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:54.871007    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fcded500-59d4-493e-bce3-fcf44d59b277-istio-data\") pod \"maas-default-gateway-openshift-default-85544d4cd7-94hzz\" (UID: \"fcded500-59d4-493e-bce3-fcf44d59b277\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-94hzz"
Oct 30 22:08:54.871445 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:54.871429    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f4952ed4-c979-43db-b64e-a5666ff61bed-istio-token\") pod \"maas-default-gateway-openshift-default-85544d4cd7-cmqdj\" (UID: \"f4952ed4-c979-43db-b64e-a5666ff61bed\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-cmqdj"
Oct 30 22:08:54.871999 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:54.871980    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f4952ed4-c979-43db-b64e-a5666ff61bed-istio-data\") pod \"maas-default-gateway-openshift-default-85544d4cd7-cmqdj\" (UID: \"f4952ed4-c979-43db-b64e-a5666ff61bed\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-cmqdj"
Oct 30 22:08:54.872432 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:54.872415    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fcded500-59d4-493e-bce3-fcf44d59b277-credential-socket\") pod \"maas-default-gateway-openshift-default-85544d4cd7-94hzz\" (UID: \"fcded500-59d4-493e-bce3-fcf44d59b277\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-94hzz"
Oct 30 22:08:54.872893 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:54.871371    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fcded500-59d4-493e-bce3-fcf44d59b277-istio-data\") pod \"maas-default-gateway-openshift-default-85544d4cd7-94hzz\" (UID: \"fcded500-59d4-493e-bce3-fcf44d59b277\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-94hzz"
Oct 30 22:08:54.873021 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:54.870389    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4952ed4-c979-43db-b64e-a5666ff61bed-credential-socket\") pod \"maas-default-gateway-openshift-default-85544d4cd7-cmqdj\" (UID: \"f4952ed4-c979-43db-b64e-a5666ff61bed\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-cmqdj"
Oct 30 22:08:54.873113 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:54.869955    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f4952ed4-c979-43db-b64e-a5666ff61bed-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-85544d4cd7-cmqdj\" (UID: \"f4952ed4-c979-43db-b64e-a5666ff61bed\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-cmqdj"
Oct 30 22:08:54.873200 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:54.872354    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f4952ed4-c979-43db-b64e-a5666ff61bed-istio-data\") pod \"maas-default-gateway-openshift-default-85544d4cd7-cmqdj\" (UID: \"f4952ed4-c979-43db-b64e-a5666ff61bed\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-cmqdj"
Oct 30 22:08:54.873281 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:54.870902    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f4952ed4-c979-43db-b64e-a5666ff61bed-workload-certs\") pod \"maas-default-gateway-openshift-default-85544d4cd7-cmqdj\" (UID: \"f4952ed4-c979-43db-b64e-a5666ff61bed\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-cmqdj"
Oct 30 22:08:54.873372 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:54.868627    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fcded500-59d4-493e-bce3-fcf44d59b277-workload-socket\") pod \"maas-default-gateway-openshift-default-85544d4cd7-94hzz\" (UID: \"fcded500-59d4-493e-bce3-fcf44d59b277\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-94hzz"
Oct 30 22:08:54.873454 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:54.872792    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fcded500-59d4-493e-bce3-fcf44d59b277-credential-socket\") pod \"maas-default-gateway-openshift-default-85544d4cd7-94hzz\" (UID: \"fcded500-59d4-493e-bce3-fcf44d59b277\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-94hzz"
Oct 30 22:08:54.873625 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:54.873609    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rr5lz\" (UniqueName: \"kubernetes.io/projected/f4952ed4-c979-43db-b64e-a5666ff61bed-kube-api-access-rr5lz\") pod \"maas-default-gateway-openshift-default-85544d4cd7-cmqdj\" (UID: \"f4952ed4-c979-43db-b64e-a5666ff61bed\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-cmqdj"
Oct 30 22:08:54.887080 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:54.887047    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fcded500-59d4-493e-bce3-fcf44d59b277-istio-podinfo\") pod \"maas-default-gateway-openshift-default-85544d4cd7-94hzz\" (UID: \"fcded500-59d4-493e-bce3-fcf44d59b277\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-94hzz"
Oct 30 22:08:54.915103 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:54.915059    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kfskv\" (UniqueName: \"kubernetes.io/projected/fcded500-59d4-493e-bce3-fcf44d59b277-kube-api-access-kfskv\") pod \"maas-default-gateway-openshift-default-85544d4cd7-94hzz\" (UID: \"fcded500-59d4-493e-bce3-fcf44d59b277\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-94hzz"
Oct 30 22:08:54.915735 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:54.915708    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fcded500-59d4-493e-bce3-fcf44d59b277-workload-certs\") pod \"maas-default-gateway-openshift-default-85544d4cd7-94hzz\" (UID: \"fcded500-59d4-493e-bce3-fcf44d59b277\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-94hzz"
Oct 30 22:08:54.916322 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:54.916217    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fcded500-59d4-493e-bce3-fcf44d59b277-workload-certs\") pod \"maas-default-gateway-openshift-default-85544d4cd7-94hzz\" (UID: \"fcded500-59d4-493e-bce3-fcf44d59b277\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-94hzz"
Oct 30 22:08:54.919764 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:54.919741    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fcded500-59d4-493e-bce3-fcf44d59b277-istio-podinfo\") pod \"maas-default-gateway-openshift-default-85544d4cd7-94hzz\" (UID: \"fcded500-59d4-493e-bce3-fcf44d59b277\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-94hzz"
Oct 30 22:08:54.920759 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:54.920707    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fcded500-59d4-493e-bce3-fcf44d59b277-istio-envoy\") pod \"maas-default-gateway-openshift-default-85544d4cd7-94hzz\" (UID: \"fcded500-59d4-493e-bce3-fcf44d59b277\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-94hzz"
Oct 30 22:08:54.921615 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:54.921411    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f4952ed4-c979-43db-b64e-a5666ff61bed-istio-podinfo\") pod \"maas-default-gateway-openshift-default-85544d4cd7-cmqdj\" (UID: \"f4952ed4-c979-43db-b64e-a5666ff61bed\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-cmqdj"
Oct 30 22:08:54.924076 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:54.924023    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f4952ed4-c979-43db-b64e-a5666ff61bed-istio-envoy\") pod \"maas-default-gateway-openshift-default-85544d4cd7-cmqdj\" (UID: \"f4952ed4-c979-43db-b64e-a5666ff61bed\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-cmqdj"
Oct 30 22:08:54.956216 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:54.954958    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rr5lz\" (UniqueName: \"kubernetes.io/projected/f4952ed4-c979-43db-b64e-a5666ff61bed-kube-api-access-rr5lz\") pod \"maas-default-gateway-openshift-default-85544d4cd7-cmqdj\" (UID: \"f4952ed4-c979-43db-b64e-a5666ff61bed\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-cmqdj"
Oct 30 22:08:54.958567 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:54.958502    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fcded500-59d4-493e-bce3-fcf44d59b277-istio-token\") pod \"maas-default-gateway-openshift-default-85544d4cd7-94hzz\" (UID: \"fcded500-59d4-493e-bce3-fcf44d59b277\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-94hzz"
Oct 30 22:08:54.963159 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:54.963121    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f4952ed4-c979-43db-b64e-a5666ff61bed-istio-token\") pod \"maas-default-gateway-openshift-default-85544d4cd7-cmqdj\" (UID: \"f4952ed4-c979-43db-b64e-a5666ff61bed\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-cmqdj"
Oct 30 22:08:54.980075 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:54.980041    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kfskv\" (UniqueName: \"kubernetes.io/projected/fcded500-59d4-493e-bce3-fcf44d59b277-kube-api-access-kfskv\") pod \"maas-default-gateway-openshift-default-85544d4cd7-94hzz\" (UID: \"fcded500-59d4-493e-bce3-fcf44d59b277\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-94hzz"
Oct 30 22:08:55.017358 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:55.017315    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-cmqdj"
Oct 30 22:08:55.275800 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:55.274912    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-94hzz"
Oct 30 22:08:55.502694 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:55.495914    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-cmqdj"]
Oct 30 22:08:55.537597 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:55.537549    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:08:55.537883 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:55.537855    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:08:55.538045 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:55.538021    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:08:56.128771 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:56.128730    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-cmqdj" event={"ID":"f4952ed4-c979-43db-b64e-a5666ff61bed","Type":"ContainerStarted","Data":"bb00e9ced4ca50e14707c8dc534d10165f1a8872dcf84fb3c7d175f6d3b54199"}
Oct 30 22:08:56.284737 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:56.284694    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-94hzz"]
Oct 30 22:08:56.310763 ip-10-0-136-210 kubenswrapper[5510]: W1030 22:08:56.291823    5510 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfcded500_59d4_493e_bce3_fcf44d59b277.slice/crio-82f086a26602e45df84749ef406f8e07078c6874220ead0bf78908f4a66c6f6f WatchSource:0}: Error finding container 82f086a26602e45df84749ef406f8e07078c6874220ead0bf78908f4a66c6f6f: Status 404 returned error can't find the container with id 82f086a26602e45df84749ef406f8e07078c6874220ead0bf78908f4a66c6f6f
Oct 30 22:08:56.319103 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:56.319041    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:08:56.319383 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:56.319349    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:08:56.319602 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:56.319580    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:08:57.135848 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:57.135809    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-cmqdj" event={"ID":"f4952ed4-c979-43db-b64e-a5666ff61bed","Type":"ContainerStarted","Data":"d1b90d7e93a28b2233e3419789ca4f482604ccd85169e7a02120f220b5a4b3b9"}
Oct 30 22:08:57.137518 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:57.137491    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-94hzz" event={"ID":"fcded500-59d4-493e-bce3-fcf44d59b277","Type":"ContainerStarted","Data":"ee36ff048c63cdf014bad75038d4fbe701e8f26c644e42558ba341e38ad04c21"}
Oct 30 22:08:57.137627 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:57.137531    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-94hzz" event={"ID":"fcded500-59d4-493e-bce3-fcf44d59b277","Type":"ContainerStarted","Data":"82f086a26602e45df84749ef406f8e07078c6874220ead0bf78908f4a66c6f6f"}
Oct 30 22:08:57.281681 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:57.280994    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-94hzz"
Oct 30 22:08:57.285839 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:57.284779    5510 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-85544d4cd7-94hzz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body=
Oct 30 22:08:57.285839 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:57.284844    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-94hzz" podUID="fcded500-59d4-493e-bce3-fcf44d59b277" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused"
Oct 30 22:08:57.330492 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:57.330402    5510 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-cmqdj" podStartSLOduration=3.330386747 podStartE2EDuration="3.330386747s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:08:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:08:57.328008398 +0000 UTC m=+1117.888818419" watchObservedRunningTime="2026-10-30 22:08:57.330386747 +0000 UTC m=+1117.891196751"
Oct 30 22:08:57.454282 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:57.453437    5510 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-94hzz" podStartSLOduration=3.453415644 podStartE2EDuration="3.453415644s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:08:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:08:57.443430397 +0000 UTC m=+1118.004240401" watchObservedRunningTime="2026-10-30 22:08:57.453415644 +0000 UTC m=+1118.014225651"
Oct 30 22:08:58.018308 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:58.018270    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-cmqdj"
Oct 30 22:08:58.020048 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:58.020024    5510 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-85544d4cd7-cmqdj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body=
Oct 30 22:08:58.020150 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:58.020073    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-cmqdj" podUID="f4952ed4-c979-43db-b64e-a5666ff61bed" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused"
Oct 30 22:08:58.279329 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:58.279219    5510 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-85544d4cd7-94hzz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body=
Oct 30 22:08:58.279981 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:58.279931    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-94hzz" podUID="fcded500-59d4-493e-bce3-fcf44d59b277" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused"
Oct 30 22:08:58.781689 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:58.776497    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:08:59.017990 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:59.017960    5510 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-85544d4cd7-cmqdj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body=
Oct 30 22:08:59.018218 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:59.018196    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-cmqdj" podUID="f4952ed4-c979-43db-b64e-a5666ff61bed" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused"
Oct 30 22:08:59.293694 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:59.292813    5510 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-85544d4cd7-94hzz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body=
Oct 30 22:08:59.293694 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:08:59.292872    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-94hzz" podUID="fcded500-59d4-493e-bce3-fcf44d59b277" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused"
Oct 30 22:09:00.018396 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:00.018364    5510 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-85544d4cd7-cmqdj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body=
Oct 30 22:09:00.018570 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:00.018416    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-cmqdj" podUID="f4952ed4-c979-43db-b64e-a5666ff61bed" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused"
Oct 30 22:09:00.286968 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:00.286860    5510 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-85544d4cd7-94hzz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body=
Oct 30 22:09:00.287279 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:00.287226    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-94hzz" podUID="fcded500-59d4-493e-bce3-fcf44d59b277" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused"
Oct 30 22:09:01.018618 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:01.018573    5510 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-85544d4cd7-cmqdj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body=
Oct 30 22:09:01.019047 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:01.018644    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-cmqdj" podUID="f4952ed4-c979-43db-b64e-a5666ff61bed" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused"
Oct 30 22:09:01.290744 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:01.281778    5510 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-85544d4cd7-94hzz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body=
Oct 30 22:09:01.290744 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:01.281846    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-94hzz" podUID="fcded500-59d4-493e-bce3-fcf44d59b277" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused"
Oct 30 22:09:02.018643 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:02.018607    5510 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-85544d4cd7-cmqdj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body=
Oct 30 22:09:02.019083 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:02.018691    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-cmqdj" podUID="f4952ed4-c979-43db-b64e-a5666ff61bed" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused"
Oct 30 22:09:02.286747 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:02.277102    5510 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-85544d4cd7-94hzz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body=
Oct 30 22:09:02.293691 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:02.287003    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-94hzz" podUID="fcded500-59d4-493e-bce3-fcf44d59b277" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused"
Oct 30 22:09:03.017994 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:03.017954    5510 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-85544d4cd7-cmqdj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body=
Oct 30 22:09:03.018166 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:03.018025    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-cmqdj" podUID="f4952ed4-c979-43db-b64e-a5666ff61bed" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused"
Oct 30 22:09:03.276963 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:03.276883    5510 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-85544d4cd7-94hzz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body=
Oct 30 22:09:03.276963 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:03.276940    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-94hzz" podUID="fcded500-59d4-493e-bce3-fcf44d59b277" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused"
Oct 30 22:09:03.575954 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:03.575856    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:09:03.941759 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:03.935358    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:09:04.018412 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:04.018372    5510 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-85544d4cd7-cmqdj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body=
Oct 30 22:09:04.018608 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:04.018453    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-cmqdj" podUID="f4952ed4-c979-43db-b64e-a5666ff61bed" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused"
Oct 30 22:09:04.276831 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:04.276792    5510 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-85544d4cd7-94hzz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body=
Oct 30 22:09:04.276982 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:04.276863    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-94hzz" podUID="fcded500-59d4-493e-bce3-fcf44d59b277" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused"
Oct 30 22:09:05.052735 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:05.052702    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-cmqdj"
Oct 30 22:09:05.053154 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:05.053126    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-cmqdj"
Oct 30 22:09:05.053975 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:05.053957    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-cmqdj"
Oct 30 22:09:05.380155 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:05.380068    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-94hzz"
Oct 30 22:09:05.381751 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:05.381729    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-94hzz"
Oct 30 22:09:05.384026 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:05.384002    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-94hzz"
Oct 30 22:09:06.546566 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:06.546527    5510 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w6-fb0b93/distinct-w6-fb0b93-kserve-58db6b6757-6mww8"]
Oct 30 22:09:06.550561 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:06.550536    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-fb0b93/distinct-w6-fb0b93-kserve-58db6b6757-6mww8"
Oct 30 22:09:06.646363 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:06.646326    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w6-fb0b93\"/\"distinct-w6-fb0b93-kserve-self-signed-certs\""
Oct 30 22:09:06.648791 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:06.648769    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w6-fb0b93/distinct-w6-fb0b93-kserve-58db6b6757-6mww8"]
Oct 30 22:09:06.690281 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:06.690245    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8ee6579b-acea-4492-862b-9883f70d00c9-tmp-dir\") pod \"distinct-w6-fb0b93-kserve-58db6b6757-6mww8\" (UID: \"8ee6579b-acea-4492-862b-9883f70d00c9\") " pod="e2e-models-e2e-worker-w6-fb0b93/distinct-w6-fb0b93-kserve-58db6b6757-6mww8"
Oct 30 22:09:06.690488 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:06.690342    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8ee6579b-acea-4492-862b-9883f70d00c9-home\") pod \"distinct-w6-fb0b93-kserve-58db6b6757-6mww8\" (UID: \"8ee6579b-acea-4492-862b-9883f70d00c9\") " pod="e2e-models-e2e-worker-w6-fb0b93/distinct-w6-fb0b93-kserve-58db6b6757-6mww8"
Oct 30 22:09:06.690488 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:06.690404    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8ee6579b-acea-4492-862b-9883f70d00c9-tls-certs\") pod \"distinct-w6-fb0b93-kserve-58db6b6757-6mww8\" (UID: \"8ee6579b-acea-4492-862b-9883f70d00c9\") " pod="e2e-models-e2e-worker-w6-fb0b93/distinct-w6-fb0b93-kserve-58db6b6757-6mww8"
Oct 30 22:09:06.690488 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:06.690441    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8ee6579b-acea-4492-862b-9883f70d00c9-dshm\") pod \"distinct-w6-fb0b93-kserve-58db6b6757-6mww8\" (UID: \"8ee6579b-acea-4492-862b-9883f70d00c9\") " pod="e2e-models-e2e-worker-w6-fb0b93/distinct-w6-fb0b93-kserve-58db6b6757-6mww8"
Oct 30 22:09:06.690488 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:06.690472    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8ee6579b-acea-4492-862b-9883f70d00c9-model-cache\") pod \"distinct-w6-fb0b93-kserve-58db6b6757-6mww8\" (UID: \"8ee6579b-acea-4492-862b-9883f70d00c9\") " pod="e2e-models-e2e-worker-w6-fb0b93/distinct-w6-fb0b93-kserve-58db6b6757-6mww8"
Oct 30 22:09:06.690744 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:06.690614    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tn89t\" (UniqueName: \"kubernetes.io/projected/8ee6579b-acea-4492-862b-9883f70d00c9-kube-api-access-tn89t\") pod \"distinct-w6-fb0b93-kserve-58db6b6757-6mww8\" (UID: \"8ee6579b-acea-4492-862b-9883f70d00c9\") " pod="e2e-models-e2e-worker-w6-fb0b93/distinct-w6-fb0b93-kserve-58db6b6757-6mww8"
Oct 30 22:09:06.694331 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:06.694299    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:09:06.791400 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:06.791357    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8ee6579b-acea-4492-862b-9883f70d00c9-tmp-dir\") pod \"distinct-w6-fb0b93-kserve-58db6b6757-6mww8\" (UID: \"8ee6579b-acea-4492-862b-9883f70d00c9\") " pod="e2e-models-e2e-worker-w6-fb0b93/distinct-w6-fb0b93-kserve-58db6b6757-6mww8"
Oct 30 22:09:06.791609 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:06.791408    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8ee6579b-acea-4492-862b-9883f70d00c9-home\") pod \"distinct-w6-fb0b93-kserve-58db6b6757-6mww8\" (UID: \"8ee6579b-acea-4492-862b-9883f70d00c9\") " pod="e2e-models-e2e-worker-w6-fb0b93/distinct-w6-fb0b93-kserve-58db6b6757-6mww8"
Oct 30 22:09:06.791609 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:06.791457    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8ee6579b-acea-4492-862b-9883f70d00c9-tls-certs\") pod \"distinct-w6-fb0b93-kserve-58db6b6757-6mww8\" (UID: \"8ee6579b-acea-4492-862b-9883f70d00c9\") " pod="e2e-models-e2e-worker-w6-fb0b93/distinct-w6-fb0b93-kserve-58db6b6757-6mww8"
Oct 30 22:09:06.791609 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:06.791488    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8ee6579b-acea-4492-862b-9883f70d00c9-dshm\") pod \"distinct-w6-fb0b93-kserve-58db6b6757-6mww8\" (UID: \"8ee6579b-acea-4492-862b-9883f70d00c9\") " pod="e2e-models-e2e-worker-w6-fb0b93/distinct-w6-fb0b93-kserve-58db6b6757-6mww8"
Oct 30 22:09:06.791880 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:06.791857    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8ee6579b-acea-4492-862b-9883f70d00c9-tmp-dir\") pod \"distinct-w6-fb0b93-kserve-58db6b6757-6mww8\" (UID: \"8ee6579b-acea-4492-862b-9883f70d00c9\") " pod="e2e-models-e2e-worker-w6-fb0b93/distinct-w6-fb0b93-kserve-58db6b6757-6mww8"
Oct 30 22:09:06.791954 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:06.791857    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8ee6579b-acea-4492-862b-9883f70d00c9-model-cache\") pod \"distinct-w6-fb0b93-kserve-58db6b6757-6mww8\" (UID: \"8ee6579b-acea-4492-862b-9883f70d00c9\") " pod="e2e-models-e2e-worker-w6-fb0b93/distinct-w6-fb0b93-kserve-58db6b6757-6mww8"
Oct 30 22:09:06.791954 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:06.791893    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8ee6579b-acea-4492-862b-9883f70d00c9-home\") pod \"distinct-w6-fb0b93-kserve-58db6b6757-6mww8\" (UID: \"8ee6579b-acea-4492-862b-9883f70d00c9\") " pod="e2e-models-e2e-worker-w6-fb0b93/distinct-w6-fb0b93-kserve-58db6b6757-6mww8"
Oct 30 22:09:06.792077 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:06.792058    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tn89t\" (UniqueName: \"kubernetes.io/projected/8ee6579b-acea-4492-862b-9883f70d00c9-kube-api-access-tn89t\") pod \"distinct-w6-fb0b93-kserve-58db6b6757-6mww8\" (UID: \"8ee6579b-acea-4492-862b-9883f70d00c9\") " pod="e2e-models-e2e-worker-w6-fb0b93/distinct-w6-fb0b93-kserve-58db6b6757-6mww8"
Oct 30 22:09:06.792195 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:06.792171    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8ee6579b-acea-4492-862b-9883f70d00c9-model-cache\") pod \"distinct-w6-fb0b93-kserve-58db6b6757-6mww8\" (UID: \"8ee6579b-acea-4492-862b-9883f70d00c9\") " pod="e2e-models-e2e-worker-w6-fb0b93/distinct-w6-fb0b93-kserve-58db6b6757-6mww8"
Oct 30 22:09:06.794120 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:06.794096    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8ee6579b-acea-4492-862b-9883f70d00c9-tls-certs\") pod \"distinct-w6-fb0b93-kserve-58db6b6757-6mww8\" (UID: \"8ee6579b-acea-4492-862b-9883f70d00c9\") " pod="e2e-models-e2e-worker-w6-fb0b93/distinct-w6-fb0b93-kserve-58db6b6757-6mww8"
Oct 30 22:09:06.794247 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:06.794096    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8ee6579b-acea-4492-862b-9883f70d00c9-dshm\") pod \"distinct-w6-fb0b93-kserve-58db6b6757-6mww8\" (UID: \"8ee6579b-acea-4492-862b-9883f70d00c9\") " pod="e2e-models-e2e-worker-w6-fb0b93/distinct-w6-fb0b93-kserve-58db6b6757-6mww8"
Oct 30 22:09:06.851198 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:06.851124    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tn89t\" (UniqueName: \"kubernetes.io/projected/8ee6579b-acea-4492-862b-9883f70d00c9-kube-api-access-tn89t\") pod \"distinct-w6-fb0b93-kserve-58db6b6757-6mww8\" (UID: \"8ee6579b-acea-4492-862b-9883f70d00c9\") " pod="e2e-models-e2e-worker-w6-fb0b93/distinct-w6-fb0b93-kserve-58db6b6757-6mww8"
Oct 30 22:09:06.862024 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:06.861991    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-fb0b93/distinct-w6-fb0b93-kserve-58db6b6757-6mww8"
Oct 30 22:09:07.298195 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:07.298170    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w6-fb0b93/distinct-w6-fb0b93-kserve-58db6b6757-6mww8"]
Oct 30 22:09:07.300543 ip-10-0-136-210 kubenswrapper[5510]: W1030 22:09:07.300509    5510 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8ee6579b_acea_4492_862b_9883f70d00c9.slice/crio-a4a36d48d33b4b00ce9271673d274ba760abb031ce06b77b78a7d926c5d2a656 WatchSource:0}: Error finding container a4a36d48d33b4b00ce9271673d274ba760abb031ce06b77b78a7d926c5d2a656: Status 404 returned error can't find the container with id a4a36d48d33b4b00ce9271673d274ba760abb031ce06b77b78a7d926c5d2a656
Oct 30 22:09:08.256349 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:08.256298    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-fb0b93/distinct-w6-fb0b93-kserve-58db6b6757-6mww8" event={"ID":"8ee6579b-acea-4492-862b-9883f70d00c9","Type":"ContainerStarted","Data":"097741694ce77df0e84010eb664b49a7ca93e0eac3e0a699133ff9d5af9e104d"}
Oct 30 22:09:08.256784 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:08.256360    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-fb0b93/distinct-w6-fb0b93-kserve-58db6b6757-6mww8" event={"ID":"8ee6579b-acea-4492-862b-9883f70d00c9","Type":"ContainerStarted","Data":"a4a36d48d33b4b00ce9271673d274ba760abb031ce06b77b78a7d926c5d2a656"}
Oct 30 22:09:08.369302 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:08.369248    5510 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w6-fb0b93/distinct-w6-fb0b93-kserve-58db6b6757-6mww8" podStartSLOduration=2.36923253 podStartE2EDuration="2.36923253s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:09:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:09:08.368448929 +0000 UTC m=+1128.929258935" watchObservedRunningTime="2026-10-30 22:09:08.36923253 +0000 UTC m=+1128.930042534"
Oct 30 22:09:08.420120 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:09:08.420084    5510 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6gjbf" containerName="extractor"
Oct 30 22:09:08.870424 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:08.870382    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-fb0b93/distinct-w6-fb0b93-kserve-58db6b6757-6mww8"
Oct 30 22:09:08.879679 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:08.877704    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:09:13.731677 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:13.726912    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:09:15.092175 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:15.092121    5510 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-jtsvv"]
Oct 30 22:09:15.100548 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:15.100233    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-jtsvv"
Oct 30 22:09:15.209356 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:15.209243    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0101f982-8c38-4576-be55-949dfc3fc2de-credential-socket\") pod \"e2e-worker-w0-779f32-openshift-default-75ddfd96d9-jtsvv\" (UID: \"0101f982-8c38-4576-be55-949dfc3fc2de\") " pod="openshift-ingress/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-jtsvv"
Oct 30 22:09:15.209648 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:15.209626    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0101f982-8c38-4576-be55-949dfc3fc2de-istio-token\") pod \"e2e-worker-w0-779f32-openshift-default-75ddfd96d9-jtsvv\" (UID: \"0101f982-8c38-4576-be55-949dfc3fc2de\") " pod="openshift-ingress/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-jtsvv"
Oct 30 22:09:15.209869 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:15.209850    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0101f982-8c38-4576-be55-949dfc3fc2de-workload-socket\") pod \"e2e-worker-w0-779f32-openshift-default-75ddfd96d9-jtsvv\" (UID: \"0101f982-8c38-4576-be55-949dfc3fc2de\") " pod="openshift-ingress/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-jtsvv"
Oct 30 22:09:15.210043 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:15.210027    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0101f982-8c38-4576-be55-949dfc3fc2de-istio-envoy\") pod \"e2e-worker-w0-779f32-openshift-default-75ddfd96d9-jtsvv\" (UID: \"0101f982-8c38-4576-be55-949dfc3fc2de\") " pod="openshift-ingress/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-jtsvv"
Oct 30 22:09:15.210177 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:15.210159    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0101f982-8c38-4576-be55-949dfc3fc2de-workload-certs\") pod \"e2e-worker-w0-779f32-openshift-default-75ddfd96d9-jtsvv\" (UID: \"0101f982-8c38-4576-be55-949dfc3fc2de\") " pod="openshift-ingress/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-jtsvv"
Oct 30 22:09:15.210346 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:15.210333    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0101f982-8c38-4576-be55-949dfc3fc2de-istiod-ca-cert\") pod \"e2e-worker-w0-779f32-openshift-default-75ddfd96d9-jtsvv\" (UID: \"0101f982-8c38-4576-be55-949dfc3fc2de\") " pod="openshift-ingress/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-jtsvv"
Oct 30 22:09:15.210473 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:15.210457    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0101f982-8c38-4576-be55-949dfc3fc2de-istio-podinfo\") pod \"e2e-worker-w0-779f32-openshift-default-75ddfd96d9-jtsvv\" (UID: \"0101f982-8c38-4576-be55-949dfc3fc2de\") " pod="openshift-ingress/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-jtsvv"
Oct 30 22:09:15.210593 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:15.210578    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8lhzp\" (UniqueName: \"kubernetes.io/projected/0101f982-8c38-4576-be55-949dfc3fc2de-kube-api-access-8lhzp\") pod \"e2e-worker-w0-779f32-openshift-default-75ddfd96d9-jtsvv\" (UID: \"0101f982-8c38-4576-be55-949dfc3fc2de\") " pod="openshift-ingress/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-jtsvv"
Oct 30 22:09:15.210732 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:15.210715    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0101f982-8c38-4576-be55-949dfc3fc2de-istio-data\") pod \"e2e-worker-w0-779f32-openshift-default-75ddfd96d9-jtsvv\" (UID: \"0101f982-8c38-4576-be55-949dfc3fc2de\") " pod="openshift-ingress/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-jtsvv"
Oct 30 22:09:15.210918 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:15.209386    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-jtsvv"]
Oct 30 22:09:15.318451 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:15.318413    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0101f982-8c38-4576-be55-949dfc3fc2de-istiod-ca-cert\") pod \"e2e-worker-w0-779f32-openshift-default-75ddfd96d9-jtsvv\" (UID: \"0101f982-8c38-4576-be55-949dfc3fc2de\") " pod="openshift-ingress/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-jtsvv"
Oct 30 22:09:15.325687 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:15.321850    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0101f982-8c38-4576-be55-949dfc3fc2de-istio-podinfo\") pod \"e2e-worker-w0-779f32-openshift-default-75ddfd96d9-jtsvv\" (UID: \"0101f982-8c38-4576-be55-949dfc3fc2de\") " pod="openshift-ingress/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-jtsvv"
Oct 30 22:09:15.334699 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:15.325957    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8lhzp\" (UniqueName: \"kubernetes.io/projected/0101f982-8c38-4576-be55-949dfc3fc2de-kube-api-access-8lhzp\") pod \"e2e-worker-w0-779f32-openshift-default-75ddfd96d9-jtsvv\" (UID: \"0101f982-8c38-4576-be55-949dfc3fc2de\") " pod="openshift-ingress/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-jtsvv"
Oct 30 22:09:15.335044 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:15.335021    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0101f982-8c38-4576-be55-949dfc3fc2de-istio-data\") pod \"e2e-worker-w0-779f32-openshift-default-75ddfd96d9-jtsvv\" (UID: \"0101f982-8c38-4576-be55-949dfc3fc2de\") " pod="openshift-ingress/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-jtsvv"
Oct 30 22:09:15.335314 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:15.335281    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0101f982-8c38-4576-be55-949dfc3fc2de-credential-socket\") pod \"e2e-worker-w0-779f32-openshift-default-75ddfd96d9-jtsvv\" (UID: \"0101f982-8c38-4576-be55-949dfc3fc2de\") " pod="openshift-ingress/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-jtsvv"
Oct 30 22:09:15.355957 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:15.350418    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0101f982-8c38-4576-be55-949dfc3fc2de-istiod-ca-cert\") pod \"e2e-worker-w0-779f32-openshift-default-75ddfd96d9-jtsvv\" (UID: \"0101f982-8c38-4576-be55-949dfc3fc2de\") " pod="openshift-ingress/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-jtsvv"
Oct 30 22:09:15.357936 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:15.357903    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0101f982-8c38-4576-be55-949dfc3fc2de-istio-token\") pod \"e2e-worker-w0-779f32-openshift-default-75ddfd96d9-jtsvv\" (UID: \"0101f982-8c38-4576-be55-949dfc3fc2de\") " pod="openshift-ingress/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-jtsvv"
Oct 30 22:09:15.358191 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:15.358156    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0101f982-8c38-4576-be55-949dfc3fc2de-workload-socket\") pod \"e2e-worker-w0-779f32-openshift-default-75ddfd96d9-jtsvv\" (UID: \"0101f982-8c38-4576-be55-949dfc3fc2de\") " pod="openshift-ingress/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-jtsvv"
Oct 30 22:09:15.373875 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:15.373844    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0101f982-8c38-4576-be55-949dfc3fc2de-istio-envoy\") pod \"e2e-worker-w0-779f32-openshift-default-75ddfd96d9-jtsvv\" (UID: \"0101f982-8c38-4576-be55-949dfc3fc2de\") " pod="openshift-ingress/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-jtsvv"
Oct 30 22:09:15.374093 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:15.374074    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0101f982-8c38-4576-be55-949dfc3fc2de-workload-certs\") pod \"e2e-worker-w0-779f32-openshift-default-75ddfd96d9-jtsvv\" (UID: \"0101f982-8c38-4576-be55-949dfc3fc2de\") " pod="openshift-ingress/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-jtsvv"
Oct 30 22:09:15.459826 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:15.366084    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0101f982-8c38-4576-be55-949dfc3fc2de-istio-data\") pod \"e2e-worker-w0-779f32-openshift-default-75ddfd96d9-jtsvv\" (UID: \"0101f982-8c38-4576-be55-949dfc3fc2de\") " pod="openshift-ingress/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-jtsvv"
Oct 30 22:09:15.459826 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:15.374502    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0101f982-8c38-4576-be55-949dfc3fc2de-workload-socket\") pod \"e2e-worker-w0-779f32-openshift-default-75ddfd96d9-jtsvv\" (UID: \"0101f982-8c38-4576-be55-949dfc3fc2de\") " pod="openshift-ingress/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-jtsvv"
Oct 30 22:09:15.459826 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:15.366312    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0101f982-8c38-4576-be55-949dfc3fc2de-credential-socket\") pod \"e2e-worker-w0-779f32-openshift-default-75ddfd96d9-jtsvv\" (UID: \"0101f982-8c38-4576-be55-949dfc3fc2de\") " pod="openshift-ingress/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-jtsvv"
Oct 30 22:09:15.459826 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:15.374790    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0101f982-8c38-4576-be55-949dfc3fc2de-workload-certs\") pod \"e2e-worker-w0-779f32-openshift-default-75ddfd96d9-jtsvv\" (UID: \"0101f982-8c38-4576-be55-949dfc3fc2de\") " pod="openshift-ingress/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-jtsvv"
Oct 30 22:09:15.459826 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:15.421776    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0101f982-8c38-4576-be55-949dfc3fc2de-istio-envoy\") pod \"e2e-worker-w0-779f32-openshift-default-75ddfd96d9-jtsvv\" (UID: \"0101f982-8c38-4576-be55-949dfc3fc2de\") " pod="openshift-ingress/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-jtsvv"
Oct 30 22:09:15.459826 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:15.434272    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0101f982-8c38-4576-be55-949dfc3fc2de-istio-podinfo\") pod \"e2e-worker-w0-779f32-openshift-default-75ddfd96d9-jtsvv\" (UID: \"0101f982-8c38-4576-be55-949dfc3fc2de\") " pod="openshift-ingress/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-jtsvv"
Oct 30 22:09:15.494691 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:15.470204    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0101f982-8c38-4576-be55-949dfc3fc2de-istio-token\") pod \"e2e-worker-w0-779f32-openshift-default-75ddfd96d9-jtsvv\" (UID: \"0101f982-8c38-4576-be55-949dfc3fc2de\") " pod="openshift-ingress/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-jtsvv"
Oct 30 22:09:15.539391 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:15.526507    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8lhzp\" (UniqueName: \"kubernetes.io/projected/0101f982-8c38-4576-be55-949dfc3fc2de-kube-api-access-8lhzp\") pod \"e2e-worker-w0-779f32-openshift-default-75ddfd96d9-jtsvv\" (UID: \"0101f982-8c38-4576-be55-949dfc3fc2de\") " pod="openshift-ingress/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-jtsvv"
Oct 30 22:09:15.732731 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:15.732086    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-jtsvv"
Oct 30 22:09:16.145178 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:16.142013    5510 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-5jqnb"]
Oct 30 22:09:16.187710 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:16.186320    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-5jqnb"
Oct 30 22:09:16.209049 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:16.209018    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-5jqnb"]
Oct 30 22:09:16.278980 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:16.278139    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eb812de4-a46c-41c5-adc7-7b325b1de62b-workload-certs\") pod \"e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-5jqnb\" (UID: \"eb812de4-a46c-41c5-adc7-7b325b1de62b\") " pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-5jqnb"
Oct 30 22:09:16.278980 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:16.278190    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eb812de4-a46c-41c5-adc7-7b325b1de62b-istio-token\") pod \"e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-5jqnb\" (UID: \"eb812de4-a46c-41c5-adc7-7b325b1de62b\") " pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-5jqnb"
Oct 30 22:09:16.278980 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:16.278257    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb812de4-a46c-41c5-adc7-7b325b1de62b-workload-socket\") pod \"e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-5jqnb\" (UID: \"eb812de4-a46c-41c5-adc7-7b325b1de62b\") " pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-5jqnb"
Oct 30 22:09:16.278980 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:16.278340    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eb812de4-a46c-41c5-adc7-7b325b1de62b-istiod-ca-cert\") pod \"e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-5jqnb\" (UID: \"eb812de4-a46c-41c5-adc7-7b325b1de62b\") " pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-5jqnb"
Oct 30 22:09:16.278980 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:16.278401    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eb812de4-a46c-41c5-adc7-7b325b1de62b-istio-envoy\") pod \"e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-5jqnb\" (UID: \"eb812de4-a46c-41c5-adc7-7b325b1de62b\") " pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-5jqnb"
Oct 30 22:09:16.278980 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:16.278457    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9m88b\" (UniqueName: \"kubernetes.io/projected/eb812de4-a46c-41c5-adc7-7b325b1de62b-kube-api-access-9m88b\") pod \"e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-5jqnb\" (UID: \"eb812de4-a46c-41c5-adc7-7b325b1de62b\") " pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-5jqnb"
Oct 30 22:09:16.278980 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:16.278512    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb812de4-a46c-41c5-adc7-7b325b1de62b-credential-socket\") pod \"e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-5jqnb\" (UID: \"eb812de4-a46c-41c5-adc7-7b325b1de62b\") " pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-5jqnb"
Oct 30 22:09:16.278980 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:16.278545    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eb812de4-a46c-41c5-adc7-7b325b1de62b-istio-podinfo\") pod \"e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-5jqnb\" (UID: \"eb812de4-a46c-41c5-adc7-7b325b1de62b\") " pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-5jqnb"
Oct 30 22:09:16.278980 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:16.278812    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eb812de4-a46c-41c5-adc7-7b325b1de62b-istio-data\") pod \"e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-5jqnb\" (UID: \"eb812de4-a46c-41c5-adc7-7b325b1de62b\") " pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-5jqnb"
Oct 30 22:09:16.380220 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:16.380139    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eb812de4-a46c-41c5-adc7-7b325b1de62b-istio-data\") pod \"e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-5jqnb\" (UID: \"eb812de4-a46c-41c5-adc7-7b325b1de62b\") " pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-5jqnb"
Oct 30 22:09:16.380402 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:16.380222    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eb812de4-a46c-41c5-adc7-7b325b1de62b-workload-certs\") pod \"e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-5jqnb\" (UID: \"eb812de4-a46c-41c5-adc7-7b325b1de62b\") " pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-5jqnb"
Oct 30 22:09:16.380402 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:16.380245    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eb812de4-a46c-41c5-adc7-7b325b1de62b-istio-token\") pod \"e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-5jqnb\" (UID: \"eb812de4-a46c-41c5-adc7-7b325b1de62b\") " pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-5jqnb"
Oct 30 22:09:16.380402 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:16.380297    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb812de4-a46c-41c5-adc7-7b325b1de62b-workload-socket\") pod \"e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-5jqnb\" (UID: \"eb812de4-a46c-41c5-adc7-7b325b1de62b\") " pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-5jqnb"
Oct 30 22:09:16.380402 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:16.380366    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eb812de4-a46c-41c5-adc7-7b325b1de62b-istiod-ca-cert\") pod \"e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-5jqnb\" (UID: \"eb812de4-a46c-41c5-adc7-7b325b1de62b\") " pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-5jqnb"
Oct 30 22:09:16.380590 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:16.380425    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eb812de4-a46c-41c5-adc7-7b325b1de62b-istio-envoy\") pod \"e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-5jqnb\" (UID: \"eb812de4-a46c-41c5-adc7-7b325b1de62b\") " pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-5jqnb"
Oct 30 22:09:16.380590 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:16.380482    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9m88b\" (UniqueName: \"kubernetes.io/projected/eb812de4-a46c-41c5-adc7-7b325b1de62b-kube-api-access-9m88b\") pod \"e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-5jqnb\" (UID: \"eb812de4-a46c-41c5-adc7-7b325b1de62b\") " pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-5jqnb"
Oct 30 22:09:16.380590 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:16.380539    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb812de4-a46c-41c5-adc7-7b325b1de62b-credential-socket\") pod \"e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-5jqnb\" (UID: \"eb812de4-a46c-41c5-adc7-7b325b1de62b\") " pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-5jqnb"
Oct 30 22:09:16.380590 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:16.380568    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eb812de4-a46c-41c5-adc7-7b325b1de62b-istio-podinfo\") pod \"e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-5jqnb\" (UID: \"eb812de4-a46c-41c5-adc7-7b325b1de62b\") " pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-5jqnb"
Oct 30 22:09:16.380815 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:16.380673    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eb812de4-a46c-41c5-adc7-7b325b1de62b-istio-data\") pod \"e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-5jqnb\" (UID: \"eb812de4-a46c-41c5-adc7-7b325b1de62b\") " pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-5jqnb"
Oct 30 22:09:16.380815 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:16.380736    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eb812de4-a46c-41c5-adc7-7b325b1de62b-workload-certs\") pod \"e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-5jqnb\" (UID: \"eb812de4-a46c-41c5-adc7-7b325b1de62b\") " pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-5jqnb"
Oct 30 22:09:16.381061 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:16.381030    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb812de4-a46c-41c5-adc7-7b325b1de62b-credential-socket\") pod \"e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-5jqnb\" (UID: \"eb812de4-a46c-41c5-adc7-7b325b1de62b\") " pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-5jqnb"
Oct 30 22:09:16.381193 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:16.381139    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb812de4-a46c-41c5-adc7-7b325b1de62b-workload-socket\") pod \"e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-5jqnb\" (UID: \"eb812de4-a46c-41c5-adc7-7b325b1de62b\") " pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-5jqnb"
Oct 30 22:09:16.381193 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:16.381159    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eb812de4-a46c-41c5-adc7-7b325b1de62b-istiod-ca-cert\") pod \"e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-5jqnb\" (UID: \"eb812de4-a46c-41c5-adc7-7b325b1de62b\") " pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-5jqnb"
Oct 30 22:09:16.382779 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:16.382760    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eb812de4-a46c-41c5-adc7-7b325b1de62b-istio-envoy\") pod \"e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-5jqnb\" (UID: \"eb812de4-a46c-41c5-adc7-7b325b1de62b\") " pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-5jqnb"
Oct 30 22:09:16.383159 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:16.383139    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eb812de4-a46c-41c5-adc7-7b325b1de62b-istio-podinfo\") pod \"e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-5jqnb\" (UID: \"eb812de4-a46c-41c5-adc7-7b325b1de62b\") " pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-5jqnb"
Oct 30 22:09:16.440600 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:16.440565    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eb812de4-a46c-41c5-adc7-7b325b1de62b-istio-token\") pod \"e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-5jqnb\" (UID: \"eb812de4-a46c-41c5-adc7-7b325b1de62b\") " pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-5jqnb"
Oct 30 22:09:16.440820 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:16.440799    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9m88b\" (UniqueName: \"kubernetes.io/projected/eb812de4-a46c-41c5-adc7-7b325b1de62b-kube-api-access-9m88b\") pod \"e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-5jqnb\" (UID: \"eb812de4-a46c-41c5-adc7-7b325b1de62b\") " pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-5jqnb"
Oct 30 22:09:16.497728 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:16.497688    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-jtsvv"]
Oct 30 22:09:16.500377 ip-10-0-136-210 kubenswrapper[5510]: W1030 22:09:16.500339    5510 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0101f982_8c38_4576_be55_949dfc3fc2de.slice/crio-5658a607c7d4d4d8a331699557783c14c93a5b8aac3870bfd4a63ae819a998c4 WatchSource:0}: Error finding container 5658a607c7d4d4d8a331699557783c14c93a5b8aac3870bfd4a63ae819a998c4: Status 404 returned error can't find the container with id 5658a607c7d4d4d8a331699557783c14c93a5b8aac3870bfd4a63ae819a998c4
Oct 30 22:09:16.502939 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:16.502904    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:09:16.503053 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:16.502975    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:09:16.503053 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:16.503003    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:09:16.521699 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:16.521674    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-5jqnb"
Oct 30 22:09:16.538455 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:16.538428    5510 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-ffkrj"]
Oct 30 22:09:16.542901 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:16.542881    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-ffkrj"
Oct 30 22:09:16.583878 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:16.583476    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2937b77d-4ec3-4b52-86aa-b2e115927327-istiod-ca-cert\") pod \"e2e-worker-w1-a00d9b-openshift-default-847df8fc59-ffkrj\" (UID: \"2937b77d-4ec3-4b52-86aa-b2e115927327\") " pod="openshift-ingress/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-ffkrj"
Oct 30 22:09:16.583878 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:16.583600    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2937b77d-4ec3-4b52-86aa-b2e115927327-istio-data\") pod \"e2e-worker-w1-a00d9b-openshift-default-847df8fc59-ffkrj\" (UID: \"2937b77d-4ec3-4b52-86aa-b2e115927327\") " pod="openshift-ingress/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-ffkrj"
Oct 30 22:09:16.583878 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:16.583729    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vw7gz\" (UniqueName: \"kubernetes.io/projected/2937b77d-4ec3-4b52-86aa-b2e115927327-kube-api-access-vw7gz\") pod \"e2e-worker-w1-a00d9b-openshift-default-847df8fc59-ffkrj\" (UID: \"2937b77d-4ec3-4b52-86aa-b2e115927327\") " pod="openshift-ingress/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-ffkrj"
Oct 30 22:09:16.583878 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:16.583785    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2937b77d-4ec3-4b52-86aa-b2e115927327-istio-podinfo\") pod \"e2e-worker-w1-a00d9b-openshift-default-847df8fc59-ffkrj\" (UID: \"2937b77d-4ec3-4b52-86aa-b2e115927327\") " pod="openshift-ingress/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-ffkrj"
Oct 30 22:09:16.585227 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:16.584273    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2937b77d-4ec3-4b52-86aa-b2e115927327-istio-envoy\") pod \"e2e-worker-w1-a00d9b-openshift-default-847df8fc59-ffkrj\" (UID: \"2937b77d-4ec3-4b52-86aa-b2e115927327\") " pod="openshift-ingress/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-ffkrj"
Oct 30 22:09:16.585227 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:16.584387    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2937b77d-4ec3-4b52-86aa-b2e115927327-workload-socket\") pod \"e2e-worker-w1-a00d9b-openshift-default-847df8fc59-ffkrj\" (UID: \"2937b77d-4ec3-4b52-86aa-b2e115927327\") " pod="openshift-ingress/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-ffkrj"
Oct 30 22:09:16.585227 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:16.584439    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2937b77d-4ec3-4b52-86aa-b2e115927327-credential-socket\") pod \"e2e-worker-w1-a00d9b-openshift-default-847df8fc59-ffkrj\" (UID: \"2937b77d-4ec3-4b52-86aa-b2e115927327\") " pod="openshift-ingress/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-ffkrj"
Oct 30 22:09:16.585227 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:16.584469    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2937b77d-4ec3-4b52-86aa-b2e115927327-workload-certs\") pod \"e2e-worker-w1-a00d9b-openshift-default-847df8fc59-ffkrj\" (UID: \"2937b77d-4ec3-4b52-86aa-b2e115927327\") " pod="openshift-ingress/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-ffkrj"
Oct 30 22:09:16.585227 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:16.584510    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2937b77d-4ec3-4b52-86aa-b2e115927327-istio-token\") pod \"e2e-worker-w1-a00d9b-openshift-default-847df8fc59-ffkrj\" (UID: \"2937b77d-4ec3-4b52-86aa-b2e115927327\") " pod="openshift-ingress/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-ffkrj"
Oct 30 22:09:16.610569 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:16.610251    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-ffkrj"]
Oct 30 22:09:16.685474 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:16.685387    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2937b77d-4ec3-4b52-86aa-b2e115927327-istio-envoy\") pod \"e2e-worker-w1-a00d9b-openshift-default-847df8fc59-ffkrj\" (UID: \"2937b77d-4ec3-4b52-86aa-b2e115927327\") " pod="openshift-ingress/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-ffkrj"
Oct 30 22:09:16.685474 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:16.685446    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2937b77d-4ec3-4b52-86aa-b2e115927327-workload-socket\") pod \"e2e-worker-w1-a00d9b-openshift-default-847df8fc59-ffkrj\" (UID: \"2937b77d-4ec3-4b52-86aa-b2e115927327\") " pod="openshift-ingress/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-ffkrj"
Oct 30 22:09:16.685739 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:16.685484    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2937b77d-4ec3-4b52-86aa-b2e115927327-credential-socket\") pod \"e2e-worker-w1-a00d9b-openshift-default-847df8fc59-ffkrj\" (UID: \"2937b77d-4ec3-4b52-86aa-b2e115927327\") " pod="openshift-ingress/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-ffkrj"
Oct 30 22:09:16.685739 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:16.685506    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2937b77d-4ec3-4b52-86aa-b2e115927327-workload-certs\") pod \"e2e-worker-w1-a00d9b-openshift-default-847df8fc59-ffkrj\" (UID: \"2937b77d-4ec3-4b52-86aa-b2e115927327\") " pod="openshift-ingress/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-ffkrj"
Oct 30 22:09:16.685739 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:16.685538    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2937b77d-4ec3-4b52-86aa-b2e115927327-istio-token\") pod \"e2e-worker-w1-a00d9b-openshift-default-847df8fc59-ffkrj\" (UID: \"2937b77d-4ec3-4b52-86aa-b2e115927327\") " pod="openshift-ingress/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-ffkrj"
Oct 30 22:09:16.685739 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:16.685599    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2937b77d-4ec3-4b52-86aa-b2e115927327-istiod-ca-cert\") pod \"e2e-worker-w1-a00d9b-openshift-default-847df8fc59-ffkrj\" (UID: \"2937b77d-4ec3-4b52-86aa-b2e115927327\") " pod="openshift-ingress/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-ffkrj"
Oct 30 22:09:16.685739 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:16.685698    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2937b77d-4ec3-4b52-86aa-b2e115927327-istio-data\") pod \"e2e-worker-w1-a00d9b-openshift-default-847df8fc59-ffkrj\" (UID: \"2937b77d-4ec3-4b52-86aa-b2e115927327\") " pod="openshift-ingress/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-ffkrj"
Oct 30 22:09:16.685981 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:16.685793    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vw7gz\" (UniqueName: \"kubernetes.io/projected/2937b77d-4ec3-4b52-86aa-b2e115927327-kube-api-access-vw7gz\") pod \"e2e-worker-w1-a00d9b-openshift-default-847df8fc59-ffkrj\" (UID: \"2937b77d-4ec3-4b52-86aa-b2e115927327\") " pod="openshift-ingress/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-ffkrj"
Oct 30 22:09:16.685981 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:16.685844    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2937b77d-4ec3-4b52-86aa-b2e115927327-istio-podinfo\") pod \"e2e-worker-w1-a00d9b-openshift-default-847df8fc59-ffkrj\" (UID: \"2937b77d-4ec3-4b52-86aa-b2e115927327\") " pod="openshift-ingress/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-ffkrj"
Oct 30 22:09:16.686120 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:16.686100    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2937b77d-4ec3-4b52-86aa-b2e115927327-workload-certs\") pod \"e2e-worker-w1-a00d9b-openshift-default-847df8fc59-ffkrj\" (UID: \"2937b77d-4ec3-4b52-86aa-b2e115927327\") " pod="openshift-ingress/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-ffkrj"
Oct 30 22:09:16.686377 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:16.686357    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2937b77d-4ec3-4b52-86aa-b2e115927327-workload-socket\") pod \"e2e-worker-w1-a00d9b-openshift-default-847df8fc59-ffkrj\" (UID: \"2937b77d-4ec3-4b52-86aa-b2e115927327\") " pod="openshift-ingress/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-ffkrj"
Oct 30 22:09:16.686525 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:16.686503    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2937b77d-4ec3-4b52-86aa-b2e115927327-credential-socket\") pod \"e2e-worker-w1-a00d9b-openshift-default-847df8fc59-ffkrj\" (UID: \"2937b77d-4ec3-4b52-86aa-b2e115927327\") " pod="openshift-ingress/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-ffkrj"
Oct 30 22:09:16.686790 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:16.686768    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2937b77d-4ec3-4b52-86aa-b2e115927327-istio-data\") pod \"e2e-worker-w1-a00d9b-openshift-default-847df8fc59-ffkrj\" (UID: \"2937b77d-4ec3-4b52-86aa-b2e115927327\") " pod="openshift-ingress/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-ffkrj"
Oct 30 22:09:16.687062 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:16.687045    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2937b77d-4ec3-4b52-86aa-b2e115927327-istiod-ca-cert\") pod \"e2e-worker-w1-a00d9b-openshift-default-847df8fc59-ffkrj\" (UID: \"2937b77d-4ec3-4b52-86aa-b2e115927327\") " pod="openshift-ingress/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-ffkrj"
Oct 30 22:09:16.687840 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:16.687800    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2937b77d-4ec3-4b52-86aa-b2e115927327-istio-envoy\") pod \"e2e-worker-w1-a00d9b-openshift-default-847df8fc59-ffkrj\" (UID: \"2937b77d-4ec3-4b52-86aa-b2e115927327\") " pod="openshift-ingress/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-ffkrj"
Oct 30 22:09:16.689935 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:16.689913    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2937b77d-4ec3-4b52-86aa-b2e115927327-istio-podinfo\") pod \"e2e-worker-w1-a00d9b-openshift-default-847df8fc59-ffkrj\" (UID: \"2937b77d-4ec3-4b52-86aa-b2e115927327\") " pod="openshift-ingress/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-ffkrj"
Oct 30 22:09:16.744551 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:16.744517    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2937b77d-4ec3-4b52-86aa-b2e115927327-istio-token\") pod \"e2e-worker-w1-a00d9b-openshift-default-847df8fc59-ffkrj\" (UID: \"2937b77d-4ec3-4b52-86aa-b2e115927327\") " pod="openshift-ingress/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-ffkrj"
Oct 30 22:09:16.744714 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:16.744616    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vw7gz\" (UniqueName: \"kubernetes.io/projected/2937b77d-4ec3-4b52-86aa-b2e115927327-kube-api-access-vw7gz\") pod \"e2e-worker-w1-a00d9b-openshift-default-847df8fc59-ffkrj\" (UID: \"2937b77d-4ec3-4b52-86aa-b2e115927327\") " pod="openshift-ingress/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-ffkrj"
Oct 30 22:09:16.863031 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:16.862990    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w6-fb0b93/distinct-w6-fb0b93-kserve-58db6b6757-6mww8"
Oct 30 22:09:16.863031 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:16.863024    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-ffkrj"
Oct 30 22:09:16.878120 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:16.878089    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w6-fb0b93/distinct-w6-fb0b93-kserve-58db6b6757-6mww8"
Oct 30 22:09:16.915228 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:16.915198    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-5jqnb"]
Oct 30 22:09:16.918217 ip-10-0-136-210 kubenswrapper[5510]: W1030 22:09:16.918076    5510 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb812de4_a46c_41c5_adc7_7b325b1de62b.slice/crio-f8259c87f9dc4c6fb17d7b4a38fd1787ffb5236baacde2ece9442d4e0882b0f5 WatchSource:0}: Error finding container f8259c87f9dc4c6fb17d7b4a38fd1787ffb5236baacde2ece9442d4e0882b0f5: Status 404 returned error can't find the container with id f8259c87f9dc4c6fb17d7b4a38fd1787ffb5236baacde2ece9442d4e0882b0f5
Oct 30 22:09:16.921316 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:16.919813    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:09:16.921316 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:16.919879    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:09:16.921316 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:16.919906    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:09:17.268699 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:17.263421    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-ffkrj"]
Oct 30 22:09:17.277033 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:17.276995    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:09:17.277160 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:17.277082    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:09:17.277160 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:17.277128    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:09:17.389148 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:17.389116    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-5jqnb" event={"ID":"eb812de4-a46c-41c5-adc7-7b325b1de62b","Type":"ContainerStarted","Data":"db66658c383f8cf461bd43d30d252998ed414cbeeb79e8126c36be3ed4a3ca86"}
Oct 30 22:09:17.389309 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:17.389182    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-5jqnb" event={"ID":"eb812de4-a46c-41c5-adc7-7b325b1de62b","Type":"ContainerStarted","Data":"f8259c87f9dc4c6fb17d7b4a38fd1787ffb5236baacde2ece9442d4e0882b0f5"}
Oct 30 22:09:17.391309 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:17.391247    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-ffkrj" event={"ID":"2937b77d-4ec3-4b52-86aa-b2e115927327","Type":"ContainerStarted","Data":"349968883e13a9093e2802f7c1bee63828f29f8f3eafc55d5d8d85f21478c610"}
Oct 30 22:09:17.391309 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:17.391290    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-ffkrj" event={"ID":"2937b77d-4ec3-4b52-86aa-b2e115927327","Type":"ContainerStarted","Data":"1f2ae9c334561f4f2997a0b0be0410a89a9df83567746ca14a214ce6bdbeb638"}
Oct 30 22:09:17.394096 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:17.393993    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-jtsvv" event={"ID":"0101f982-8c38-4576-be55-949dfc3fc2de","Type":"ContainerStarted","Data":"97e7fc3b556f7b3864d94a4ad7254ea0ff9badb4a6e1da68d91af1e3451c4a30"}
Oct 30 22:09:17.394096 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:17.394045    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-jtsvv" event={"ID":"0101f982-8c38-4576-be55-949dfc3fc2de","Type":"ContainerStarted","Data":"5658a607c7d4d4d8a331699557783c14c93a5b8aac3870bfd4a63ae819a998c4"}
Oct 30 22:09:17.410744 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:17.410680    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-fb0b93/distinct-w6-fb0b93-kserve-58db6b6757-6mww8"
Oct 30 22:09:17.511707 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:17.509470    5510 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-5jqnb" podStartSLOduration=1.5094530000000002 podStartE2EDuration="1.509453s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:09:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:09:17.507927922 +0000 UTC m=+1138.068737927" watchObservedRunningTime="2026-10-30 22:09:17.509453 +0000 UTC m=+1138.070263002"
Oct 30 22:09:17.522093 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:17.522020    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-5jqnb"
Oct 30 22:09:17.737678 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:17.732986    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-jtsvv"
Oct 30 22:09:17.737678 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:17.734436    5510 patch_prober.go:29] interesting pod/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-jtsvv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body=
Oct 30 22:09:17.737678 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:17.734483    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-jtsvv" podUID="0101f982-8c38-4576-be55-949dfc3fc2de" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused"
Oct 30 22:09:17.800739 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:17.799582    5510 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-jtsvv" podStartSLOduration=2.799558288 podStartE2EDuration="2.799558288s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:09:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:09:17.766008784 +0000 UTC m=+1138.326818790" watchObservedRunningTime="2026-10-30 22:09:17.799558288 +0000 UTC m=+1138.360368293"
Oct 30 22:09:17.865812 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:17.865769    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-ffkrj"
Oct 30 22:09:17.906686 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:17.906368    5510 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-ffkrj" podStartSLOduration=1.906346402 podStartE2EDuration="1.906346402s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:09:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:09:17.899954493 +0000 UTC m=+1138.460764500" watchObservedRunningTime="2026-10-30 22:09:17.906346402 +0000 UTC m=+1138.467156407"
Oct 30 22:09:18.524611 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:18.524576    5510 patch_prober.go:29] interesting pod/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-5jqnb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body=
Oct 30 22:09:18.525105 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:18.524637    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-5jqnb" podUID="eb812de4-a46c-41c5-adc7-7b325b1de62b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused"
Oct 30 22:09:18.733263 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:18.733227    5510 patch_prober.go:29] interesting pod/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-jtsvv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body=
Oct 30 22:09:18.733470 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:18.733286    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-jtsvv" podUID="0101f982-8c38-4576-be55-949dfc3fc2de" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused"
Oct 30 22:09:18.865739 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:18.865502    5510 patch_prober.go:29] interesting pod/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-ffkrj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body=
Oct 30 22:09:18.865739 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:18.865564    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-ffkrj" podUID="2937b77d-4ec3-4b52-86aa-b2e115927327" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused"
Oct 30 22:09:19.137733 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:19.137456    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:09:19.523150 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:19.523108    5510 patch_prober.go:29] interesting pod/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-5jqnb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body=
Oct 30 22:09:19.523346 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:19.523177    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-5jqnb" podUID="eb812de4-a46c-41c5-adc7-7b325b1de62b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused"
Oct 30 22:09:19.735500 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:19.735458    5510 patch_prober.go:29] interesting pod/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-jtsvv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body=
Oct 30 22:09:19.736000 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:19.735548    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-jtsvv" podUID="0101f982-8c38-4576-be55-949dfc3fc2de" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused"
Oct 30 22:09:19.869362 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:19.868865    5510 patch_prober.go:29] interesting pod/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-ffkrj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body=
Oct 30 22:09:19.869362 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:19.868939    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-ffkrj" podUID="2937b77d-4ec3-4b52-86aa-b2e115927327" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused"
Oct 30 22:09:20.524460 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:20.524200    5510 patch_prober.go:29] interesting pod/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-5jqnb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body=
Oct 30 22:09:20.524460 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:20.524267    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-5jqnb" podUID="eb812de4-a46c-41c5-adc7-7b325b1de62b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused"
Oct 30 22:09:20.733172 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:20.733135    5510 patch_prober.go:29] interesting pod/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-jtsvv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body=
Oct 30 22:09:20.733324 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:20.733192    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-jtsvv" podUID="0101f982-8c38-4576-be55-949dfc3fc2de" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused"
Oct 30 22:09:20.864190 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:20.864089    5510 patch_prober.go:29] interesting pod/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-ffkrj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body=
Oct 30 22:09:20.864811 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:20.864772    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-ffkrj" podUID="2937b77d-4ec3-4b52-86aa-b2e115927327" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused"
Oct 30 22:09:21.523034 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:21.523000    5510 patch_prober.go:29] interesting pod/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-5jqnb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body=
Oct 30 22:09:21.523359 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:21.523327    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-5jqnb" podUID="eb812de4-a46c-41c5-adc7-7b325b1de62b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused"
Oct 30 22:09:21.733296 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:21.733254    5510 patch_prober.go:29] interesting pod/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-jtsvv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body=
Oct 30 22:09:21.733500 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:21.733314    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-jtsvv" podUID="0101f982-8c38-4576-be55-949dfc3fc2de" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused"
Oct 30 22:09:21.872003 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:21.871914    5510 patch_prober.go:29] interesting pod/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-ffkrj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body=
Oct 30 22:09:21.872587 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:21.872557    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-ffkrj" podUID="2937b77d-4ec3-4b52-86aa-b2e115927327" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused"
Oct 30 22:09:22.522458 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:22.522419    5510 patch_prober.go:29] interesting pod/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-5jqnb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body=
Oct 30 22:09:22.522641 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:22.522489    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-5jqnb" podUID="eb812de4-a46c-41c5-adc7-7b325b1de62b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused"
Oct 30 22:09:22.733602 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:22.733562    5510 patch_prober.go:29] interesting pod/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-jtsvv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body=
Oct 30 22:09:22.733784 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:22.733625    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-jtsvv" podUID="0101f982-8c38-4576-be55-949dfc3fc2de" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused"
Oct 30 22:09:22.869550 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:22.869454    5510 patch_prober.go:29] interesting pod/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-ffkrj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body=
Oct 30 22:09:22.869843 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:22.869811    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-ffkrj" podUID="2937b77d-4ec3-4b52-86aa-b2e115927327" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused"
Oct 30 22:09:23.522554 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:23.522515    5510 patch_prober.go:29] interesting pod/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-5jqnb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body=
Oct 30 22:09:23.523081 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:23.522586    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-5jqnb" podUID="eb812de4-a46c-41c5-adc7-7b325b1de62b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused"
Oct 30 22:09:23.867739 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:23.867082    5510 patch_prober.go:29] interesting pod/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-ffkrj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body=
Oct 30 22:09:23.867739 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:23.867150    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-ffkrj" podUID="2937b77d-4ec3-4b52-86aa-b2e115927327" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused"
Oct 30 22:09:23.887220 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:23.887190    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-jtsvv"
Oct 30 22:09:23.888135 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:23.888104    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-jtsvv"
Oct 30 22:09:23.889136 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:23.889114    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-jtsvv"
Oct 30 22:09:24.060508 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:24.060467    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:09:24.511695 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:24.507175    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:09:24.539261 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:24.539224    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-5jqnb"
Oct 30 22:09:24.539979 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:24.539822    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-5jqnb"
Oct 30 22:09:24.541084 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:24.541061    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-5jqnb"
Oct 30 22:09:24.868600 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:24.868569    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-ffkrj"
Oct 30 22:09:24.868904 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:24.868884    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-ffkrj"
Oct 30 22:09:24.869510 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:24.869490    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-ffkrj"
Oct 30 22:09:28.957424 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:28.957392    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:09:29.435451 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:29.435408    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:09:30.900161 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:30.900120    5510 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-w8zj5"]
Oct 30 22:09:30.905711 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:30.905684    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-w8zj5"
Oct 30 22:09:31.002178 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:31.002131    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-w8zj5"]
Oct 30 22:09:31.052866 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:31.052827    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/07ed7cb3-6111-4809-8b1f-27a41ff6cafb-workload-certs\") pod \"e2e-worker-w2-bc714d-openshift-default-67558b95cf-w8zj5\" (UID: \"07ed7cb3-6111-4809-8b1f-27a41ff6cafb\") " pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-w8zj5"
Oct 30 22:09:31.053077 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:31.052982    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/07ed7cb3-6111-4809-8b1f-27a41ff6cafb-istio-token\") pod \"e2e-worker-w2-bc714d-openshift-default-67558b95cf-w8zj5\" (UID: \"07ed7cb3-6111-4809-8b1f-27a41ff6cafb\") " pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-w8zj5"
Oct 30 22:09:31.053077 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:31.053039    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/07ed7cb3-6111-4809-8b1f-27a41ff6cafb-istio-podinfo\") pod \"e2e-worker-w2-bc714d-openshift-default-67558b95cf-w8zj5\" (UID: \"07ed7cb3-6111-4809-8b1f-27a41ff6cafb\") " pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-w8zj5"
Oct 30 22:09:31.053077 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:31.053072    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/07ed7cb3-6111-4809-8b1f-27a41ff6cafb-istiod-ca-cert\") pod \"e2e-worker-w2-bc714d-openshift-default-67558b95cf-w8zj5\" (UID: \"07ed7cb3-6111-4809-8b1f-27a41ff6cafb\") " pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-w8zj5"
Oct 30 22:09:31.053227 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:31.053101    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wcj85\" (UniqueName: \"kubernetes.io/projected/07ed7cb3-6111-4809-8b1f-27a41ff6cafb-kube-api-access-wcj85\") pod \"e2e-worker-w2-bc714d-openshift-default-67558b95cf-w8zj5\" (UID: \"07ed7cb3-6111-4809-8b1f-27a41ff6cafb\") " pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-w8zj5"
Oct 30 22:09:31.053227 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:31.053209    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/07ed7cb3-6111-4809-8b1f-27a41ff6cafb-credential-socket\") pod \"e2e-worker-w2-bc714d-openshift-default-67558b95cf-w8zj5\" (UID: \"07ed7cb3-6111-4809-8b1f-27a41ff6cafb\") " pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-w8zj5"
Oct 30 22:09:31.053304 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:31.053277    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/07ed7cb3-6111-4809-8b1f-27a41ff6cafb-workload-socket\") pod \"e2e-worker-w2-bc714d-openshift-default-67558b95cf-w8zj5\" (UID: \"07ed7cb3-6111-4809-8b1f-27a41ff6cafb\") " pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-w8zj5"
Oct 30 22:09:31.053346 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:31.053316    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/07ed7cb3-6111-4809-8b1f-27a41ff6cafb-istio-data\") pod \"e2e-worker-w2-bc714d-openshift-default-67558b95cf-w8zj5\" (UID: \"07ed7cb3-6111-4809-8b1f-27a41ff6cafb\") " pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-w8zj5"
Oct 30 22:09:31.053391 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:31.053346    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/07ed7cb3-6111-4809-8b1f-27a41ff6cafb-istio-envoy\") pod \"e2e-worker-w2-bc714d-openshift-default-67558b95cf-w8zj5\" (UID: \"07ed7cb3-6111-4809-8b1f-27a41ff6cafb\") " pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-w8zj5"
Oct 30 22:09:31.154873 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:31.154777    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/07ed7cb3-6111-4809-8b1f-27a41ff6cafb-workload-socket\") pod \"e2e-worker-w2-bc714d-openshift-default-67558b95cf-w8zj5\" (UID: \"07ed7cb3-6111-4809-8b1f-27a41ff6cafb\") " pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-w8zj5"
Oct 30 22:09:31.154873 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:31.154862    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/07ed7cb3-6111-4809-8b1f-27a41ff6cafb-istio-data\") pod \"e2e-worker-w2-bc714d-openshift-default-67558b95cf-w8zj5\" (UID: \"07ed7cb3-6111-4809-8b1f-27a41ff6cafb\") " pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-w8zj5"
Oct 30 22:09:31.155113 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:31.154908    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/07ed7cb3-6111-4809-8b1f-27a41ff6cafb-istio-envoy\") pod \"e2e-worker-w2-bc714d-openshift-default-67558b95cf-w8zj5\" (UID: \"07ed7cb3-6111-4809-8b1f-27a41ff6cafb\") " pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-w8zj5"
Oct 30 22:09:31.155113 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:31.154991    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/07ed7cb3-6111-4809-8b1f-27a41ff6cafb-workload-certs\") pod \"e2e-worker-w2-bc714d-openshift-default-67558b95cf-w8zj5\" (UID: \"07ed7cb3-6111-4809-8b1f-27a41ff6cafb\") " pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-w8zj5"
Oct 30 22:09:31.155113 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:31.155100    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/07ed7cb3-6111-4809-8b1f-27a41ff6cafb-istio-token\") pod \"e2e-worker-w2-bc714d-openshift-default-67558b95cf-w8zj5\" (UID: \"07ed7cb3-6111-4809-8b1f-27a41ff6cafb\") " pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-w8zj5"
Oct 30 22:09:31.155282 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:31.155129    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/07ed7cb3-6111-4809-8b1f-27a41ff6cafb-istio-podinfo\") pod \"e2e-worker-w2-bc714d-openshift-default-67558b95cf-w8zj5\" (UID: \"07ed7cb3-6111-4809-8b1f-27a41ff6cafb\") " pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-w8zj5"
Oct 30 22:09:31.155282 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:31.155161    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/07ed7cb3-6111-4809-8b1f-27a41ff6cafb-istiod-ca-cert\") pod \"e2e-worker-w2-bc714d-openshift-default-67558b95cf-w8zj5\" (UID: \"07ed7cb3-6111-4809-8b1f-27a41ff6cafb\") " pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-w8zj5"
Oct 30 22:09:31.155646 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:31.155614    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/07ed7cb3-6111-4809-8b1f-27a41ff6cafb-workload-certs\") pod \"e2e-worker-w2-bc714d-openshift-default-67558b95cf-w8zj5\" (UID: \"07ed7cb3-6111-4809-8b1f-27a41ff6cafb\") " pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-w8zj5"
Oct 30 22:09:31.155786 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:31.155639    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/07ed7cb3-6111-4809-8b1f-27a41ff6cafb-istio-data\") pod \"e2e-worker-w2-bc714d-openshift-default-67558b95cf-w8zj5\" (UID: \"07ed7cb3-6111-4809-8b1f-27a41ff6cafb\") " pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-w8zj5"
Oct 30 22:09:31.155786 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:31.155642    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wcj85\" (UniqueName: \"kubernetes.io/projected/07ed7cb3-6111-4809-8b1f-27a41ff6cafb-kube-api-access-wcj85\") pod \"e2e-worker-w2-bc714d-openshift-default-67558b95cf-w8zj5\" (UID: \"07ed7cb3-6111-4809-8b1f-27a41ff6cafb\") " pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-w8zj5"
Oct 30 22:09:31.155786 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:31.155768    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/07ed7cb3-6111-4809-8b1f-27a41ff6cafb-credential-socket\") pod \"e2e-worker-w2-bc714d-openshift-default-67558b95cf-w8zj5\" (UID: \"07ed7cb3-6111-4809-8b1f-27a41ff6cafb\") " pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-w8zj5"
Oct 30 22:09:31.155958 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:31.155295    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/07ed7cb3-6111-4809-8b1f-27a41ff6cafb-workload-socket\") pod \"e2e-worker-w2-bc714d-openshift-default-67558b95cf-w8zj5\" (UID: \"07ed7cb3-6111-4809-8b1f-27a41ff6cafb\") " pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-w8zj5"
Oct 30 22:09:31.156042 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:31.156017    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/07ed7cb3-6111-4809-8b1f-27a41ff6cafb-istiod-ca-cert\") pod \"e2e-worker-w2-bc714d-openshift-default-67558b95cf-w8zj5\" (UID: \"07ed7cb3-6111-4809-8b1f-27a41ff6cafb\") " pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-w8zj5"
Oct 30 22:09:31.156153 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:31.156133    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/07ed7cb3-6111-4809-8b1f-27a41ff6cafb-credential-socket\") pod \"e2e-worker-w2-bc714d-openshift-default-67558b95cf-w8zj5\" (UID: \"07ed7cb3-6111-4809-8b1f-27a41ff6cafb\") " pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-w8zj5"
Oct 30 22:09:31.157489 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:31.157473    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/07ed7cb3-6111-4809-8b1f-27a41ff6cafb-istio-envoy\") pod \"e2e-worker-w2-bc714d-openshift-default-67558b95cf-w8zj5\" (UID: \"07ed7cb3-6111-4809-8b1f-27a41ff6cafb\") " pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-w8zj5"
Oct 30 22:09:31.157813 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:31.157792    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/07ed7cb3-6111-4809-8b1f-27a41ff6cafb-istio-podinfo\") pod \"e2e-worker-w2-bc714d-openshift-default-67558b95cf-w8zj5\" (UID: \"07ed7cb3-6111-4809-8b1f-27a41ff6cafb\") " pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-w8zj5"
Oct 30 22:09:31.213594 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:31.213556    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/07ed7cb3-6111-4809-8b1f-27a41ff6cafb-istio-token\") pod \"e2e-worker-w2-bc714d-openshift-default-67558b95cf-w8zj5\" (UID: \"07ed7cb3-6111-4809-8b1f-27a41ff6cafb\") " pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-w8zj5"
Oct 30 22:09:31.213771 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:31.213755    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wcj85\" (UniqueName: \"kubernetes.io/projected/07ed7cb3-6111-4809-8b1f-27a41ff6cafb-kube-api-access-wcj85\") pod \"e2e-worker-w2-bc714d-openshift-default-67558b95cf-w8zj5\" (UID: \"07ed7cb3-6111-4809-8b1f-27a41ff6cafb\") " pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-w8zj5"
Oct 30 22:09:31.219601 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:31.219577    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-w8zj5"
Oct 30 22:09:31.658331 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:31.658304    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-w8zj5"]
Oct 30 22:09:31.660467 ip-10-0-136-210 kubenswrapper[5510]: W1030 22:09:31.660423    5510 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07ed7cb3_6111_4809_8b1f_27a41ff6cafb.slice/crio-99e97160a33ea2adec740144c2730b43556389a5a6f0c711ad0e0e414d2acf51 WatchSource:0}: Error finding container 99e97160a33ea2adec740144c2730b43556389a5a6f0c711ad0e0e414d2acf51: Status 404 returned error can't find the container with id 99e97160a33ea2adec740144c2730b43556389a5a6f0c711ad0e0e414d2acf51
Oct 30 22:09:31.662940 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:31.662903    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:09:31.663066 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:31.662987    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:09:31.663066 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:31.663038    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:09:32.593526 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:32.593492    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-w8zj5" event={"ID":"07ed7cb3-6111-4809-8b1f-27a41ff6cafb","Type":"ContainerStarted","Data":"b03b2355a03dd6578e26e989fa3ba975166ed2d768c8f1ca4112d6a484404acd"}
Oct 30 22:09:32.594091 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:32.593559    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-w8zj5" event={"ID":"07ed7cb3-6111-4809-8b1f-27a41ff6cafb","Type":"ContainerStarted","Data":"99e97160a33ea2adec740144c2730b43556389a5a6f0c711ad0e0e414d2acf51"}
Oct 30 22:09:32.765681 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:32.763838    5510 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-w8zj5" podStartSLOduration=2.763818185 podStartE2EDuration="2.763818185s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:09:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:09:32.763211114 +0000 UTC m=+1153.324021121" watchObservedRunningTime="2026-10-30 22:09:32.763818185 +0000 UTC m=+1153.324628192"
Oct 30 22:09:33.220023 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:33.219983    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-w8zj5"
Oct 30 22:09:33.221571 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:33.221545    5510 patch_prober.go:29] interesting pod/e2e-worker-w2-bc714d-openshift-default-67558b95cf-w8zj5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body=
Oct 30 22:09:33.221724 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:33.221590    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-w8zj5" podUID="07ed7cb3-6111-4809-8b1f-27a41ff6cafb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused"
Oct 30 22:09:34.220068 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:34.220028    5510 patch_prober.go:29] interesting pod/e2e-worker-w2-bc714d-openshift-default-67558b95cf-w8zj5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body=
Oct 30 22:09:34.220506 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:34.220086    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-w8zj5" podUID="07ed7cb3-6111-4809-8b1f-27a41ff6cafb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused"
Oct 30 22:09:34.372681 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:34.365564    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:09:35.221067 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:35.220987    5510 patch_prober.go:29] interesting pod/e2e-worker-w2-bc714d-openshift-default-67558b95cf-w8zj5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body=
Oct 30 22:09:35.221445 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:35.221058    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-w8zj5" podUID="07ed7cb3-6111-4809-8b1f-27a41ff6cafb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused"
Oct 30 22:09:36.220460 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:36.220428    5510 patch_prober.go:29] interesting pod/e2e-worker-w2-bc714d-openshift-default-67558b95cf-w8zj5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body=
Oct 30 22:09:36.220635 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:36.220481    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-w8zj5" podUID="07ed7cb3-6111-4809-8b1f-27a41ff6cafb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused"
Oct 30 22:09:37.224487 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:37.224449    5510 patch_prober.go:29] interesting pod/e2e-worker-w2-bc714d-openshift-default-67558b95cf-w8zj5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body=
Oct 30 22:09:37.225099 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:37.225065    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-w8zj5" podUID="07ed7cb3-6111-4809-8b1f-27a41ff6cafb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused"
Oct 30 22:09:38.223689 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:38.221819    5510 patch_prober.go:29] interesting pod/e2e-worker-w2-bc714d-openshift-default-67558b95cf-w8zj5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body=
Oct 30 22:09:38.223689 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:38.221892    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-w8zj5" podUID="07ed7cb3-6111-4809-8b1f-27a41ff6cafb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused"
Oct 30 22:09:38.459932 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:09:38.459890    5510 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6gjbf" containerName="extractor"
Oct 30 22:09:39.458718 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:39.451130    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:09:39.567380 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:39.567347    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-w8zj5"
Oct 30 22:09:39.567919 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:39.567833    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-w8zj5"
Oct 30 22:09:39.568924 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:39.568901    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-w8zj5"
Oct 30 22:09:44.431483 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:44.429319    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:09:49.340296 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:49.340254    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:09:54.245441 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:54.245395    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:09:58.199043 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:58.199002    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:09:59.190692 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:09:59.174795    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:10:01.461481 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:01.461286    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:10:04.245683 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:04.242838    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:10:08.419635 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:10:08.419600    5510 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6gjbf" containerName="extractor"
Oct 30 22:10:09.153342 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:09.153302    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:10:14.168427 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:14.168374    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:10:14.432979 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:14.432897    5510 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w2-bc714d/distinct-w2-bc714d-kserve-66cfb968dd-jvgqj"]
Oct 30 22:10:14.437046 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:14.437025    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-bc714d/distinct-w2-bc714d-kserve-66cfb968dd-jvgqj"
Oct 30 22:10:14.486687 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:14.486626    5510 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w2-bc714d\"/\"openshift-service-ca.crt\""
Oct 30 22:10:14.487264 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:14.487229    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g5h9s\" (UniqueName: \"kubernetes.io/projected/9e9b11b8-1cc9-4fc8-8097-1afcffa8c422-kube-api-access-g5h9s\") pod \"distinct-w2-bc714d-kserve-66cfb968dd-jvgqj\" (UID: \"9e9b11b8-1cc9-4fc8-8097-1afcffa8c422\") " pod="e2e-models-e2e-worker-w2-bc714d/distinct-w2-bc714d-kserve-66cfb968dd-jvgqj"
Oct 30 22:10:14.487417 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:14.487287    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9e9b11b8-1cc9-4fc8-8097-1afcffa8c422-dshm\") pod \"distinct-w2-bc714d-kserve-66cfb968dd-jvgqj\" (UID: \"9e9b11b8-1cc9-4fc8-8097-1afcffa8c422\") " pod="e2e-models-e2e-worker-w2-bc714d/distinct-w2-bc714d-kserve-66cfb968dd-jvgqj"
Oct 30 22:10:14.487417 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:14.487334    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9e9b11b8-1cc9-4fc8-8097-1afcffa8c422-home\") pod \"distinct-w2-bc714d-kserve-66cfb968dd-jvgqj\" (UID: \"9e9b11b8-1cc9-4fc8-8097-1afcffa8c422\") " pod="e2e-models-e2e-worker-w2-bc714d/distinct-w2-bc714d-kserve-66cfb968dd-jvgqj"
Oct 30 22:10:14.487417 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:14.487355    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9e9b11b8-1cc9-4fc8-8097-1afcffa8c422-model-cache\") pod \"distinct-w2-bc714d-kserve-66cfb968dd-jvgqj\" (UID: \"9e9b11b8-1cc9-4fc8-8097-1afcffa8c422\") " pod="e2e-models-e2e-worker-w2-bc714d/distinct-w2-bc714d-kserve-66cfb968dd-jvgqj"
Oct 30 22:10:14.487417 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:14.487393    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9e9b11b8-1cc9-4fc8-8097-1afcffa8c422-tls-certs\") pod \"distinct-w2-bc714d-kserve-66cfb968dd-jvgqj\" (UID: \"9e9b11b8-1cc9-4fc8-8097-1afcffa8c422\") " pod="e2e-models-e2e-worker-w2-bc714d/distinct-w2-bc714d-kserve-66cfb968dd-jvgqj"
Oct 30 22:10:14.487702 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:14.487474    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9e9b11b8-1cc9-4fc8-8097-1afcffa8c422-tmp-dir\") pod \"distinct-w2-bc714d-kserve-66cfb968dd-jvgqj\" (UID: \"9e9b11b8-1cc9-4fc8-8097-1afcffa8c422\") " pod="e2e-models-e2e-worker-w2-bc714d/distinct-w2-bc714d-kserve-66cfb968dd-jvgqj"
Oct 30 22:10:14.533165 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:14.533135    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w2-bc714d\"/\"default-dockercfg-hzf47\""
Oct 30 22:10:14.533357 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:14.533190    5510 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w2-bc714d\"/\"kube-root-ca.crt\""
Oct 30 22:10:14.533412 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:14.533382    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w2-bc714d\"/\"distinct-w2-bc714d-kserve-self-signed-certs\""
Oct 30 22:10:14.536463 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:14.536441    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-bc714d/distinct-w2-bc714d-kserve-66cfb968dd-jvgqj"]
Oct 30 22:10:14.608366 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:14.607822    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9e9b11b8-1cc9-4fc8-8097-1afcffa8c422-tls-certs\") pod \"distinct-w2-bc714d-kserve-66cfb968dd-jvgqj\" (UID: \"9e9b11b8-1cc9-4fc8-8097-1afcffa8c422\") " pod="e2e-models-e2e-worker-w2-bc714d/distinct-w2-bc714d-kserve-66cfb968dd-jvgqj"
Oct 30 22:10:14.608366 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:14.607948    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9e9b11b8-1cc9-4fc8-8097-1afcffa8c422-tmp-dir\") pod \"distinct-w2-bc714d-kserve-66cfb968dd-jvgqj\" (UID: \"9e9b11b8-1cc9-4fc8-8097-1afcffa8c422\") " pod="e2e-models-e2e-worker-w2-bc714d/distinct-w2-bc714d-kserve-66cfb968dd-jvgqj"
Oct 30 22:10:14.608366 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:14.608151    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g5h9s\" (UniqueName: \"kubernetes.io/projected/9e9b11b8-1cc9-4fc8-8097-1afcffa8c422-kube-api-access-g5h9s\") pod \"distinct-w2-bc714d-kserve-66cfb968dd-jvgqj\" (UID: \"9e9b11b8-1cc9-4fc8-8097-1afcffa8c422\") " pod="e2e-models-e2e-worker-w2-bc714d/distinct-w2-bc714d-kserve-66cfb968dd-jvgqj"
Oct 30 22:10:14.608366 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:14.608200    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9e9b11b8-1cc9-4fc8-8097-1afcffa8c422-dshm\") pod \"distinct-w2-bc714d-kserve-66cfb968dd-jvgqj\" (UID: \"9e9b11b8-1cc9-4fc8-8097-1afcffa8c422\") " pod="e2e-models-e2e-worker-w2-bc714d/distinct-w2-bc714d-kserve-66cfb968dd-jvgqj"
Oct 30 22:10:14.608366 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:14.608276    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9e9b11b8-1cc9-4fc8-8097-1afcffa8c422-home\") pod \"distinct-w2-bc714d-kserve-66cfb968dd-jvgqj\" (UID: \"9e9b11b8-1cc9-4fc8-8097-1afcffa8c422\") " pod="e2e-models-e2e-worker-w2-bc714d/distinct-w2-bc714d-kserve-66cfb968dd-jvgqj"
Oct 30 22:10:14.608366 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:14.608326    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9e9b11b8-1cc9-4fc8-8097-1afcffa8c422-model-cache\") pod \"distinct-w2-bc714d-kserve-66cfb968dd-jvgqj\" (UID: \"9e9b11b8-1cc9-4fc8-8097-1afcffa8c422\") " pod="e2e-models-e2e-worker-w2-bc714d/distinct-w2-bc714d-kserve-66cfb968dd-jvgqj"
Oct 30 22:10:14.609116 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:14.609087    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9e9b11b8-1cc9-4fc8-8097-1afcffa8c422-model-cache\") pod \"distinct-w2-bc714d-kserve-66cfb968dd-jvgqj\" (UID: \"9e9b11b8-1cc9-4fc8-8097-1afcffa8c422\") " pod="e2e-models-e2e-worker-w2-bc714d/distinct-w2-bc714d-kserve-66cfb968dd-jvgqj"
Oct 30 22:10:14.623961 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:14.609411    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:10:14.625763 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:14.625675    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9e9b11b8-1cc9-4fc8-8097-1afcffa8c422-home\") pod \"distinct-w2-bc714d-kserve-66cfb968dd-jvgqj\" (UID: \"9e9b11b8-1cc9-4fc8-8097-1afcffa8c422\") " pod="e2e-models-e2e-worker-w2-bc714d/distinct-w2-bc714d-kserve-66cfb968dd-jvgqj"
Oct 30 22:10:14.625954 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:14.620772    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9e9b11b8-1cc9-4fc8-8097-1afcffa8c422-tmp-dir\") pod \"distinct-w2-bc714d-kserve-66cfb968dd-jvgqj\" (UID: \"9e9b11b8-1cc9-4fc8-8097-1afcffa8c422\") " pod="e2e-models-e2e-worker-w2-bc714d/distinct-w2-bc714d-kserve-66cfb968dd-jvgqj"
Oct 30 22:10:14.637530 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:14.632022    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9e9b11b8-1cc9-4fc8-8097-1afcffa8c422-dshm\") pod \"distinct-w2-bc714d-kserve-66cfb968dd-jvgqj\" (UID: \"9e9b11b8-1cc9-4fc8-8097-1afcffa8c422\") " pod="e2e-models-e2e-worker-w2-bc714d/distinct-w2-bc714d-kserve-66cfb968dd-jvgqj"
Oct 30 22:10:14.637530 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:14.633294    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9e9b11b8-1cc9-4fc8-8097-1afcffa8c422-tls-certs\") pod \"distinct-w2-bc714d-kserve-66cfb968dd-jvgqj\" (UID: \"9e9b11b8-1cc9-4fc8-8097-1afcffa8c422\") " pod="e2e-models-e2e-worker-w2-bc714d/distinct-w2-bc714d-kserve-66cfb968dd-jvgqj"
Oct 30 22:10:14.683169 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:14.683097    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g5h9s\" (UniqueName: \"kubernetes.io/projected/9e9b11b8-1cc9-4fc8-8097-1afcffa8c422-kube-api-access-g5h9s\") pod \"distinct-w2-bc714d-kserve-66cfb968dd-jvgqj\" (UID: \"9e9b11b8-1cc9-4fc8-8097-1afcffa8c422\") " pod="e2e-models-e2e-worker-w2-bc714d/distinct-w2-bc714d-kserve-66cfb968dd-jvgqj"
Oct 30 22:10:14.749693 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:14.749622    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-bc714d/distinct-w2-bc714d-kserve-66cfb968dd-jvgqj"
Oct 30 22:10:15.225698 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:15.225676    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-bc714d/distinct-w2-bc714d-kserve-66cfb968dd-jvgqj"]
Oct 30 22:10:15.228116 ip-10-0-136-210 kubenswrapper[5510]: W1030 22:10:15.228084    5510 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9e9b11b8_1cc9_4fc8_8097_1afcffa8c422.slice/crio-cddc7762e6edd87a2f3cbf01401782f9d43711dfe340c1d7e34b311acc293fb6 WatchSource:0}: Error finding container cddc7762e6edd87a2f3cbf01401782f9d43711dfe340c1d7e34b311acc293fb6: Status 404 returned error can't find the container with id cddc7762e6edd87a2f3cbf01401782f9d43711dfe340c1d7e34b311acc293fb6
Oct 30 22:10:15.914963 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:15.914930    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-bc714d/distinct-w2-bc714d-kserve-66cfb968dd-jvgqj" event={"ID":"9e9b11b8-1cc9-4fc8-8097-1afcffa8c422","Type":"ContainerStarted","Data":"f70ce9298fd0c8b101c2375c907e006433e5cd428a64b0ac937ffc099299bf84"}
Oct 30 22:10:15.915154 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:15.914974    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-bc714d/distinct-w2-bc714d-kserve-66cfb968dd-jvgqj" event={"ID":"9e9b11b8-1cc9-4fc8-8097-1afcffa8c422","Type":"ContainerStarted","Data":"cddc7762e6edd87a2f3cbf01401782f9d43711dfe340c1d7e34b311acc293fb6"}
Oct 30 22:10:16.026470 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:16.026371    5510 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w2-bc714d/distinct-w2-bc714d-kserve-66cfb968dd-jvgqj" podStartSLOduration=2.026356522 podStartE2EDuration="2.026356522s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:10:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:10:16.025816552 +0000 UTC m=+1196.586626574" watchObservedRunningTime="2026-10-30 22:10:16.026356522 +0000 UTC m=+1196.587166526"
Oct 30 22:10:16.750380 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:16.750344    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-bc714d/distinct-w2-bc714d-kserve-66cfb968dd-jvgqj"
Oct 30 22:10:19.129813 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:19.129768    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:10:20.259495 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:20.259450    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:10:24.157189 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:24.157149    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:10:24.461235 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:24.461143    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:10:24.750865 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:24.750778    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w2-bc714d/distinct-w2-bc714d-kserve-66cfb968dd-jvgqj"
Oct 30 22:10:24.763021 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:24.762997    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w2-bc714d/distinct-w2-bc714d-kserve-66cfb968dd-jvgqj"
Oct 30 22:10:24.987714 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:24.987683    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-bc714d/distinct-w2-bc714d-kserve-66cfb968dd-jvgqj"
Oct 30 22:10:27.299612 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:27.299577    5510 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-779f32/distinct-w0-779f32-kserve-6dcbc948c7-2vzdf"]
Oct 30 22:10:27.304001 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:27.303979    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-779f32/distinct-w0-779f32-kserve-6dcbc948c7-2vzdf"
Oct 30 22:10:27.333483 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:27.333443    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e17cedd4-7d48-42b3-b127-b6192c1b3795-dshm\") pod \"distinct-w0-779f32-kserve-6dcbc948c7-2vzdf\" (UID: \"e17cedd4-7d48-42b3-b127-b6192c1b3795\") " pod="e2e-models-e2e-worker-w0-779f32/distinct-w0-779f32-kserve-6dcbc948c7-2vzdf"
Oct 30 22:10:27.333635 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:27.333503    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e17cedd4-7d48-42b3-b127-b6192c1b3795-tls-certs\") pod \"distinct-w0-779f32-kserve-6dcbc948c7-2vzdf\" (UID: \"e17cedd4-7d48-42b3-b127-b6192c1b3795\") " pod="e2e-models-e2e-worker-w0-779f32/distinct-w0-779f32-kserve-6dcbc948c7-2vzdf"
Oct 30 22:10:27.333635 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:27.333594    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e17cedd4-7d48-42b3-b127-b6192c1b3795-tmp-dir\") pod \"distinct-w0-779f32-kserve-6dcbc948c7-2vzdf\" (UID: \"e17cedd4-7d48-42b3-b127-b6192c1b3795\") " pod="e2e-models-e2e-worker-w0-779f32/distinct-w0-779f32-kserve-6dcbc948c7-2vzdf"
Oct 30 22:10:27.333635 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:27.333620    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6rmpw\" (UniqueName: \"kubernetes.io/projected/e17cedd4-7d48-42b3-b127-b6192c1b3795-kube-api-access-6rmpw\") pod \"distinct-w0-779f32-kserve-6dcbc948c7-2vzdf\" (UID: \"e17cedd4-7d48-42b3-b127-b6192c1b3795\") " pod="e2e-models-e2e-worker-w0-779f32/distinct-w0-779f32-kserve-6dcbc948c7-2vzdf"
Oct 30 22:10:27.333895 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:27.333800    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e17cedd4-7d48-42b3-b127-b6192c1b3795-model-cache\") pod \"distinct-w0-779f32-kserve-6dcbc948c7-2vzdf\" (UID: \"e17cedd4-7d48-42b3-b127-b6192c1b3795\") " pod="e2e-models-e2e-worker-w0-779f32/distinct-w0-779f32-kserve-6dcbc948c7-2vzdf"
Oct 30 22:10:27.333895 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:27.333863    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e17cedd4-7d48-42b3-b127-b6192c1b3795-home\") pod \"distinct-w0-779f32-kserve-6dcbc948c7-2vzdf\" (UID: \"e17cedd4-7d48-42b3-b127-b6192c1b3795\") " pod="e2e-models-e2e-worker-w0-779f32/distinct-w0-779f32-kserve-6dcbc948c7-2vzdf"
Oct 30 22:10:27.400717 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:27.400683    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-779f32\"/\"distinct-w0-779f32-kserve-self-signed-certs\""
Oct 30 22:10:27.400717 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:27.400685    5510 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-779f32\"/\"openshift-service-ca.crt\""
Oct 30 22:10:27.400717 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:27.400684    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-779f32\"/\"default-dockercfg-lcntz\""
Oct 30 22:10:27.401018 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:27.400757    5510 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-779f32\"/\"kube-root-ca.crt\""
Oct 30 22:10:27.403628 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:27.403608    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-779f32/distinct-w0-779f32-kserve-6dcbc948c7-2vzdf"]
Oct 30 22:10:27.431101 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:27.431066    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:10:27.458307 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:27.435319    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e17cedd4-7d48-42b3-b127-b6192c1b3795-model-cache\") pod \"distinct-w0-779f32-kserve-6dcbc948c7-2vzdf\" (UID: \"e17cedd4-7d48-42b3-b127-b6192c1b3795\") " pod="e2e-models-e2e-worker-w0-779f32/distinct-w0-779f32-kserve-6dcbc948c7-2vzdf"
Oct 30 22:10:27.458307 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:27.435410    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e17cedd4-7d48-42b3-b127-b6192c1b3795-home\") pod \"distinct-w0-779f32-kserve-6dcbc948c7-2vzdf\" (UID: \"e17cedd4-7d48-42b3-b127-b6192c1b3795\") " pod="e2e-models-e2e-worker-w0-779f32/distinct-w0-779f32-kserve-6dcbc948c7-2vzdf"
Oct 30 22:10:27.458307 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:27.435491    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e17cedd4-7d48-42b3-b127-b6192c1b3795-dshm\") pod \"distinct-w0-779f32-kserve-6dcbc948c7-2vzdf\" (UID: \"e17cedd4-7d48-42b3-b127-b6192c1b3795\") " pod="e2e-models-e2e-worker-w0-779f32/distinct-w0-779f32-kserve-6dcbc948c7-2vzdf"
Oct 30 22:10:27.458307 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:27.435533    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e17cedd4-7d48-42b3-b127-b6192c1b3795-tls-certs\") pod \"distinct-w0-779f32-kserve-6dcbc948c7-2vzdf\" (UID: \"e17cedd4-7d48-42b3-b127-b6192c1b3795\") " pod="e2e-models-e2e-worker-w0-779f32/distinct-w0-779f32-kserve-6dcbc948c7-2vzdf"
Oct 30 22:10:27.458307 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:27.435575    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e17cedd4-7d48-42b3-b127-b6192c1b3795-tmp-dir\") pod \"distinct-w0-779f32-kserve-6dcbc948c7-2vzdf\" (UID: \"e17cedd4-7d48-42b3-b127-b6192c1b3795\") " pod="e2e-models-e2e-worker-w0-779f32/distinct-w0-779f32-kserve-6dcbc948c7-2vzdf"
Oct 30 22:10:27.458307 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:27.435608    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6rmpw\" (UniqueName: \"kubernetes.io/projected/e17cedd4-7d48-42b3-b127-b6192c1b3795-kube-api-access-6rmpw\") pod \"distinct-w0-779f32-kserve-6dcbc948c7-2vzdf\" (UID: \"e17cedd4-7d48-42b3-b127-b6192c1b3795\") " pod="e2e-models-e2e-worker-w0-779f32/distinct-w0-779f32-kserve-6dcbc948c7-2vzdf"
Oct 30 22:10:27.458307 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:27.436899    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e17cedd4-7d48-42b3-b127-b6192c1b3795-model-cache\") pod \"distinct-w0-779f32-kserve-6dcbc948c7-2vzdf\" (UID: \"e17cedd4-7d48-42b3-b127-b6192c1b3795\") " pod="e2e-models-e2e-worker-w0-779f32/distinct-w0-779f32-kserve-6dcbc948c7-2vzdf"
Oct 30 22:10:27.458307 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:27.437469    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e17cedd4-7d48-42b3-b127-b6192c1b3795-home\") pod \"distinct-w0-779f32-kserve-6dcbc948c7-2vzdf\" (UID: \"e17cedd4-7d48-42b3-b127-b6192c1b3795\") " pod="e2e-models-e2e-worker-w0-779f32/distinct-w0-779f32-kserve-6dcbc948c7-2vzdf"
Oct 30 22:10:27.458307 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:27.437715    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e17cedd4-7d48-42b3-b127-b6192c1b3795-tmp-dir\") pod \"distinct-w0-779f32-kserve-6dcbc948c7-2vzdf\" (UID: \"e17cedd4-7d48-42b3-b127-b6192c1b3795\") " pod="e2e-models-e2e-worker-w0-779f32/distinct-w0-779f32-kserve-6dcbc948c7-2vzdf"
Oct 30 22:10:27.467380 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:27.466763    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e17cedd4-7d48-42b3-b127-b6192c1b3795-tls-certs\") pod \"distinct-w0-779f32-kserve-6dcbc948c7-2vzdf\" (UID: \"e17cedd4-7d48-42b3-b127-b6192c1b3795\") " pod="e2e-models-e2e-worker-w0-779f32/distinct-w0-779f32-kserve-6dcbc948c7-2vzdf"
Oct 30 22:10:27.468282 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:27.468226    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e17cedd4-7d48-42b3-b127-b6192c1b3795-dshm\") pod \"distinct-w0-779f32-kserve-6dcbc948c7-2vzdf\" (UID: \"e17cedd4-7d48-42b3-b127-b6192c1b3795\") " pod="e2e-models-e2e-worker-w0-779f32/distinct-w0-779f32-kserve-6dcbc948c7-2vzdf"
Oct 30 22:10:27.501908 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:27.501842    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6rmpw\" (UniqueName: \"kubernetes.io/projected/e17cedd4-7d48-42b3-b127-b6192c1b3795-kube-api-access-6rmpw\") pod \"distinct-w0-779f32-kserve-6dcbc948c7-2vzdf\" (UID: \"e17cedd4-7d48-42b3-b127-b6192c1b3795\") " pod="e2e-models-e2e-worker-w0-779f32/distinct-w0-779f32-kserve-6dcbc948c7-2vzdf"
Oct 30 22:10:27.615940 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:27.615851    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-779f32/distinct-w0-779f32-kserve-6dcbc948c7-2vzdf"
Oct 30 22:10:28.087047 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:28.087022    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-779f32/distinct-w0-779f32-kserve-6dcbc948c7-2vzdf"]
Oct 30 22:10:28.089268 ip-10-0-136-210 kubenswrapper[5510]: W1030 22:10:28.089241    5510 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode17cedd4_7d48_42b3_b127_b6192c1b3795.slice/crio-58a9e3cfb38d27bbd767479bde3692688592ed2c5076102fd005a3bfabd2ae9c WatchSource:0}: Error finding container 58a9e3cfb38d27bbd767479bde3692688592ed2c5076102fd005a3bfabd2ae9c: Status 404 returned error can't find the container with id 58a9e3cfb38d27bbd767479bde3692688592ed2c5076102fd005a3bfabd2ae9c
Oct 30 22:10:28.999795 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:28.999754    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-779f32/distinct-w0-779f32-kserve-6dcbc948c7-2vzdf" event={"ID":"e17cedd4-7d48-42b3-b127-b6192c1b3795","Type":"ContainerStarted","Data":"f7de5feaffd118764ebef90ea7f126c3c8b7974d6a32da832873599fffae0343"}
Oct 30 22:10:29.000485 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:28.999806    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-779f32/distinct-w0-779f32-kserve-6dcbc948c7-2vzdf" event={"ID":"e17cedd4-7d48-42b3-b127-b6192c1b3795","Type":"ContainerStarted","Data":"58a9e3cfb38d27bbd767479bde3692688592ed2c5076102fd005a3bfabd2ae9c"}
Oct 30 22:10:29.113181 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:29.113127    5510 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-779f32/distinct-w0-779f32-kserve-6dcbc948c7-2vzdf" podStartSLOduration=2.113114124 podStartE2EDuration="2.113114124s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:10:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:10:29.110763604 +0000 UTC m=+1209.671573609" watchObservedRunningTime="2026-10-30 22:10:29.113114124 +0000 UTC m=+1209.673924497"
Oct 30 22:10:29.158686 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:29.151094    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:10:29.616305 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:29.616261    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-779f32/distinct-w0-779f32-kserve-6dcbc948c7-2vzdf"
Oct 30 22:10:31.739335 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:31.739293    5510 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-n6tv4"]
Oct 30 22:10:31.743954 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:31.743927    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-n6tv4"
Oct 30 22:10:31.804996 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:31.804961    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lq4gs\" (UniqueName: \"kubernetes.io/projected/e15eace0-bc5d-43ec-b352-74536b28a56c-kube-api-access-lq4gs\") pod \"e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-n6tv4\" (UID: \"e15eace0-bc5d-43ec-b352-74536b28a56c\") " pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-n6tv4"
Oct 30 22:10:31.804996 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:31.804998    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e15eace0-bc5d-43ec-b352-74536b28a56c-istio-data\") pod \"e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-n6tv4\" (UID: \"e15eace0-bc5d-43ec-b352-74536b28a56c\") " pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-n6tv4"
Oct 30 22:10:31.805224 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:31.805018    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e15eace0-bc5d-43ec-b352-74536b28a56c-istio-token\") pod \"e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-n6tv4\" (UID: \"e15eace0-bc5d-43ec-b352-74536b28a56c\") " pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-n6tv4"
Oct 30 22:10:31.805224 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:31.805147    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e15eace0-bc5d-43ec-b352-74536b28a56c-istio-podinfo\") pod \"e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-n6tv4\" (UID: \"e15eace0-bc5d-43ec-b352-74536b28a56c\") " pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-n6tv4"
Oct 30 22:10:31.805224 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:31.805196    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e15eace0-bc5d-43ec-b352-74536b28a56c-istio-envoy\") pod \"e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-n6tv4\" (UID: \"e15eace0-bc5d-43ec-b352-74536b28a56c\") " pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-n6tv4"
Oct 30 22:10:31.805341 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:31.805255    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e15eace0-bc5d-43ec-b352-74536b28a56c-istiod-ca-cert\") pod \"e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-n6tv4\" (UID: \"e15eace0-bc5d-43ec-b352-74536b28a56c\") " pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-n6tv4"
Oct 30 22:10:31.805381 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:31.805357    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e15eace0-bc5d-43ec-b352-74536b28a56c-workload-certs\") pod \"e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-n6tv4\" (UID: \"e15eace0-bc5d-43ec-b352-74536b28a56c\") " pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-n6tv4"
Oct 30 22:10:31.805433 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:31.805417    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e15eace0-bc5d-43ec-b352-74536b28a56c-workload-socket\") pod \"e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-n6tv4\" (UID: \"e15eace0-bc5d-43ec-b352-74536b28a56c\") " pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-n6tv4"
Oct 30 22:10:31.805516 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:31.805502    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e15eace0-bc5d-43ec-b352-74536b28a56c-credential-socket\") pod \"e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-n6tv4\" (UID: \"e15eace0-bc5d-43ec-b352-74536b28a56c\") " pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-n6tv4"
Oct 30 22:10:31.906509 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:31.906470    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lq4gs\" (UniqueName: \"kubernetes.io/projected/e15eace0-bc5d-43ec-b352-74536b28a56c-kube-api-access-lq4gs\") pod \"e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-n6tv4\" (UID: \"e15eace0-bc5d-43ec-b352-74536b28a56c\") " pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-n6tv4"
Oct 30 22:10:31.906509 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:31.906509    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e15eace0-bc5d-43ec-b352-74536b28a56c-istio-data\") pod \"e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-n6tv4\" (UID: \"e15eace0-bc5d-43ec-b352-74536b28a56c\") " pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-n6tv4"
Oct 30 22:10:31.906806 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:31.906526    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e15eace0-bc5d-43ec-b352-74536b28a56c-istio-token\") pod \"e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-n6tv4\" (UID: \"e15eace0-bc5d-43ec-b352-74536b28a56c\") " pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-n6tv4"
Oct 30 22:10:31.906806 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:31.906651    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e15eace0-bc5d-43ec-b352-74536b28a56c-istio-podinfo\") pod \"e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-n6tv4\" (UID: \"e15eace0-bc5d-43ec-b352-74536b28a56c\") " pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-n6tv4"
Oct 30 22:10:31.906806 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:31.906755    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e15eace0-bc5d-43ec-b352-74536b28a56c-istio-envoy\") pod \"e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-n6tv4\" (UID: \"e15eace0-bc5d-43ec-b352-74536b28a56c\") " pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-n6tv4"
Oct 30 22:10:31.906806 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:31.906791    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e15eace0-bc5d-43ec-b352-74536b28a56c-istiod-ca-cert\") pod \"e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-n6tv4\" (UID: \"e15eace0-bc5d-43ec-b352-74536b28a56c\") " pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-n6tv4"
Oct 30 22:10:31.907055 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:31.906868    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e15eace0-bc5d-43ec-b352-74536b28a56c-workload-certs\") pod \"e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-n6tv4\" (UID: \"e15eace0-bc5d-43ec-b352-74536b28a56c\") " pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-n6tv4"
Oct 30 22:10:31.907055 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:31.906916    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e15eace0-bc5d-43ec-b352-74536b28a56c-istio-data\") pod \"e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-n6tv4\" (UID: \"e15eace0-bc5d-43ec-b352-74536b28a56c\") " pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-n6tv4"
Oct 30 22:10:31.907055 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:31.906956    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e15eace0-bc5d-43ec-b352-74536b28a56c-workload-socket\") pod \"e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-n6tv4\" (UID: \"e15eace0-bc5d-43ec-b352-74536b28a56c\") " pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-n6tv4"
Oct 30 22:10:31.907211 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:31.907067    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e15eace0-bc5d-43ec-b352-74536b28a56c-credential-socket\") pod \"e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-n6tv4\" (UID: \"e15eace0-bc5d-43ec-b352-74536b28a56c\") " pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-n6tv4"
Oct 30 22:10:31.907211 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:31.907192    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e15eace0-bc5d-43ec-b352-74536b28a56c-workload-certs\") pod \"e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-n6tv4\" (UID: \"e15eace0-bc5d-43ec-b352-74536b28a56c\") " pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-n6tv4"
Oct 30 22:10:31.907438 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:31.907410    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e15eace0-bc5d-43ec-b352-74536b28a56c-workload-socket\") pod \"e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-n6tv4\" (UID: \"e15eace0-bc5d-43ec-b352-74536b28a56c\") " pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-n6tv4"
Oct 30 22:10:31.907524 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:31.907469    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e15eace0-bc5d-43ec-b352-74536b28a56c-credential-socket\") pod \"e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-n6tv4\" (UID: \"e15eace0-bc5d-43ec-b352-74536b28a56c\") " pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-n6tv4"
Oct 30 22:10:31.907583 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:31.907547    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e15eace0-bc5d-43ec-b352-74536b28a56c-istiod-ca-cert\") pod \"e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-n6tv4\" (UID: \"e15eace0-bc5d-43ec-b352-74536b28a56c\") " pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-n6tv4"
Oct 30 22:10:31.909644 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:31.909621    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e15eace0-bc5d-43ec-b352-74536b28a56c-istio-envoy\") pod \"e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-n6tv4\" (UID: \"e15eace0-bc5d-43ec-b352-74536b28a56c\") " pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-n6tv4"
Oct 30 22:10:31.909892 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:31.909873    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e15eace0-bc5d-43ec-b352-74536b28a56c-istio-podinfo\") pod \"e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-n6tv4\" (UID: \"e15eace0-bc5d-43ec-b352-74536b28a56c\") " pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-n6tv4"
Oct 30 22:10:31.950012 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:31.949800    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-n6tv4"]
Oct 30 22:10:31.970195 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:31.970159    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e15eace0-bc5d-43ec-b352-74536b28a56c-istio-token\") pod \"e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-n6tv4\" (UID: \"e15eace0-bc5d-43ec-b352-74536b28a56c\") " pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-n6tv4"
Oct 30 22:10:31.972398 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:31.972367    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lq4gs\" (UniqueName: \"kubernetes.io/projected/e15eace0-bc5d-43ec-b352-74536b28a56c-kube-api-access-lq4gs\") pod \"e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-n6tv4\" (UID: \"e15eace0-bc5d-43ec-b352-74536b28a56c\") " pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-n6tv4"
Oct 30 22:10:32.059573 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:32.059539    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-n6tv4"
Oct 30 22:10:32.479085 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:32.478984    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-n6tv4"]
Oct 30 22:10:32.481916 ip-10-0-136-210 kubenswrapper[5510]: W1030 22:10:32.481886    5510 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode15eace0_bc5d_43ec_b352_74536b28a56c.slice/crio-874ae3914468dfb0d1ac37c819397d91444ab864245bba5659fd8761abd9d873 WatchSource:0}: Error finding container 874ae3914468dfb0d1ac37c819397d91444ab864245bba5659fd8761abd9d873: Status 404 returned error can't find the container with id 874ae3914468dfb0d1ac37c819397d91444ab864245bba5659fd8761abd9d873
Oct 30 22:10:32.484427 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:32.484391    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:10:32.484511 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:32.484464    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:10:32.484511 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:32.484492    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:10:33.022544 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:33.022500    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-n6tv4" event={"ID":"e15eace0-bc5d-43ec-b352-74536b28a56c","Type":"ContainerStarted","Data":"3e27cd938544e7aa190cb973c1a2b4a48728b899be1b90d874d262abdd407a01"}
Oct 30 22:10:33.022987 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:33.022560    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-n6tv4" event={"ID":"e15eace0-bc5d-43ec-b352-74536b28a56c","Type":"ContainerStarted","Data":"874ae3914468dfb0d1ac37c819397d91444ab864245bba5659fd8761abd9d873"}
Oct 30 22:10:33.060254 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:33.060220    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-n6tv4"
Oct 30 22:10:33.061668 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:33.061627    5510 patch_prober.go:29] interesting pod/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-n6tv4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body=
Oct 30 22:10:33.061799 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:33.061695    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-n6tv4" podUID="e15eace0-bc5d-43ec-b352-74536b28a56c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused"
Oct 30 22:10:34.078685 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:34.074956    5510 patch_prober.go:29] interesting pod/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-n6tv4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body=
Oct 30 22:10:34.078685 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:34.075050    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-n6tv4" podUID="e15eace0-bc5d-43ec-b352-74536b28a56c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused"
Oct 30 22:10:34.078685 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:34.077513    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:10:35.064676 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:35.064257    5510 patch_prober.go:29] interesting pod/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-n6tv4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body=
Oct 30 22:10:35.064676 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:35.064341    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-n6tv4" podUID="e15eace0-bc5d-43ec-b352-74536b28a56c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused"
Oct 30 22:10:36.065681 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:36.065121    5510 patch_prober.go:29] interesting pod/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-n6tv4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body=
Oct 30 22:10:36.065681 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:36.065205    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-n6tv4" podUID="e15eace0-bc5d-43ec-b352-74536b28a56c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused"
Oct 30 22:10:37.065681 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:37.065087    5510 patch_prober.go:29] interesting pod/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-n6tv4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body=
Oct 30 22:10:37.065681 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:37.065152    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-n6tv4" podUID="e15eace0-bc5d-43ec-b352-74536b28a56c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused"
Oct 30 22:10:37.616531 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:37.616496    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-779f32/distinct-w0-779f32-kserve-6dcbc948c7-2vzdf"
Oct 30 22:10:37.629456 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:37.629428    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-779f32/distinct-w0-779f32-kserve-6dcbc948c7-2vzdf"
Oct 30 22:10:37.769312 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:37.769254    5510 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-n6tv4" podStartSLOduration=6.769234318 podStartE2EDuration="6.769234318s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:10:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:10:33.218904168 +0000 UTC m=+1213.779714172" watchObservedRunningTime="2026-10-30 22:10:37.769234318 +0000 UTC m=+1218.330044322"
Oct 30 22:10:38.063224 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:38.063184    5510 patch_prober.go:29] interesting pod/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-n6tv4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body=
Oct 30 22:10:38.063512 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:38.063483    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-n6tv4" podUID="e15eace0-bc5d-43ec-b352-74536b28a56c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused"
Oct 30 22:10:38.122886 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:38.122851    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-779f32/distinct-w0-779f32-kserve-6dcbc948c7-2vzdf"
Oct 30 22:10:38.522025 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:38.521976    5510 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w5-c9dbe6/distinct-w5-c9dbe6-kserve-7599fd86d9-bdhxd"]
Oct 30 22:10:38.530722 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:10:38.530100    5510 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6gjbf" containerName="extractor"
Oct 30 22:10:38.530922 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:38.530903    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-c9dbe6/distinct-w5-c9dbe6-kserve-7599fd86d9-bdhxd"
Oct 30 22:10:38.597695 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:38.597486    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w5-c9dbe6\"/\"default-dockercfg-fhdlt\""
Oct 30 22:10:38.606695 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:38.598351    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w5-c9dbe6\"/\"distinct-w5-c9dbe6-kserve-self-signed-certs\""
Oct 30 22:10:38.606695 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:38.598526    5510 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w5-c9dbe6\"/\"openshift-service-ca.crt\""
Oct 30 22:10:38.606695 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:38.598629    5510 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w5-c9dbe6\"/\"kube-root-ca.crt\""
Oct 30 22:10:38.627125 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:38.626865    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:10:38.628103 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:38.628073    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vs5hl\" (UniqueName: \"kubernetes.io/projected/e903db90-f2cd-4244-b00b-f99b21873f6e-kube-api-access-vs5hl\") pod \"distinct-w5-c9dbe6-kserve-7599fd86d9-bdhxd\" (UID: \"e903db90-f2cd-4244-b00b-f99b21873f6e\") " pod="e2e-models-e2e-worker-w5-c9dbe6/distinct-w5-c9dbe6-kserve-7599fd86d9-bdhxd"
Oct 30 22:10:38.628388 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:38.628365    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e903db90-f2cd-4244-b00b-f99b21873f6e-tmp-dir\") pod \"distinct-w5-c9dbe6-kserve-7599fd86d9-bdhxd\" (UID: \"e903db90-f2cd-4244-b00b-f99b21873f6e\") " pod="e2e-models-e2e-worker-w5-c9dbe6/distinct-w5-c9dbe6-kserve-7599fd86d9-bdhxd"
Oct 30 22:10:38.628574 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:38.628545    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e903db90-f2cd-4244-b00b-f99b21873f6e-model-cache\") pod \"distinct-w5-c9dbe6-kserve-7599fd86d9-bdhxd\" (UID: \"e903db90-f2cd-4244-b00b-f99b21873f6e\") " pod="e2e-models-e2e-worker-w5-c9dbe6/distinct-w5-c9dbe6-kserve-7599fd86d9-bdhxd"
Oct 30 22:10:38.628959 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:38.628939    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e903db90-f2cd-4244-b00b-f99b21873f6e-home\") pod \"distinct-w5-c9dbe6-kserve-7599fd86d9-bdhxd\" (UID: \"e903db90-f2cd-4244-b00b-f99b21873f6e\") " pod="e2e-models-e2e-worker-w5-c9dbe6/distinct-w5-c9dbe6-kserve-7599fd86d9-bdhxd"
Oct 30 22:10:38.629108 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:38.629092    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e903db90-f2cd-4244-b00b-f99b21873f6e-dshm\") pod \"distinct-w5-c9dbe6-kserve-7599fd86d9-bdhxd\" (UID: \"e903db90-f2cd-4244-b00b-f99b21873f6e\") " pod="e2e-models-e2e-worker-w5-c9dbe6/distinct-w5-c9dbe6-kserve-7599fd86d9-bdhxd"
Oct 30 22:10:38.629248 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:38.629231    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e903db90-f2cd-4244-b00b-f99b21873f6e-tls-certs\") pod \"distinct-w5-c9dbe6-kserve-7599fd86d9-bdhxd\" (UID: \"e903db90-f2cd-4244-b00b-f99b21873f6e\") " pod="e2e-models-e2e-worker-w5-c9dbe6/distinct-w5-c9dbe6-kserve-7599fd86d9-bdhxd"
Oct 30 22:10:38.914738 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:38.889332    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vs5hl\" (UniqueName: \"kubernetes.io/projected/e903db90-f2cd-4244-b00b-f99b21873f6e-kube-api-access-vs5hl\") pod \"distinct-w5-c9dbe6-kserve-7599fd86d9-bdhxd\" (UID: \"e903db90-f2cd-4244-b00b-f99b21873f6e\") " pod="e2e-models-e2e-worker-w5-c9dbe6/distinct-w5-c9dbe6-kserve-7599fd86d9-bdhxd"
Oct 30 22:10:38.920887 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:38.920850    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e903db90-f2cd-4244-b00b-f99b21873f6e-tmp-dir\") pod \"distinct-w5-c9dbe6-kserve-7599fd86d9-bdhxd\" (UID: \"e903db90-f2cd-4244-b00b-f99b21873f6e\") " pod="e2e-models-e2e-worker-w5-c9dbe6/distinct-w5-c9dbe6-kserve-7599fd86d9-bdhxd"
Oct 30 22:10:38.921219 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:38.921200    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e903db90-f2cd-4244-b00b-f99b21873f6e-model-cache\") pod \"distinct-w5-c9dbe6-kserve-7599fd86d9-bdhxd\" (UID: \"e903db90-f2cd-4244-b00b-f99b21873f6e\") " pod="e2e-models-e2e-worker-w5-c9dbe6/distinct-w5-c9dbe6-kserve-7599fd86d9-bdhxd"
Oct 30 22:10:38.923290 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:38.923246    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e903db90-f2cd-4244-b00b-f99b21873f6e-home\") pod \"distinct-w5-c9dbe6-kserve-7599fd86d9-bdhxd\" (UID: \"e903db90-f2cd-4244-b00b-f99b21873f6e\") " pod="e2e-models-e2e-worker-w5-c9dbe6/distinct-w5-c9dbe6-kserve-7599fd86d9-bdhxd"
Oct 30 22:10:38.931198 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:38.923484    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e903db90-f2cd-4244-b00b-f99b21873f6e-dshm\") pod \"distinct-w5-c9dbe6-kserve-7599fd86d9-bdhxd\" (UID: \"e903db90-f2cd-4244-b00b-f99b21873f6e\") " pod="e2e-models-e2e-worker-w5-c9dbe6/distinct-w5-c9dbe6-kserve-7599fd86d9-bdhxd"
Oct 30 22:10:38.931461 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:38.931435    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e903db90-f2cd-4244-b00b-f99b21873f6e-tls-certs\") pod \"distinct-w5-c9dbe6-kserve-7599fd86d9-bdhxd\" (UID: \"e903db90-f2cd-4244-b00b-f99b21873f6e\") " pod="e2e-models-e2e-worker-w5-c9dbe6/distinct-w5-c9dbe6-kserve-7599fd86d9-bdhxd"
Oct 30 22:10:38.933691 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:38.933647    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e903db90-f2cd-4244-b00b-f99b21873f6e-tmp-dir\") pod \"distinct-w5-c9dbe6-kserve-7599fd86d9-bdhxd\" (UID: \"e903db90-f2cd-4244-b00b-f99b21873f6e\") " pod="e2e-models-e2e-worker-w5-c9dbe6/distinct-w5-c9dbe6-kserve-7599fd86d9-bdhxd"
Oct 30 22:10:38.935803 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:38.934914    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e903db90-f2cd-4244-b00b-f99b21873f6e-home\") pod \"distinct-w5-c9dbe6-kserve-7599fd86d9-bdhxd\" (UID: \"e903db90-f2cd-4244-b00b-f99b21873f6e\") " pod="e2e-models-e2e-worker-w5-c9dbe6/distinct-w5-c9dbe6-kserve-7599fd86d9-bdhxd"
Oct 30 22:10:38.936016 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:38.935229    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e903db90-f2cd-4244-b00b-f99b21873f6e-model-cache\") pod \"distinct-w5-c9dbe6-kserve-7599fd86d9-bdhxd\" (UID: \"e903db90-f2cd-4244-b00b-f99b21873f6e\") " pod="e2e-models-e2e-worker-w5-c9dbe6/distinct-w5-c9dbe6-kserve-7599fd86d9-bdhxd"
Oct 30 22:10:38.940179 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:38.940150    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w5-c9dbe6/distinct-w5-c9dbe6-kserve-7599fd86d9-bdhxd"]
Oct 30 22:10:39.108412 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:39.097158    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e903db90-f2cd-4244-b00b-f99b21873f6e-dshm\") pod \"distinct-w5-c9dbe6-kserve-7599fd86d9-bdhxd\" (UID: \"e903db90-f2cd-4244-b00b-f99b21873f6e\") " pod="e2e-models-e2e-worker-w5-c9dbe6/distinct-w5-c9dbe6-kserve-7599fd86d9-bdhxd"
Oct 30 22:10:39.134686 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:39.126611    5510 patch_prober.go:29] interesting pod/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-n6tv4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body=
Oct 30 22:10:39.134686 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:39.126713    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-n6tv4" podUID="e15eace0-bc5d-43ec-b352-74536b28a56c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused"
Oct 30 22:10:39.134686 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:39.127538    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e903db90-f2cd-4244-b00b-f99b21873f6e-tls-certs\") pod \"distinct-w5-c9dbe6-kserve-7599fd86d9-bdhxd\" (UID: \"e903db90-f2cd-4244-b00b-f99b21873f6e\") " pod="e2e-models-e2e-worker-w5-c9dbe6/distinct-w5-c9dbe6-kserve-7599fd86d9-bdhxd"
Oct 30 22:10:39.163053 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:39.161340    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vs5hl\" (UniqueName: \"kubernetes.io/projected/e903db90-f2cd-4244-b00b-f99b21873f6e-kube-api-access-vs5hl\") pod \"distinct-w5-c9dbe6-kserve-7599fd86d9-bdhxd\" (UID: \"e903db90-f2cd-4244-b00b-f99b21873f6e\") " pod="e2e-models-e2e-worker-w5-c9dbe6/distinct-w5-c9dbe6-kserve-7599fd86d9-bdhxd"
Oct 30 22:10:39.391680 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:39.390920    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-c9dbe6/distinct-w5-c9dbe6-kserve-7599fd86d9-bdhxd"
Oct 30 22:10:39.822685 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:39.821858    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:10:40.132380 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:40.129019    5510 patch_prober.go:29] interesting pod/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-n6tv4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body=
Oct 30 22:10:40.132380 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:40.129099    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-n6tv4" podUID="e15eace0-bc5d-43ec-b352-74536b28a56c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused"
Oct 30 22:10:40.148307 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:40.148134    5510 patch_prober.go:29] interesting pod/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-jtsvv container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 22:10:40.148307 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:40.148201    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-jtsvv" podUID="0101f982-8c38-4576-be55-949dfc3fc2de" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:10:40.459956 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:40.457176    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w5-c9dbe6/distinct-w5-c9dbe6-kserve-7599fd86d9-bdhxd"]
Oct 30 22:10:41.065828 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:41.065790    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-n6tv4"
Oct 30 22:10:41.066233 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:41.066205    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-n6tv4"
Oct 30 22:10:41.066980 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:41.066962    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-n6tv4"
Oct 30 22:10:41.461696 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:41.461576    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-c9dbe6/distinct-w5-c9dbe6-kserve-7599fd86d9-bdhxd" event={"ID":"e903db90-f2cd-4244-b00b-f99b21873f6e","Type":"ContainerStarted","Data":"7be64b84b2721816f365dba73b93a2aa877727429b64ea25017ce45976c2e14e"}
Oct 30 22:10:41.461696 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:41.461626    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-c9dbe6/distinct-w5-c9dbe6-kserve-7599fd86d9-bdhxd" event={"ID":"e903db90-f2cd-4244-b00b-f99b21873f6e","Type":"ContainerStarted","Data":"5f783695670c08a6944643d6667c213eb055e0fa8d5975604f56da6f3bb06f31"}
Oct 30 22:10:41.575416 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:41.575276    5510 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w5-c9dbe6/distinct-w5-c9dbe6-kserve-7599fd86d9-bdhxd" podStartSLOduration=3.575257141 podStartE2EDuration="3.575257141s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:10:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:10:41.574091223 +0000 UTC m=+1222.134901394" watchObservedRunningTime="2026-10-30 22:10:41.575257141 +0000 UTC m=+1222.136067147"
Oct 30 22:10:42.391418 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:42.391374    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-c9dbe6/distinct-w5-c9dbe6-kserve-7599fd86d9-bdhxd"
Oct 30 22:10:45.062688 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:45.050179    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:10:49.391930 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:49.391884    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w5-c9dbe6/distinct-w5-c9dbe6-kserve-7599fd86d9-bdhxd"
Oct 30 22:10:49.405172 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:49.405140    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w5-c9dbe6/distinct-w5-c9dbe6-kserve-7599fd86d9-bdhxd"
Oct 30 22:10:49.522608 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:49.522577    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-c9dbe6/distinct-w5-c9dbe6-kserve-7599fd86d9-bdhxd"
Oct 30 22:10:49.969870 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:49.969169    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:10:54.937679 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:54.936717    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:10:59.824444 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:10:59.822123    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:11:00.989556 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:00.989522    5510 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-qcxcm"]
Oct 30 22:11:00.994229 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:00.994200    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-qcxcm"
Oct 30 22:11:00.996983 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:00.996957    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eb115651-ae12-43d8-8022-766c30575782-istiod-ca-cert\") pod \"e2e-worker-w0-779f32-openshift-default-75ddfd96d9-qcxcm\" (UID: \"eb115651-ae12-43d8-8022-766c30575782\") " pod="openshift-ingress/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-qcxcm"
Oct 30 22:11:00.997131 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:00.997089    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eb115651-ae12-43d8-8022-766c30575782-istio-data\") pod \"e2e-worker-w0-779f32-openshift-default-75ddfd96d9-qcxcm\" (UID: \"eb115651-ae12-43d8-8022-766c30575782\") " pod="openshift-ingress/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-qcxcm"
Oct 30 22:11:00.997201 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:00.997131    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb115651-ae12-43d8-8022-766c30575782-credential-socket\") pod \"e2e-worker-w0-779f32-openshift-default-75ddfd96d9-qcxcm\" (UID: \"eb115651-ae12-43d8-8022-766c30575782\") " pod="openshift-ingress/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-qcxcm"
Oct 30 22:11:00.997201 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:00.997175    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zlf5v\" (UniqueName: \"kubernetes.io/projected/eb115651-ae12-43d8-8022-766c30575782-kube-api-access-zlf5v\") pod \"e2e-worker-w0-779f32-openshift-default-75ddfd96d9-qcxcm\" (UID: \"eb115651-ae12-43d8-8022-766c30575782\") " pod="openshift-ingress/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-qcxcm"
Oct 30 22:11:00.997301 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:00.997283    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eb115651-ae12-43d8-8022-766c30575782-istio-envoy\") pod \"e2e-worker-w0-779f32-openshift-default-75ddfd96d9-qcxcm\" (UID: \"eb115651-ae12-43d8-8022-766c30575782\") " pod="openshift-ingress/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-qcxcm"
Oct 30 22:11:00.997337 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:00.997318    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eb115651-ae12-43d8-8022-766c30575782-workload-certs\") pod \"e2e-worker-w0-779f32-openshift-default-75ddfd96d9-qcxcm\" (UID: \"eb115651-ae12-43d8-8022-766c30575782\") " pod="openshift-ingress/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-qcxcm"
Oct 30 22:11:00.997390 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:00.997342    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eb115651-ae12-43d8-8022-766c30575782-istio-token\") pod \"e2e-worker-w0-779f32-openshift-default-75ddfd96d9-qcxcm\" (UID: \"eb115651-ae12-43d8-8022-766c30575782\") " pod="openshift-ingress/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-qcxcm"
Oct 30 22:11:00.997477 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:00.997458    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eb115651-ae12-43d8-8022-766c30575782-istio-podinfo\") pod \"e2e-worker-w0-779f32-openshift-default-75ddfd96d9-qcxcm\" (UID: \"eb115651-ae12-43d8-8022-766c30575782\") " pod="openshift-ingress/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-qcxcm"
Oct 30 22:11:00.997542 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:00.997529    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb115651-ae12-43d8-8022-766c30575782-workload-socket\") pod \"e2e-worker-w0-779f32-openshift-default-75ddfd96d9-qcxcm\" (UID: \"eb115651-ae12-43d8-8022-766c30575782\") " pod="openshift-ingress/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-qcxcm"
Oct 30 22:11:01.098308 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:01.098274    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eb115651-ae12-43d8-8022-766c30575782-istio-podinfo\") pod \"e2e-worker-w0-779f32-openshift-default-75ddfd96d9-qcxcm\" (UID: \"eb115651-ae12-43d8-8022-766c30575782\") " pod="openshift-ingress/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-qcxcm"
Oct 30 22:11:01.098499 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:01.098365    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb115651-ae12-43d8-8022-766c30575782-workload-socket\") pod \"e2e-worker-w0-779f32-openshift-default-75ddfd96d9-qcxcm\" (UID: \"eb115651-ae12-43d8-8022-766c30575782\") " pod="openshift-ingress/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-qcxcm"
Oct 30 22:11:01.098499 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:01.098417    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eb115651-ae12-43d8-8022-766c30575782-istiod-ca-cert\") pod \"e2e-worker-w0-779f32-openshift-default-75ddfd96d9-qcxcm\" (UID: \"eb115651-ae12-43d8-8022-766c30575782\") " pod="openshift-ingress/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-qcxcm"
Oct 30 22:11:01.098616 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:01.098552    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eb115651-ae12-43d8-8022-766c30575782-istio-data\") pod \"e2e-worker-w0-779f32-openshift-default-75ddfd96d9-qcxcm\" (UID: \"eb115651-ae12-43d8-8022-766c30575782\") " pod="openshift-ingress/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-qcxcm"
Oct 30 22:11:01.098616 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:01.098587    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb115651-ae12-43d8-8022-766c30575782-credential-socket\") pod \"e2e-worker-w0-779f32-openshift-default-75ddfd96d9-qcxcm\" (UID: \"eb115651-ae12-43d8-8022-766c30575782\") " pod="openshift-ingress/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-qcxcm"
Oct 30 22:11:01.098760 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:01.098630    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zlf5v\" (UniqueName: \"kubernetes.io/projected/eb115651-ae12-43d8-8022-766c30575782-kube-api-access-zlf5v\") pod \"e2e-worker-w0-779f32-openshift-default-75ddfd96d9-qcxcm\" (UID: \"eb115651-ae12-43d8-8022-766c30575782\") " pod="openshift-ingress/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-qcxcm"
Oct 30 22:11:01.098760 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:01.098729    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eb115651-ae12-43d8-8022-766c30575782-istio-envoy\") pod \"e2e-worker-w0-779f32-openshift-default-75ddfd96d9-qcxcm\" (UID: \"eb115651-ae12-43d8-8022-766c30575782\") " pod="openshift-ingress/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-qcxcm"
Oct 30 22:11:01.098863 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:01.098759    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eb115651-ae12-43d8-8022-766c30575782-workload-certs\") pod \"e2e-worker-w0-779f32-openshift-default-75ddfd96d9-qcxcm\" (UID: \"eb115651-ae12-43d8-8022-766c30575782\") " pod="openshift-ingress/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-qcxcm"
Oct 30 22:11:01.098863 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:01.098784    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eb115651-ae12-43d8-8022-766c30575782-istio-token\") pod \"e2e-worker-w0-779f32-openshift-default-75ddfd96d9-qcxcm\" (UID: \"eb115651-ae12-43d8-8022-766c30575782\") " pod="openshift-ingress/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-qcxcm"
Oct 30 22:11:01.099008 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:01.098892    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb115651-ae12-43d8-8022-766c30575782-workload-socket\") pod \"e2e-worker-w0-779f32-openshift-default-75ddfd96d9-qcxcm\" (UID: \"eb115651-ae12-43d8-8022-766c30575782\") " pod="openshift-ingress/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-qcxcm"
Oct 30 22:11:01.099008 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:01.098906    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eb115651-ae12-43d8-8022-766c30575782-istio-data\") pod \"e2e-worker-w0-779f32-openshift-default-75ddfd96d9-qcxcm\" (UID: \"eb115651-ae12-43d8-8022-766c30575782\") " pod="openshift-ingress/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-qcxcm"
Oct 30 22:11:01.099008 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:01.098974    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb115651-ae12-43d8-8022-766c30575782-credential-socket\") pod \"e2e-worker-w0-779f32-openshift-default-75ddfd96d9-qcxcm\" (UID: \"eb115651-ae12-43d8-8022-766c30575782\") " pod="openshift-ingress/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-qcxcm"
Oct 30 22:11:01.099152 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:01.099139    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eb115651-ae12-43d8-8022-766c30575782-workload-certs\") pod \"e2e-worker-w0-779f32-openshift-default-75ddfd96d9-qcxcm\" (UID: \"eb115651-ae12-43d8-8022-766c30575782\") " pod="openshift-ingress/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-qcxcm"
Oct 30 22:11:01.099634 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:01.099606    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eb115651-ae12-43d8-8022-766c30575782-istiod-ca-cert\") pod \"e2e-worker-w0-779f32-openshift-default-75ddfd96d9-qcxcm\" (UID: \"eb115651-ae12-43d8-8022-766c30575782\") " pod="openshift-ingress/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-qcxcm"
Oct 30 22:11:01.101120 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:01.101100    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eb115651-ae12-43d8-8022-766c30575782-istio-envoy\") pod \"e2e-worker-w0-779f32-openshift-default-75ddfd96d9-qcxcm\" (UID: \"eb115651-ae12-43d8-8022-766c30575782\") " pod="openshift-ingress/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-qcxcm"
Oct 30 22:11:01.101572 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:01.101541    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eb115651-ae12-43d8-8022-766c30575782-istio-podinfo\") pod \"e2e-worker-w0-779f32-openshift-default-75ddfd96d9-qcxcm\" (UID: \"eb115651-ae12-43d8-8022-766c30575782\") " pod="openshift-ingress/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-qcxcm"
Oct 30 22:11:01.117636 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:01.117555    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-qcxcm"]
Oct 30 22:11:01.165589 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:01.165553    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zlf5v\" (UniqueName: \"kubernetes.io/projected/eb115651-ae12-43d8-8022-766c30575782-kube-api-access-zlf5v\") pod \"e2e-worker-w0-779f32-openshift-default-75ddfd96d9-qcxcm\" (UID: \"eb115651-ae12-43d8-8022-766c30575782\") " pod="openshift-ingress/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-qcxcm"
Oct 30 22:11:01.165820 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:01.165799    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eb115651-ae12-43d8-8022-766c30575782-istio-token\") pod \"e2e-worker-w0-779f32-openshift-default-75ddfd96d9-qcxcm\" (UID: \"eb115651-ae12-43d8-8022-766c30575782\") " pod="openshift-ingress/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-qcxcm"
Oct 30 22:11:01.307732 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:01.307698    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-qcxcm"
Oct 30 22:11:01.779389 ip-10-0-136-210 kubenswrapper[5510]: W1030 22:11:01.779353    5510 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb115651_ae12_43d8_8022_766c30575782.slice/crio-6e4bf47dc2eba883d5079cafe215df80e7bb2da6cd01062a6d09426f0a64d36f WatchSource:0}: Error finding container 6e4bf47dc2eba883d5079cafe215df80e7bb2da6cd01062a6d09426f0a64d36f: Status 404 returned error can't find the container with id 6e4bf47dc2eba883d5079cafe215df80e7bb2da6cd01062a6d09426f0a64d36f
Oct 30 22:11:01.780228 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:01.780204    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-qcxcm"]
Oct 30 22:11:01.781903 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:01.781865    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:11:01.782002 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:01.781943    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:11:01.782002 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:01.781988    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:11:02.596209 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:02.596167    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-qcxcm" event={"ID":"eb115651-ae12-43d8-8022-766c30575782","Type":"ContainerStarted","Data":"c27a66e8c84934801603a06cb32d4bd8174ddf789f08009f1f9fa9a48ce966ee"}
Oct 30 22:11:02.596617 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:02.596220    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-qcxcm" event={"ID":"eb115651-ae12-43d8-8022-766c30575782","Type":"ContainerStarted","Data":"6e4bf47dc2eba883d5079cafe215df80e7bb2da6cd01062a6d09426f0a64d36f"}
Oct 30 22:11:02.730702 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:02.730607    5510 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-v8m49"]
Oct 30 22:11:02.734638 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:02.734621    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-v8m49"
Oct 30 22:11:02.782307 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:02.782257    5510 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-qcxcm" podStartSLOduration=2.782242257 podStartE2EDuration="2.782242257s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:11:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:11:02.781416395 +0000 UTC m=+1243.342226427" watchObservedRunningTime="2026-10-30 22:11:02.782242257 +0000 UTC m=+1243.343052256"
Oct 30 22:11:02.825034 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:02.825001    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mfpzg\" (UniqueName: \"kubernetes.io/projected/bd8fbe72-d370-4949-9314-ff91cdab6ff6-kube-api-access-mfpzg\") pod \"e2e-worker-w4-65e59f-openshift-default-596d74bb4f-v8m49\" (UID: \"bd8fbe72-d370-4949-9314-ff91cdab6ff6\") " pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-v8m49"
Oct 30 22:11:02.825268 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:02.825238    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd8fbe72-d370-4949-9314-ff91cdab6ff6-credential-socket\") pod \"e2e-worker-w4-65e59f-openshift-default-596d74bb4f-v8m49\" (UID: \"bd8fbe72-d370-4949-9314-ff91cdab6ff6\") " pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-v8m49"
Oct 30 22:11:02.825374 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:02.825301    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd8fbe72-d370-4949-9314-ff91cdab6ff6-workload-socket\") pod \"e2e-worker-w4-65e59f-openshift-default-596d74bb4f-v8m49\" (UID: \"bd8fbe72-d370-4949-9314-ff91cdab6ff6\") " pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-v8m49"
Oct 30 22:11:02.825436 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:02.825392    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bd8fbe72-d370-4949-9314-ff91cdab6ff6-istio-envoy\") pod \"e2e-worker-w4-65e59f-openshift-default-596d74bb4f-v8m49\" (UID: \"bd8fbe72-d370-4949-9314-ff91cdab6ff6\") " pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-v8m49"
Oct 30 22:11:02.825492 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:02.825437    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bd8fbe72-d370-4949-9314-ff91cdab6ff6-istio-podinfo\") pod \"e2e-worker-w4-65e59f-openshift-default-596d74bb4f-v8m49\" (UID: \"bd8fbe72-d370-4949-9314-ff91cdab6ff6\") " pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-v8m49"
Oct 30 22:11:02.825533 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:02.825515    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bd8fbe72-d370-4949-9314-ff91cdab6ff6-istio-token\") pod \"e2e-worker-w4-65e59f-openshift-default-596d74bb4f-v8m49\" (UID: \"bd8fbe72-d370-4949-9314-ff91cdab6ff6\") " pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-v8m49"
Oct 30 22:11:02.825608 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:02.825595    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bd8fbe72-d370-4949-9314-ff91cdab6ff6-workload-certs\") pod \"e2e-worker-w4-65e59f-openshift-default-596d74bb4f-v8m49\" (UID: \"bd8fbe72-d370-4949-9314-ff91cdab6ff6\") " pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-v8m49"
Oct 30 22:11:02.825706 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:02.825680    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bd8fbe72-d370-4949-9314-ff91cdab6ff6-istiod-ca-cert\") pod \"e2e-worker-w4-65e59f-openshift-default-596d74bb4f-v8m49\" (UID: \"bd8fbe72-d370-4949-9314-ff91cdab6ff6\") " pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-v8m49"
Oct 30 22:11:02.825764 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:02.825734    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bd8fbe72-d370-4949-9314-ff91cdab6ff6-istio-data\") pod \"e2e-worker-w4-65e59f-openshift-default-596d74bb4f-v8m49\" (UID: \"bd8fbe72-d370-4949-9314-ff91cdab6ff6\") " pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-v8m49"
Oct 30 22:11:02.834921 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:02.834895    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-v8m49"]
Oct 30 22:11:02.926590 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:02.926499    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bd8fbe72-d370-4949-9314-ff91cdab6ff6-workload-certs\") pod \"e2e-worker-w4-65e59f-openshift-default-596d74bb4f-v8m49\" (UID: \"bd8fbe72-d370-4949-9314-ff91cdab6ff6\") " pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-v8m49"
Oct 30 22:11:02.926590 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:02.926538    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bd8fbe72-d370-4949-9314-ff91cdab6ff6-istiod-ca-cert\") pod \"e2e-worker-w4-65e59f-openshift-default-596d74bb4f-v8m49\" (UID: \"bd8fbe72-d370-4949-9314-ff91cdab6ff6\") " pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-v8m49"
Oct 30 22:11:02.926590 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:02.926566    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bd8fbe72-d370-4949-9314-ff91cdab6ff6-istio-data\") pod \"e2e-worker-w4-65e59f-openshift-default-596d74bb4f-v8m49\" (UID: \"bd8fbe72-d370-4949-9314-ff91cdab6ff6\") " pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-v8m49"
Oct 30 22:11:02.926901 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:02.926637    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mfpzg\" (UniqueName: \"kubernetes.io/projected/bd8fbe72-d370-4949-9314-ff91cdab6ff6-kube-api-access-mfpzg\") pod \"e2e-worker-w4-65e59f-openshift-default-596d74bb4f-v8m49\" (UID: \"bd8fbe72-d370-4949-9314-ff91cdab6ff6\") " pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-v8m49"
Oct 30 22:11:02.926901 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:02.926829    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd8fbe72-d370-4949-9314-ff91cdab6ff6-credential-socket\") pod \"e2e-worker-w4-65e59f-openshift-default-596d74bb4f-v8m49\" (UID: \"bd8fbe72-d370-4949-9314-ff91cdab6ff6\") " pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-v8m49"
Oct 30 22:11:02.926901 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:02.926877    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd8fbe72-d370-4949-9314-ff91cdab6ff6-workload-socket\") pod \"e2e-worker-w4-65e59f-openshift-default-596d74bb4f-v8m49\" (UID: \"bd8fbe72-d370-4949-9314-ff91cdab6ff6\") " pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-v8m49"
Oct 30 22:11:02.927069 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:02.926922    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bd8fbe72-d370-4949-9314-ff91cdab6ff6-istio-envoy\") pod \"e2e-worker-w4-65e59f-openshift-default-596d74bb4f-v8m49\" (UID: \"bd8fbe72-d370-4949-9314-ff91cdab6ff6\") " pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-v8m49"
Oct 30 22:11:02.927069 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:02.926957    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bd8fbe72-d370-4949-9314-ff91cdab6ff6-istio-podinfo\") pod \"e2e-worker-w4-65e59f-openshift-default-596d74bb4f-v8m49\" (UID: \"bd8fbe72-d370-4949-9314-ff91cdab6ff6\") " pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-v8m49"
Oct 30 22:11:02.927069 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:02.927029    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bd8fbe72-d370-4949-9314-ff91cdab6ff6-workload-certs\") pod \"e2e-worker-w4-65e59f-openshift-default-596d74bb4f-v8m49\" (UID: \"bd8fbe72-d370-4949-9314-ff91cdab6ff6\") " pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-v8m49"
Oct 30 22:11:02.927069 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:02.927039    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bd8fbe72-d370-4949-9314-ff91cdab6ff6-istio-token\") pod \"e2e-worker-w4-65e59f-openshift-default-596d74bb4f-v8m49\" (UID: \"bd8fbe72-d370-4949-9314-ff91cdab6ff6\") " pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-v8m49"
Oct 30 22:11:02.927264 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:02.927162    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bd8fbe72-d370-4949-9314-ff91cdab6ff6-istio-data\") pod \"e2e-worker-w4-65e59f-openshift-default-596d74bb4f-v8m49\" (UID: \"bd8fbe72-d370-4949-9314-ff91cdab6ff6\") " pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-v8m49"
Oct 30 22:11:02.927264 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:02.927235    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd8fbe72-d370-4949-9314-ff91cdab6ff6-credential-socket\") pod \"e2e-worker-w4-65e59f-openshift-default-596d74bb4f-v8m49\" (UID: \"bd8fbe72-d370-4949-9314-ff91cdab6ff6\") " pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-v8m49"
Oct 30 22:11:02.927370 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:02.927342    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd8fbe72-d370-4949-9314-ff91cdab6ff6-workload-socket\") pod \"e2e-worker-w4-65e59f-openshift-default-596d74bb4f-v8m49\" (UID: \"bd8fbe72-d370-4949-9314-ff91cdab6ff6\") " pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-v8m49"
Oct 30 22:11:02.927438 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:02.927407    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bd8fbe72-d370-4949-9314-ff91cdab6ff6-istiod-ca-cert\") pod \"e2e-worker-w4-65e59f-openshift-default-596d74bb4f-v8m49\" (UID: \"bd8fbe72-d370-4949-9314-ff91cdab6ff6\") " pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-v8m49"
Oct 30 22:11:02.929281 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:02.929263    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bd8fbe72-d370-4949-9314-ff91cdab6ff6-istio-envoy\") pod \"e2e-worker-w4-65e59f-openshift-default-596d74bb4f-v8m49\" (UID: \"bd8fbe72-d370-4949-9314-ff91cdab6ff6\") " pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-v8m49"
Oct 30 22:11:02.929443 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:02.929415    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bd8fbe72-d370-4949-9314-ff91cdab6ff6-istio-podinfo\") pod \"e2e-worker-w4-65e59f-openshift-default-596d74bb4f-v8m49\" (UID: \"bd8fbe72-d370-4949-9314-ff91cdab6ff6\") " pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-v8m49"
Oct 30 22:11:02.993334 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:02.993301    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mfpzg\" (UniqueName: \"kubernetes.io/projected/bd8fbe72-d370-4949-9314-ff91cdab6ff6-kube-api-access-mfpzg\") pod \"e2e-worker-w4-65e59f-openshift-default-596d74bb4f-v8m49\" (UID: \"bd8fbe72-d370-4949-9314-ff91cdab6ff6\") " pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-v8m49"
Oct 30 22:11:02.995825 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:02.995798    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bd8fbe72-d370-4949-9314-ff91cdab6ff6-istio-token\") pod \"e2e-worker-w4-65e59f-openshift-default-596d74bb4f-v8m49\" (UID: \"bd8fbe72-d370-4949-9314-ff91cdab6ff6\") " pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-v8m49"
Oct 30 22:11:03.067682 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:03.066935    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-v8m49"
Oct 30 22:11:03.307971 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:03.307936    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-qcxcm"
Oct 30 22:11:03.309546 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:03.309524    5510 patch_prober.go:29] interesting pod/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-qcxcm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body=
Oct 30 22:11:03.309620 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:03.309571    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-qcxcm" podUID="eb115651-ae12-43d8-8022-766c30575782" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused"
Oct 30 22:11:03.499518 ip-10-0-136-210 kubenswrapper[5510]: W1030 22:11:03.499488    5510 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd8fbe72_d370_4949_9314_ff91cdab6ff6.slice/crio-8364e2239b982db9f7fb65650f6932d668b8b53a0fa70184b9c0a2c787fad8cc WatchSource:0}: Error finding container 8364e2239b982db9f7fb65650f6932d668b8b53a0fa70184b9c0a2c787fad8cc: Status 404 returned error can't find the container with id 8364e2239b982db9f7fb65650f6932d668b8b53a0fa70184b9c0a2c787fad8cc
Oct 30 22:11:03.501925 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:03.501897    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-v8m49"]
Oct 30 22:11:03.502161 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:03.502130    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:11:03.502221 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:03.502193    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:11:03.502255 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:03.502220    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:11:03.604817 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:03.604787    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-v8m49" event={"ID":"bd8fbe72-d370-4949-9314-ff91cdab6ff6","Type":"ContainerStarted","Data":"8364e2239b982db9f7fb65650f6932d668b8b53a0fa70184b9c0a2c787fad8cc"}
Oct 30 22:11:04.307972 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:04.307938    5510 patch_prober.go:29] interesting pod/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-qcxcm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body=
Oct 30 22:11:04.308157 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:04.307994    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-qcxcm" podUID="eb115651-ae12-43d8-8022-766c30575782" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused"
Oct 30 22:11:04.613191 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:04.613094    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-v8m49" event={"ID":"bd8fbe72-d370-4949-9314-ff91cdab6ff6","Type":"ContainerStarted","Data":"11632c267999841c4d2731b19a4d82fd58583e965ea9ca58b7dc3ac939125328"}
Oct 30 22:11:04.848683 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:04.844603    5510 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-v8m49" podStartSLOduration=2.844585038 podStartE2EDuration="2.844585038s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:11:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:11:04.793156739 +0000 UTC m=+1245.353966780" watchObservedRunningTime="2026-10-30 22:11:04.844585038 +0000 UTC m=+1245.405395043"
Oct 30 22:11:04.959735 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:04.959277    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:11:05.072382 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:05.072298    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-v8m49"
Oct 30 22:11:05.072689 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:05.072645    5510 patch_prober.go:29] interesting pod/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-v8m49 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body=
Oct 30 22:11:05.072864 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:05.072838    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-v8m49" podUID="bd8fbe72-d370-4949-9314-ff91cdab6ff6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused"
Oct 30 22:11:05.308681 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:05.308396    5510 patch_prober.go:29] interesting pod/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-qcxcm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body=
Oct 30 22:11:05.308681 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:05.308457    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-qcxcm" podUID="eb115651-ae12-43d8-8022-766c30575782" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused"
Oct 30 22:11:06.072133 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:06.072088    5510 patch_prober.go:29] interesting pod/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-v8m49 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body=
Oct 30 22:11:06.072806 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:06.072772    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-v8m49" podUID="bd8fbe72-d370-4949-9314-ff91cdab6ff6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused"
Oct 30 22:11:06.308598 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:06.308143    5510 patch_prober.go:29] interesting pod/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-qcxcm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body=
Oct 30 22:11:06.308598 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:06.308207    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-qcxcm" podUID="eb115651-ae12-43d8-8022-766c30575782" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused"
Oct 30 22:11:07.073107 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:07.073067    5510 patch_prober.go:29] interesting pod/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-v8m49 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body=
Oct 30 22:11:07.073758 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:07.073724    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-v8m49" podUID="bd8fbe72-d370-4949-9314-ff91cdab6ff6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused"
Oct 30 22:11:07.309000 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:07.308963    5510 patch_prober.go:29] interesting pod/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-qcxcm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body=
Oct 30 22:11:07.309210 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:07.309190    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-qcxcm" podUID="eb115651-ae12-43d8-8022-766c30575782" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused"
Oct 30 22:11:08.073693 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:08.073074    5510 patch_prober.go:29] interesting pod/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-v8m49 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body=
Oct 30 22:11:08.073693 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:08.073140    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-v8m49" podUID="bd8fbe72-d370-4949-9314-ff91cdab6ff6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused"
Oct 30 22:11:08.308668 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:08.308620    5510 patch_prober.go:29] interesting pod/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-qcxcm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body=
Oct 30 22:11:08.308852 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:08.308704    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-qcxcm" podUID="eb115651-ae12-43d8-8022-766c30575782" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused"
Oct 30 22:11:08.422134 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:11:08.422038    5510 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6gjbf" containerName="extractor"
Oct 30 22:11:09.069078 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:09.069035    5510 patch_prober.go:29] interesting pod/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-v8m49 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body=
Oct 30 22:11:09.069368 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:09.069335    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-v8m49" podUID="bd8fbe72-d370-4949-9314-ff91cdab6ff6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused"
Oct 30 22:11:09.400871 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:09.400783    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-qcxcm"
Oct 30 22:11:09.401861 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:09.401836    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-qcxcm"
Oct 30 22:11:09.402651 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:09.402631    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-qcxcm"
Oct 30 22:11:10.068531 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:10.068492    5510 patch_prober.go:29] interesting pod/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-v8m49 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body=
Oct 30 22:11:10.068873 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:10.068841    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-v8m49" podUID="bd8fbe72-d370-4949-9314-ff91cdab6ff6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused"
Oct 30 22:11:10.069729 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:10.069701    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:11:10.295546 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:10.295512    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:11:11.072886 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:11.072856    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-v8m49"
Oct 30 22:11:11.073311 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:11.073134    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-v8m49"
Oct 30 22:11:11.073831 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:11.073813    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-v8m49"
Oct 30 22:11:15.169680 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:15.153260    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:11:20.025412 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:20.025376    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:11:24.806946 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:24.806912    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:11:25.169445 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:25.169358    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:11:29.758876 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:29.758835    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:11:32.816867 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:32.816825    5510 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-bxlgp"]
Oct 30 22:11:32.822245 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:32.822223    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-bxlgp"
Oct 30 22:11:32.875989 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:32.875960    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w6-fb0b93-openshift-default-dockercfg-2kctd\""
Oct 30 22:11:32.929433 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:32.929394    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-bxlgp"]
Oct 30 22:11:32.951391 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:32.951350    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f789c7e9-26e6-4b2d-b1d7-536dc5d526d8-credential-socket\") pod \"e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-bxlgp\" (UID: \"f789c7e9-26e6-4b2d-b1d7-536dc5d526d8\") " pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-bxlgp"
Oct 30 22:11:32.951586 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:32.951525    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f789c7e9-26e6-4b2d-b1d7-536dc5d526d8-istiod-ca-cert\") pod \"e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-bxlgp\" (UID: \"f789c7e9-26e6-4b2d-b1d7-536dc5d526d8\") " pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-bxlgp"
Oct 30 22:11:32.951586 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:32.951561    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f789c7e9-26e6-4b2d-b1d7-536dc5d526d8-istio-token\") pod \"e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-bxlgp\" (UID: \"f789c7e9-26e6-4b2d-b1d7-536dc5d526d8\") " pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-bxlgp"
Oct 30 22:11:32.951762 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:32.951734    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f789c7e9-26e6-4b2d-b1d7-536dc5d526d8-istio-envoy\") pod \"e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-bxlgp\" (UID: \"f789c7e9-26e6-4b2d-b1d7-536dc5d526d8\") " pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-bxlgp"
Oct 30 22:11:32.951826 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:32.951779    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8ckpx\" (UniqueName: \"kubernetes.io/projected/f789c7e9-26e6-4b2d-b1d7-536dc5d526d8-kube-api-access-8ckpx\") pod \"e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-bxlgp\" (UID: \"f789c7e9-26e6-4b2d-b1d7-536dc5d526d8\") " pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-bxlgp"
Oct 30 22:11:32.951884 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:32.951845    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f789c7e9-26e6-4b2d-b1d7-536dc5d526d8-workload-socket\") pod \"e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-bxlgp\" (UID: \"f789c7e9-26e6-4b2d-b1d7-536dc5d526d8\") " pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-bxlgp"
Oct 30 22:11:32.951993 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:32.951963    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f789c7e9-26e6-4b2d-b1d7-536dc5d526d8-workload-certs\") pod \"e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-bxlgp\" (UID: \"f789c7e9-26e6-4b2d-b1d7-536dc5d526d8\") " pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-bxlgp"
Oct 30 22:11:32.952054 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:32.952022    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f789c7e9-26e6-4b2d-b1d7-536dc5d526d8-istio-data\") pod \"e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-bxlgp\" (UID: \"f789c7e9-26e6-4b2d-b1d7-536dc5d526d8\") " pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-bxlgp"
Oct 30 22:11:32.952120 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:32.952059    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f789c7e9-26e6-4b2d-b1d7-536dc5d526d8-istio-podinfo\") pod \"e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-bxlgp\" (UID: \"f789c7e9-26e6-4b2d-b1d7-536dc5d526d8\") " pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-bxlgp"
Oct 30 22:11:33.052962 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:33.052925    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f789c7e9-26e6-4b2d-b1d7-536dc5d526d8-istiod-ca-cert\") pod \"e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-bxlgp\" (UID: \"f789c7e9-26e6-4b2d-b1d7-536dc5d526d8\") " pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-bxlgp"
Oct 30 22:11:33.052962 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:33.052963    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f789c7e9-26e6-4b2d-b1d7-536dc5d526d8-istio-token\") pod \"e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-bxlgp\" (UID: \"f789c7e9-26e6-4b2d-b1d7-536dc5d526d8\") " pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-bxlgp"
Oct 30 22:11:33.053209 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:33.053046    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f789c7e9-26e6-4b2d-b1d7-536dc5d526d8-istio-envoy\") pod \"e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-bxlgp\" (UID: \"f789c7e9-26e6-4b2d-b1d7-536dc5d526d8\") " pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-bxlgp"
Oct 30 22:11:33.053209 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:33.053071    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8ckpx\" (UniqueName: \"kubernetes.io/projected/f789c7e9-26e6-4b2d-b1d7-536dc5d526d8-kube-api-access-8ckpx\") pod \"e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-bxlgp\" (UID: \"f789c7e9-26e6-4b2d-b1d7-536dc5d526d8\") " pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-bxlgp"
Oct 30 22:11:33.053209 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:33.053102    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f789c7e9-26e6-4b2d-b1d7-536dc5d526d8-workload-socket\") pod \"e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-bxlgp\" (UID: \"f789c7e9-26e6-4b2d-b1d7-536dc5d526d8\") " pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-bxlgp"
Oct 30 22:11:33.053209 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:33.053138    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f789c7e9-26e6-4b2d-b1d7-536dc5d526d8-workload-certs\") pod \"e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-bxlgp\" (UID: \"f789c7e9-26e6-4b2d-b1d7-536dc5d526d8\") " pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-bxlgp"
Oct 30 22:11:33.053209 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:33.053169    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f789c7e9-26e6-4b2d-b1d7-536dc5d526d8-istio-data\") pod \"e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-bxlgp\" (UID: \"f789c7e9-26e6-4b2d-b1d7-536dc5d526d8\") " pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-bxlgp"
Oct 30 22:11:33.053209 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:33.053206    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f789c7e9-26e6-4b2d-b1d7-536dc5d526d8-istio-podinfo\") pod \"e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-bxlgp\" (UID: \"f789c7e9-26e6-4b2d-b1d7-536dc5d526d8\") " pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-bxlgp"
Oct 30 22:11:33.053602 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:33.053272    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f789c7e9-26e6-4b2d-b1d7-536dc5d526d8-credential-socket\") pod \"e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-bxlgp\" (UID: \"f789c7e9-26e6-4b2d-b1d7-536dc5d526d8\") " pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-bxlgp"
Oct 30 22:11:33.053602 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:33.053568    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f789c7e9-26e6-4b2d-b1d7-536dc5d526d8-workload-socket\") pod \"e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-bxlgp\" (UID: \"f789c7e9-26e6-4b2d-b1d7-536dc5d526d8\") " pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-bxlgp"
Oct 30 22:11:33.053727 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:33.053638    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f789c7e9-26e6-4b2d-b1d7-536dc5d526d8-workload-certs\") pod \"e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-bxlgp\" (UID: \"f789c7e9-26e6-4b2d-b1d7-536dc5d526d8\") " pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-bxlgp"
Oct 30 22:11:33.053785 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:33.053724    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f789c7e9-26e6-4b2d-b1d7-536dc5d526d8-credential-socket\") pod \"e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-bxlgp\" (UID: \"f789c7e9-26e6-4b2d-b1d7-536dc5d526d8\") " pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-bxlgp"
Oct 30 22:11:33.053844 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:33.053791    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f789c7e9-26e6-4b2d-b1d7-536dc5d526d8-istio-data\") pod \"e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-bxlgp\" (UID: \"f789c7e9-26e6-4b2d-b1d7-536dc5d526d8\") " pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-bxlgp"
Oct 30 22:11:33.053844 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:33.053820    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f789c7e9-26e6-4b2d-b1d7-536dc5d526d8-istiod-ca-cert\") pod \"e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-bxlgp\" (UID: \"f789c7e9-26e6-4b2d-b1d7-536dc5d526d8\") " pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-bxlgp"
Oct 30 22:11:33.055365 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:33.055344    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f789c7e9-26e6-4b2d-b1d7-536dc5d526d8-istio-envoy\") pod \"e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-bxlgp\" (UID: \"f789c7e9-26e6-4b2d-b1d7-536dc5d526d8\") " pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-bxlgp"
Oct 30 22:11:33.055773 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:33.055751    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f789c7e9-26e6-4b2d-b1d7-536dc5d526d8-istio-podinfo\") pod \"e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-bxlgp\" (UID: \"f789c7e9-26e6-4b2d-b1d7-536dc5d526d8\") " pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-bxlgp"
Oct 30 22:11:33.116872 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:33.116795    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f789c7e9-26e6-4b2d-b1d7-536dc5d526d8-istio-token\") pod \"e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-bxlgp\" (UID: \"f789c7e9-26e6-4b2d-b1d7-536dc5d526d8\") " pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-bxlgp"
Oct 30 22:11:33.117038 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:33.116891    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8ckpx\" (UniqueName: \"kubernetes.io/projected/f789c7e9-26e6-4b2d-b1d7-536dc5d526d8-kube-api-access-8ckpx\") pod \"e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-bxlgp\" (UID: \"f789c7e9-26e6-4b2d-b1d7-536dc5d526d8\") " pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-bxlgp"
Oct 30 22:11:33.137400 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:33.137372    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-bxlgp"
Oct 30 22:11:33.562573 ip-10-0-136-210 kubenswrapper[5510]: W1030 22:11:33.562540    5510 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf789c7e9_26e6_4b2d_b1d7_536dc5d526d8.slice/crio-0c127be2182b40f2dd4e92eaa96fc43698b20e9e54f975988d2b68929bd8ede2 WatchSource:0}: Error finding container 0c127be2182b40f2dd4e92eaa96fc43698b20e9e54f975988d2b68929bd8ede2: Status 404 returned error can't find the container with id 0c127be2182b40f2dd4e92eaa96fc43698b20e9e54f975988d2b68929bd8ede2
Oct 30 22:11:33.563476 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:33.563453    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-bxlgp"]
Oct 30 22:11:33.564987 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:33.564956    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:11:33.565048 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:33.565023    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:11:33.565083 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:33.565051    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:11:33.842406 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:33.842314    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-bxlgp" event={"ID":"f789c7e9-26e6-4b2d-b1d7-536dc5d526d8","Type":"ContainerStarted","Data":"8dee132045c366b87c195a3373e8435381cc8da438522107be31803f8cafbc07"}
Oct 30 22:11:33.842406 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:33.842368    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-bxlgp" event={"ID":"f789c7e9-26e6-4b2d-b1d7-536dc5d526d8","Type":"ContainerStarted","Data":"0c127be2182b40f2dd4e92eaa96fc43698b20e9e54f975988d2b68929bd8ede2"}
Oct 30 22:11:33.967558 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:33.967504    5510 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-bxlgp" podStartSLOduration=1.967489045 podStartE2EDuration="1.967489045s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:11:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:11:33.965407256 +0000 UTC m=+1274.526217285" watchObservedRunningTime="2026-10-30 22:11:33.967489045 +0000 UTC m=+1274.528299050"
Oct 30 22:11:34.138456 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:34.138365    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-bxlgp"
Oct 30 22:11:34.139947 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:34.139922    5510 patch_prober.go:29] interesting pod/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-bxlgp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body=
Oct 30 22:11:34.140026 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:34.140003    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-bxlgp" podUID="f789c7e9-26e6-4b2d-b1d7-536dc5d526d8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused"
Oct 30 22:11:34.299609 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:34.299519    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:11:35.137830 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:35.137736    5510 patch_prober.go:29] interesting pod/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-bxlgp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body=
Oct 30 22:11:35.137830 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:35.137798    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-bxlgp" podUID="f789c7e9-26e6-4b2d-b1d7-536dc5d526d8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused"
Oct 30 22:11:36.138645 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:36.138609    5510 patch_prober.go:29] interesting pod/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-bxlgp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body=
Oct 30 22:11:36.139089 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:36.138715    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-bxlgp" podUID="f789c7e9-26e6-4b2d-b1d7-536dc5d526d8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused"
Oct 30 22:11:37.138876 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:37.138837    5510 patch_prober.go:29] interesting pod/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-bxlgp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body=
Oct 30 22:11:37.139348 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:37.138908    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-bxlgp" podUID="f789c7e9-26e6-4b2d-b1d7-536dc5d526d8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused"
Oct 30 22:11:38.138306 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:38.138268    5510 patch_prober.go:29] interesting pod/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-bxlgp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body=
Oct 30 22:11:38.138495 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:38.138325    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-bxlgp" podUID="f789c7e9-26e6-4b2d-b1d7-536dc5d526d8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused"
Oct 30 22:11:38.420183 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:11:38.420096    5510 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6gjbf" containerName="extractor"
Oct 30 22:11:38.938965 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:38.938849    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:11:39.138732 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:39.138698    5510 patch_prober.go:29] interesting pod/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-bxlgp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body=
Oct 30 22:11:39.138916 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:39.138777    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-bxlgp" podUID="f789c7e9-26e6-4b2d-b1d7-536dc5d526d8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused"
Oct 30 22:11:40.137827 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:40.137783    5510 patch_prober.go:29] interesting pod/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-bxlgp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body=
Oct 30 22:11:40.138319 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:40.137845    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-bxlgp" podUID="f789c7e9-26e6-4b2d-b1d7-536dc5d526d8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused"
Oct 30 22:11:41.142515 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:41.142480    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-bxlgp"
Oct 30 22:11:41.143008 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:41.142812    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-bxlgp"
Oct 30 22:11:41.143725 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:41.143709    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-bxlgp"
Oct 30 22:11:43.605390 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:43.605292    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:11:44.264357 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:44.264314    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:11:48.239543 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:48.239502    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:11:51.143310 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:51.143265    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:11:52.931234 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:52.931192    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:11:57.495228 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:11:57.495186    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:12:02.282094 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:02.282056    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:12:07.295451 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:07.295408    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:12:08.420724 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:12:08.420686    5510 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6gjbf" containerName="extractor"
Oct 30 22:12:11.883967 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:11.883771    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:12:16.414825 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:16.414779    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:12:17.820709 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:17.820642    5510 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-m2c8l"]
Oct 30 22:12:17.825093 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:17.825069    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-m2c8l"
Oct 30 22:12:17.835300 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:17.835277    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4798995a-2305-4dae-adf7-45aa66e0cb18-istio-token\") pod \"e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-m2c8l\" (UID: \"4798995a-2305-4dae-adf7-45aa66e0cb18\") " pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-m2c8l"
Oct 30 22:12:17.835386 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:17.835314    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4798995a-2305-4dae-adf7-45aa66e0cb18-workload-certs\") pod \"e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-m2c8l\" (UID: \"4798995a-2305-4dae-adf7-45aa66e0cb18\") " pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-m2c8l"
Oct 30 22:12:17.835461 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:17.835442    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4798995a-2305-4dae-adf7-45aa66e0cb18-istio-data\") pod \"e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-m2c8l\" (UID: \"4798995a-2305-4dae-adf7-45aa66e0cb18\") " pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-m2c8l"
Oct 30 22:12:17.835628 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:17.835606    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4798995a-2305-4dae-adf7-45aa66e0cb18-credential-socket\") pod \"e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-m2c8l\" (UID: \"4798995a-2305-4dae-adf7-45aa66e0cb18\") " pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-m2c8l"
Oct 30 22:12:17.835700 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:17.835644    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4798995a-2305-4dae-adf7-45aa66e0cb18-istiod-ca-cert\") pod \"e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-m2c8l\" (UID: \"4798995a-2305-4dae-adf7-45aa66e0cb18\") " pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-m2c8l"
Oct 30 22:12:17.835745 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:17.835730    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g5np9\" (UniqueName: \"kubernetes.io/projected/4798995a-2305-4dae-adf7-45aa66e0cb18-kube-api-access-g5np9\") pod \"e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-m2c8l\" (UID: \"4798995a-2305-4dae-adf7-45aa66e0cb18\") " pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-m2c8l"
Oct 30 22:12:17.835803 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:17.835787    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4798995a-2305-4dae-adf7-45aa66e0cb18-istio-podinfo\") pod \"e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-m2c8l\" (UID: \"4798995a-2305-4dae-adf7-45aa66e0cb18\") " pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-m2c8l"
Oct 30 22:12:17.835840 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:17.835813    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4798995a-2305-4dae-adf7-45aa66e0cb18-istio-envoy\") pod \"e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-m2c8l\" (UID: \"4798995a-2305-4dae-adf7-45aa66e0cb18\") " pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-m2c8l"
Oct 30 22:12:17.835840 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:17.835832    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4798995a-2305-4dae-adf7-45aa66e0cb18-workload-socket\") pod \"e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-m2c8l\" (UID: \"4798995a-2305-4dae-adf7-45aa66e0cb18\") " pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-m2c8l"
Oct 30 22:12:17.931905 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:17.931862    5510 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-xbr94"]
Oct 30 22:12:17.935979 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:17.935954    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-xbr94"
Oct 30 22:12:17.936927 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:17.936901    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4798995a-2305-4dae-adf7-45aa66e0cb18-credential-socket\") pod \"e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-m2c8l\" (UID: \"4798995a-2305-4dae-adf7-45aa66e0cb18\") " pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-m2c8l"
Oct 30 22:12:17.937038 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:17.936937    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4798995a-2305-4dae-adf7-45aa66e0cb18-istiod-ca-cert\") pod \"e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-m2c8l\" (UID: \"4798995a-2305-4dae-adf7-45aa66e0cb18\") " pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-m2c8l"
Oct 30 22:12:17.937038 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:17.936965    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g5np9\" (UniqueName: \"kubernetes.io/projected/4798995a-2305-4dae-adf7-45aa66e0cb18-kube-api-access-g5np9\") pod \"e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-m2c8l\" (UID: \"4798995a-2305-4dae-adf7-45aa66e0cb18\") " pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-m2c8l"
Oct 30 22:12:17.937038 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:17.937031    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4798995a-2305-4dae-adf7-45aa66e0cb18-istio-podinfo\") pod \"e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-m2c8l\" (UID: \"4798995a-2305-4dae-adf7-45aa66e0cb18\") " pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-m2c8l"
Oct 30 22:12:17.937213 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:17.937059    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4798995a-2305-4dae-adf7-45aa66e0cb18-istio-envoy\") pod \"e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-m2c8l\" (UID: \"4798995a-2305-4dae-adf7-45aa66e0cb18\") " pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-m2c8l"
Oct 30 22:12:17.937213 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:17.937088    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4798995a-2305-4dae-adf7-45aa66e0cb18-workload-socket\") pod \"e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-m2c8l\" (UID: \"4798995a-2305-4dae-adf7-45aa66e0cb18\") " pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-m2c8l"
Oct 30 22:12:17.937213 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:17.937143    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4798995a-2305-4dae-adf7-45aa66e0cb18-istio-token\") pod \"e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-m2c8l\" (UID: \"4798995a-2305-4dae-adf7-45aa66e0cb18\") " pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-m2c8l"
Oct 30 22:12:17.937213 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:17.937181    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4798995a-2305-4dae-adf7-45aa66e0cb18-workload-certs\") pod \"e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-m2c8l\" (UID: \"4798995a-2305-4dae-adf7-45aa66e0cb18\") " pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-m2c8l"
Oct 30 22:12:17.937433 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:17.937316    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4798995a-2305-4dae-adf7-45aa66e0cb18-istio-data\") pod \"e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-m2c8l\" (UID: \"4798995a-2305-4dae-adf7-45aa66e0cb18\") " pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-m2c8l"
Oct 30 22:12:17.937518 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:17.937319    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4798995a-2305-4dae-adf7-45aa66e0cb18-credential-socket\") pod \"e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-m2c8l\" (UID: \"4798995a-2305-4dae-adf7-45aa66e0cb18\") " pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-m2c8l"
Oct 30 22:12:17.937575 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:17.937552    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4798995a-2305-4dae-adf7-45aa66e0cb18-workload-socket\") pod \"e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-m2c8l\" (UID: \"4798995a-2305-4dae-adf7-45aa66e0cb18\") " pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-m2c8l"
Oct 30 22:12:17.938089 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:17.937744    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4798995a-2305-4dae-adf7-45aa66e0cb18-istiod-ca-cert\") pod \"e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-m2c8l\" (UID: \"4798995a-2305-4dae-adf7-45aa66e0cb18\") " pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-m2c8l"
Oct 30 22:12:17.938089 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:17.938026    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4798995a-2305-4dae-adf7-45aa66e0cb18-workload-certs\") pod \"e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-m2c8l\" (UID: \"4798995a-2305-4dae-adf7-45aa66e0cb18\") " pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-m2c8l"
Oct 30 22:12:17.938089 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:17.938034    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4798995a-2305-4dae-adf7-45aa66e0cb18-istio-data\") pod \"e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-m2c8l\" (UID: \"4798995a-2305-4dae-adf7-45aa66e0cb18\") " pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-m2c8l"
Oct 30 22:12:17.940112 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:17.940088    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4798995a-2305-4dae-adf7-45aa66e0cb18-istio-envoy\") pod \"e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-m2c8l\" (UID: \"4798995a-2305-4dae-adf7-45aa66e0cb18\") " pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-m2c8l"
Oct 30 22:12:17.940319 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:17.940303    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4798995a-2305-4dae-adf7-45aa66e0cb18-istio-podinfo\") pod \"e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-m2c8l\" (UID: \"4798995a-2305-4dae-adf7-45aa66e0cb18\") " pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-m2c8l"
Oct 30 22:12:17.967862 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:17.967834    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-xbr94"]
Oct 30 22:12:18.004187 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:18.002870    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4798995a-2305-4dae-adf7-45aa66e0cb18-istio-token\") pod \"e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-m2c8l\" (UID: \"4798995a-2305-4dae-adf7-45aa66e0cb18\") " pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-m2c8l"
Oct 30 22:12:18.005237 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:18.005213    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g5np9\" (UniqueName: \"kubernetes.io/projected/4798995a-2305-4dae-adf7-45aa66e0cb18-kube-api-access-g5np9\") pod \"e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-m2c8l\" (UID: \"4798995a-2305-4dae-adf7-45aa66e0cb18\") " pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-m2c8l"
Oct 30 22:12:18.039400 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:18.038710    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/af4ee930-6db5-4c5c-9727-1ca855360b34-workload-certs\") pod \"e2e-worker-w1-a00d9b-openshift-default-847df8fc59-xbr94\" (UID: \"af4ee930-6db5-4c5c-9727-1ca855360b34\") " pod="openshift-ingress/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-xbr94"
Oct 30 22:12:18.039400 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:18.038776    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9g7vz\" (UniqueName: \"kubernetes.io/projected/af4ee930-6db5-4c5c-9727-1ca855360b34-kube-api-access-9g7vz\") pod \"e2e-worker-w1-a00d9b-openshift-default-847df8fc59-xbr94\" (UID: \"af4ee930-6db5-4c5c-9727-1ca855360b34\") " pod="openshift-ingress/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-xbr94"
Oct 30 22:12:18.039400 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:18.038825    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/af4ee930-6db5-4c5c-9727-1ca855360b34-istio-data\") pod \"e2e-worker-w1-a00d9b-openshift-default-847df8fc59-xbr94\" (UID: \"af4ee930-6db5-4c5c-9727-1ca855360b34\") " pod="openshift-ingress/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-xbr94"
Oct 30 22:12:18.039400 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:18.038880    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/af4ee930-6db5-4c5c-9727-1ca855360b34-istio-envoy\") pod \"e2e-worker-w1-a00d9b-openshift-default-847df8fc59-xbr94\" (UID: \"af4ee930-6db5-4c5c-9727-1ca855360b34\") " pod="openshift-ingress/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-xbr94"
Oct 30 22:12:18.039400 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:18.038951    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/af4ee930-6db5-4c5c-9727-1ca855360b34-credential-socket\") pod \"e2e-worker-w1-a00d9b-openshift-default-847df8fc59-xbr94\" (UID: \"af4ee930-6db5-4c5c-9727-1ca855360b34\") " pod="openshift-ingress/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-xbr94"
Oct 30 22:12:18.039400 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:18.039034    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/af4ee930-6db5-4c5c-9727-1ca855360b34-istio-podinfo\") pod \"e2e-worker-w1-a00d9b-openshift-default-847df8fc59-xbr94\" (UID: \"af4ee930-6db5-4c5c-9727-1ca855360b34\") " pod="openshift-ingress/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-xbr94"
Oct 30 22:12:18.039400 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:18.039146    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/af4ee930-6db5-4c5c-9727-1ca855360b34-istiod-ca-cert\") pod \"e2e-worker-w1-a00d9b-openshift-default-847df8fc59-xbr94\" (UID: \"af4ee930-6db5-4c5c-9727-1ca855360b34\") " pod="openshift-ingress/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-xbr94"
Oct 30 22:12:18.039400 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:18.039180    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/af4ee930-6db5-4c5c-9727-1ca855360b34-workload-socket\") pod \"e2e-worker-w1-a00d9b-openshift-default-847df8fc59-xbr94\" (UID: \"af4ee930-6db5-4c5c-9727-1ca855360b34\") " pod="openshift-ingress/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-xbr94"
Oct 30 22:12:18.039400 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:18.039203    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/af4ee930-6db5-4c5c-9727-1ca855360b34-istio-token\") pod \"e2e-worker-w1-a00d9b-openshift-default-847df8fc59-xbr94\" (UID: \"af4ee930-6db5-4c5c-9727-1ca855360b34\") " pod="openshift-ingress/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-xbr94"
Oct 30 22:12:18.041437 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:18.041410    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-m2c8l"]
Oct 30 22:12:18.139516 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:18.139426    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-m2c8l"
Oct 30 22:12:18.140162 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:18.140138    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9g7vz\" (UniqueName: \"kubernetes.io/projected/af4ee930-6db5-4c5c-9727-1ca855360b34-kube-api-access-9g7vz\") pod \"e2e-worker-w1-a00d9b-openshift-default-847df8fc59-xbr94\" (UID: \"af4ee930-6db5-4c5c-9727-1ca855360b34\") " pod="openshift-ingress/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-xbr94"
Oct 30 22:12:18.140234 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:18.140189    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/af4ee930-6db5-4c5c-9727-1ca855360b34-istio-data\") pod \"e2e-worker-w1-a00d9b-openshift-default-847df8fc59-xbr94\" (UID: \"af4ee930-6db5-4c5c-9727-1ca855360b34\") " pod="openshift-ingress/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-xbr94"
Oct 30 22:12:18.140234 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:18.140228    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/af4ee930-6db5-4c5c-9727-1ca855360b34-istio-envoy\") pod \"e2e-worker-w1-a00d9b-openshift-default-847df8fc59-xbr94\" (UID: \"af4ee930-6db5-4c5c-9727-1ca855360b34\") " pod="openshift-ingress/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-xbr94"
Oct 30 22:12:18.140321 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:18.140282    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/af4ee930-6db5-4c5c-9727-1ca855360b34-credential-socket\") pod \"e2e-worker-w1-a00d9b-openshift-default-847df8fc59-xbr94\" (UID: \"af4ee930-6db5-4c5c-9727-1ca855360b34\") " pod="openshift-ingress/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-xbr94"
Oct 30 22:12:18.140376 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:18.140363    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/af4ee930-6db5-4c5c-9727-1ca855360b34-istio-podinfo\") pod \"e2e-worker-w1-a00d9b-openshift-default-847df8fc59-xbr94\" (UID: \"af4ee930-6db5-4c5c-9727-1ca855360b34\") " pod="openshift-ingress/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-xbr94"
Oct 30 22:12:18.140494 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:18.140477    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/af4ee930-6db5-4c5c-9727-1ca855360b34-istiod-ca-cert\") pod \"e2e-worker-w1-a00d9b-openshift-default-847df8fc59-xbr94\" (UID: \"af4ee930-6db5-4c5c-9727-1ca855360b34\") " pod="openshift-ingress/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-xbr94"
Oct 30 22:12:18.140549 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:18.140524    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/af4ee930-6db5-4c5c-9727-1ca855360b34-workload-socket\") pod \"e2e-worker-w1-a00d9b-openshift-default-847df8fc59-xbr94\" (UID: \"af4ee930-6db5-4c5c-9727-1ca855360b34\") " pod="openshift-ingress/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-xbr94"
Oct 30 22:12:18.140600 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:18.140559    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/af4ee930-6db5-4c5c-9727-1ca855360b34-istio-token\") pod \"e2e-worker-w1-a00d9b-openshift-default-847df8fc59-xbr94\" (UID: \"af4ee930-6db5-4c5c-9727-1ca855360b34\") " pod="openshift-ingress/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-xbr94"
Oct 30 22:12:18.140734 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:18.140710    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/af4ee930-6db5-4c5c-9727-1ca855360b34-workload-certs\") pod \"e2e-worker-w1-a00d9b-openshift-default-847df8fc59-xbr94\" (UID: \"af4ee930-6db5-4c5c-9727-1ca855360b34\") " pod="openshift-ingress/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-xbr94"
Oct 30 22:12:18.140890 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:18.140764    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/af4ee930-6db5-4c5c-9727-1ca855360b34-istio-data\") pod \"e2e-worker-w1-a00d9b-openshift-default-847df8fc59-xbr94\" (UID: \"af4ee930-6db5-4c5c-9727-1ca855360b34\") " pod="openshift-ingress/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-xbr94"
Oct 30 22:12:18.140890 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:18.140795    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/af4ee930-6db5-4c5c-9727-1ca855360b34-credential-socket\") pod \"e2e-worker-w1-a00d9b-openshift-default-847df8fc59-xbr94\" (UID: \"af4ee930-6db5-4c5c-9727-1ca855360b34\") " pod="openshift-ingress/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-xbr94"
Oct 30 22:12:18.141211 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:18.141184    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/af4ee930-6db5-4c5c-9727-1ca855360b34-workload-socket\") pod \"e2e-worker-w1-a00d9b-openshift-default-847df8fc59-xbr94\" (UID: \"af4ee930-6db5-4c5c-9727-1ca855360b34\") " pod="openshift-ingress/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-xbr94"
Oct 30 22:12:18.141318 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:18.141280    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/af4ee930-6db5-4c5c-9727-1ca855360b34-workload-certs\") pod \"e2e-worker-w1-a00d9b-openshift-default-847df8fc59-xbr94\" (UID: \"af4ee930-6db5-4c5c-9727-1ca855360b34\") " pod="openshift-ingress/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-xbr94"
Oct 30 22:12:18.141384 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:18.141342    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/af4ee930-6db5-4c5c-9727-1ca855360b34-istiod-ca-cert\") pod \"e2e-worker-w1-a00d9b-openshift-default-847df8fc59-xbr94\" (UID: \"af4ee930-6db5-4c5c-9727-1ca855360b34\") " pod="openshift-ingress/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-xbr94"
Oct 30 22:12:18.142715 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:18.142688    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/af4ee930-6db5-4c5c-9727-1ca855360b34-istio-envoy\") pod \"e2e-worker-w1-a00d9b-openshift-default-847df8fc59-xbr94\" (UID: \"af4ee930-6db5-4c5c-9727-1ca855360b34\") " pod="openshift-ingress/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-xbr94"
Oct 30 22:12:18.143213 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:18.143105    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/af4ee930-6db5-4c5c-9727-1ca855360b34-istio-podinfo\") pod \"e2e-worker-w1-a00d9b-openshift-default-847df8fc59-xbr94\" (UID: \"af4ee930-6db5-4c5c-9727-1ca855360b34\") " pod="openshift-ingress/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-xbr94"
Oct 30 22:12:18.204860 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:18.204806    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/af4ee930-6db5-4c5c-9727-1ca855360b34-istio-token\") pod \"e2e-worker-w1-a00d9b-openshift-default-847df8fc59-xbr94\" (UID: \"af4ee930-6db5-4c5c-9727-1ca855360b34\") " pod="openshift-ingress/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-xbr94"
Oct 30 22:12:18.205352 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:18.205307    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9g7vz\" (UniqueName: \"kubernetes.io/projected/af4ee930-6db5-4c5c-9727-1ca855360b34-kube-api-access-9g7vz\") pod \"e2e-worker-w1-a00d9b-openshift-default-847df8fc59-xbr94\" (UID: \"af4ee930-6db5-4c5c-9727-1ca855360b34\") " pod="openshift-ingress/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-xbr94"
Oct 30 22:12:18.266695 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:18.263462    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-xbr94"
Oct 30 22:12:18.573992 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:18.573959    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-m2c8l"]
Oct 30 22:12:18.574413 ip-10-0-136-210 kubenswrapper[5510]: W1030 22:12:18.574389    5510 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4798995a_2305_4dae_adf7_45aa66e0cb18.slice/crio-dd4108f2006d0df98cdd4f005e4ac484549b45a0e4a191de3895412e1047e59e WatchSource:0}: Error finding container dd4108f2006d0df98cdd4f005e4ac484549b45a0e4a191de3895412e1047e59e: Status 404 returned error can't find the container with id dd4108f2006d0df98cdd4f005e4ac484549b45a0e4a191de3895412e1047e59e
Oct 30 22:12:18.576929 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:18.576894    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:12:18.577029 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:18.576966    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:12:18.577029 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:18.576997    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:12:18.670160 ip-10-0-136-210 kubenswrapper[5510]: W1030 22:12:18.670119    5510 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf4ee930_6db5_4c5c_9727_1ca855360b34.slice/crio-a1edd121ab99acf681dd5503aca8b5edffc0e491c9c0a77c11a0a89f84469978 WatchSource:0}: Error finding container a1edd121ab99acf681dd5503aca8b5edffc0e491c9c0a77c11a0a89f84469978: Status 404 returned error can't find the container with id a1edd121ab99acf681dd5503aca8b5edffc0e491c9c0a77c11a0a89f84469978
Oct 30 22:12:18.671770 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:18.671731    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-xbr94"]
Oct 30 22:12:18.673473 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:18.673426    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:12:18.673592 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:18.673525    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:12:18.673592 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:18.673572    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:12:19.201444 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:19.201404    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-xbr94" event={"ID":"af4ee930-6db5-4c5c-9727-1ca855360b34","Type":"ContainerStarted","Data":"445c0c4f1af25dc592c4a25f8f5a08758b91b16fb928dd8ae7996c4bc64f0544"}
Oct 30 22:12:19.203122 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:19.203088    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-xbr94" event={"ID":"af4ee930-6db5-4c5c-9727-1ca855360b34","Type":"ContainerStarted","Data":"a1edd121ab99acf681dd5503aca8b5edffc0e491c9c0a77c11a0a89f84469978"}
Oct 30 22:12:19.205571 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:19.205542    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-m2c8l" event={"ID":"4798995a-2305-4dae-adf7-45aa66e0cb18","Type":"ContainerStarted","Data":"23d5aae41a6a6b3345ee01975c2b297980a9c3b6129f27ad2156d594c524e4cd"}
Oct 30 22:12:19.205814 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:19.205793    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-m2c8l" event={"ID":"4798995a-2305-4dae-adf7-45aa66e0cb18","Type":"ContainerStarted","Data":"dd4108f2006d0df98cdd4f005e4ac484549b45a0e4a191de3895412e1047e59e"}
Oct 30 22:12:19.294193 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:19.294113    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-xbr94"
Oct 30 22:12:19.297183 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:19.296800    5510 patch_prober.go:29] interesting pod/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-xbr94 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body=
Oct 30 22:12:19.297183 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:19.296860    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-xbr94" podUID="af4ee930-6db5-4c5c-9727-1ca855360b34" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused"
Oct 30 22:12:19.505720 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:19.505571    5510 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-m2c8l" podStartSLOduration=2.505553797 podStartE2EDuration="2.505553797s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:12:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:12:19.504463087 +0000 UTC m=+1320.065273092" watchObservedRunningTime="2026-10-30 22:12:19.505553797 +0000 UTC m=+1320.066363804"
Oct 30 22:12:19.508391 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:19.508355    5510 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-xbr94" podStartSLOduration=2.508342015 podStartE2EDuration="2.508342015s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:12:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:12:19.375634705 +0000 UTC m=+1319.936444734" watchObservedRunningTime="2026-10-30 22:12:19.508342015 +0000 UTC m=+1320.069152024"
Oct 30 22:12:20.140063 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:20.140026    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-m2c8l"
Oct 30 22:12:20.141477 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:20.141453    5510 patch_prober.go:29] interesting pod/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-m2c8l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" start-of-body=
Oct 30 22:12:20.141581 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:20.141502    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-m2c8l" podUID="4798995a-2305-4dae-adf7-45aa66e0cb18" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused"
Oct 30 22:12:20.266698 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:20.266058    5510 patch_prober.go:29] interesting pod/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-xbr94 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body=
Oct 30 22:12:20.266698 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:20.266118    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-xbr94" podUID="af4ee930-6db5-4c5c-9727-1ca855360b34" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused"
Oct 30 22:12:21.140511 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:21.140472    5510 patch_prober.go:29] interesting pod/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-m2c8l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" start-of-body=
Oct 30 22:12:21.140751 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:21.140531    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-m2c8l" podUID="4798995a-2305-4dae-adf7-45aa66e0cb18" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused"
Oct 30 22:12:21.271738 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:21.266117    5510 patch_prober.go:29] interesting pod/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-xbr94 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body=
Oct 30 22:12:21.271738 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:21.266180    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-xbr94" podUID="af4ee930-6db5-4c5c-9727-1ca855360b34" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused"
Oct 30 22:12:21.280684 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:21.272946    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:12:22.139814 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:22.139779    5510 patch_prober.go:29] interesting pod/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-m2c8l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" start-of-body=
Oct 30 22:12:22.140000 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:22.139832    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-m2c8l" podUID="4798995a-2305-4dae-adf7-45aa66e0cb18" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused"
Oct 30 22:12:22.266900 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:22.266857    5510 patch_prober.go:29] interesting pod/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-xbr94 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body=
Oct 30 22:12:22.267152 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:22.267120    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-xbr94" podUID="af4ee930-6db5-4c5c-9727-1ca855360b34" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused"
Oct 30 22:12:23.140405 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:23.140368    5510 patch_prober.go:29] interesting pod/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-m2c8l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" start-of-body=
Oct 30 22:12:23.140820 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:23.140429    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-m2c8l" podUID="4798995a-2305-4dae-adf7-45aa66e0cb18" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused"
Oct 30 22:12:23.269680 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:23.269327    5510 patch_prober.go:29] interesting pod/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-xbr94 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body=
Oct 30 22:12:23.269680 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:23.269391    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-xbr94" podUID="af4ee930-6db5-4c5c-9727-1ca855360b34" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused"
Oct 30 22:12:24.140117 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:24.140087    5510 patch_prober.go:29] interesting pod/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-m2c8l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" start-of-body=
Oct 30 22:12:24.140363 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:24.140340    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-m2c8l" podUID="4798995a-2305-4dae-adf7-45aa66e0cb18" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused"
Oct 30 22:12:24.271680 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:24.270821    5510 patch_prober.go:29] interesting pod/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-xbr94 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body=
Oct 30 22:12:24.271680 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:24.270887    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-xbr94" podUID="af4ee930-6db5-4c5c-9727-1ca855360b34" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused"
Oct 30 22:12:25.140059 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:25.140017    5510 patch_prober.go:29] interesting pod/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-m2c8l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" start-of-body=
Oct 30 22:12:25.140222 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:25.140073    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-m2c8l" podUID="4798995a-2305-4dae-adf7-45aa66e0cb18" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused"
Oct 30 22:12:25.226092 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:25.226054    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:12:25.264616 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:25.264574    5510 patch_prober.go:29] interesting pod/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-xbr94 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body=
Oct 30 22:12:25.264802 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:25.264644    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-xbr94" podUID="af4ee930-6db5-4c5c-9727-1ca855360b34" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused"
Oct 30 22:12:25.479224 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:25.479133    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:12:26.166225 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:26.166182    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-m2c8l"
Oct 30 22:12:26.171060 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:26.171033    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-m2c8l"
Oct 30 22:12:26.171600 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:26.171579    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-m2c8l"
Oct 30 22:12:26.195774 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:26.193824    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:12:26.268555 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:26.268528    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-xbr94"
Oct 30 22:12:26.268865 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:26.268841    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-xbr94"
Oct 30 22:12:26.269442 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:26.269424    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-xbr94"
Oct 30 22:12:31.081679 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:31.080881    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:12:33.660274 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:33.660240    5510 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w3-7bb1c2/distinct-2-w3-7bb1c2-kserve-c549df844-zt595"]
Oct 30 22:12:33.665946 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:33.665926    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-7bb1c2/distinct-2-w3-7bb1c2-kserve-c549df844-zt595"
Oct 30 22:12:33.690252 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:33.690219    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1916e87c-a1a4-4967-8640-8bbcd9d12903-tmp-dir\") pod \"distinct-2-w3-7bb1c2-kserve-c549df844-zt595\" (UID: \"1916e87c-a1a4-4967-8640-8bbcd9d12903\") " pod="e2e-models-e2e-worker-w3-7bb1c2/distinct-2-w3-7bb1c2-kserve-c549df844-zt595"
Oct 30 22:12:33.690413 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:33.690290    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1916e87c-a1a4-4967-8640-8bbcd9d12903-home\") pod \"distinct-2-w3-7bb1c2-kserve-c549df844-zt595\" (UID: \"1916e87c-a1a4-4967-8640-8bbcd9d12903\") " pod="e2e-models-e2e-worker-w3-7bb1c2/distinct-2-w3-7bb1c2-kserve-c549df844-zt595"
Oct 30 22:12:33.690413 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:33.690373    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1916e87c-a1a4-4967-8640-8bbcd9d12903-tls-certs\") pod \"distinct-2-w3-7bb1c2-kserve-c549df844-zt595\" (UID: \"1916e87c-a1a4-4967-8640-8bbcd9d12903\") " pod="e2e-models-e2e-worker-w3-7bb1c2/distinct-2-w3-7bb1c2-kserve-c549df844-zt595"
Oct 30 22:12:33.690482 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:33.690472    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1916e87c-a1a4-4967-8640-8bbcd9d12903-model-cache\") pod \"distinct-2-w3-7bb1c2-kserve-c549df844-zt595\" (UID: \"1916e87c-a1a4-4967-8640-8bbcd9d12903\") " pod="e2e-models-e2e-worker-w3-7bb1c2/distinct-2-w3-7bb1c2-kserve-c549df844-zt595"
Oct 30 22:12:33.690629 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:33.690605    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1916e87c-a1a4-4967-8640-8bbcd9d12903-dshm\") pod \"distinct-2-w3-7bb1c2-kserve-c549df844-zt595\" (UID: \"1916e87c-a1a4-4967-8640-8bbcd9d12903\") " pod="e2e-models-e2e-worker-w3-7bb1c2/distinct-2-w3-7bb1c2-kserve-c549df844-zt595"
Oct 30 22:12:33.690722 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:33.690704    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6k57f\" (UniqueName: \"kubernetes.io/projected/1916e87c-a1a4-4967-8640-8bbcd9d12903-kube-api-access-6k57f\") pod \"distinct-2-w3-7bb1c2-kserve-c549df844-zt595\" (UID: \"1916e87c-a1a4-4967-8640-8bbcd9d12903\") " pod="e2e-models-e2e-worker-w3-7bb1c2/distinct-2-w3-7bb1c2-kserve-c549df844-zt595"
Oct 30 22:12:33.770875 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:33.770830    5510 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w3-7bb1c2\"/\"openshift-service-ca.crt\""
Oct 30 22:12:33.770875 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:33.770833    5510 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w3-7bb1c2\"/\"kube-root-ca.crt\""
Oct 30 22:12:33.771072 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:33.770906    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-7bb1c2\"/\"distinct-2-w3-7bb1c2-kserve-self-signed-certs\""
Oct 30 22:12:33.771072 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:33.770991    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-7bb1c2\"/\"default-dockercfg-jf9c9\""
Oct 30 22:12:33.774119 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:33.774100    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-7bb1c2/distinct-2-w3-7bb1c2-kserve-c549df844-zt595"]
Oct 30 22:12:33.851431 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:33.839796    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:12:33.851431 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:33.845119    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1916e87c-a1a4-4967-8640-8bbcd9d12903-home\") pod \"distinct-2-w3-7bb1c2-kserve-c549df844-zt595\" (UID: \"1916e87c-a1a4-4967-8640-8bbcd9d12903\") " pod="e2e-models-e2e-worker-w3-7bb1c2/distinct-2-w3-7bb1c2-kserve-c549df844-zt595"
Oct 30 22:12:33.858692 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:33.828900    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1916e87c-a1a4-4967-8640-8bbcd9d12903-home\") pod \"distinct-2-w3-7bb1c2-kserve-c549df844-zt595\" (UID: \"1916e87c-a1a4-4967-8640-8bbcd9d12903\") " pod="e2e-models-e2e-worker-w3-7bb1c2/distinct-2-w3-7bb1c2-kserve-c549df844-zt595"
Oct 30 22:12:33.858692 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:33.854866    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1916e87c-a1a4-4967-8640-8bbcd9d12903-tls-certs\") pod \"distinct-2-w3-7bb1c2-kserve-c549df844-zt595\" (UID: \"1916e87c-a1a4-4967-8640-8bbcd9d12903\") " pod="e2e-models-e2e-worker-w3-7bb1c2/distinct-2-w3-7bb1c2-kserve-c549df844-zt595"
Oct 30 22:12:33.858692 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:33.855090    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1916e87c-a1a4-4967-8640-8bbcd9d12903-model-cache\") pod \"distinct-2-w3-7bb1c2-kserve-c549df844-zt595\" (UID: \"1916e87c-a1a4-4967-8640-8bbcd9d12903\") " pod="e2e-models-e2e-worker-w3-7bb1c2/distinct-2-w3-7bb1c2-kserve-c549df844-zt595"
Oct 30 22:12:33.858692 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:33.855225    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1916e87c-a1a4-4967-8640-8bbcd9d12903-dshm\") pod \"distinct-2-w3-7bb1c2-kserve-c549df844-zt595\" (UID: \"1916e87c-a1a4-4967-8640-8bbcd9d12903\") " pod="e2e-models-e2e-worker-w3-7bb1c2/distinct-2-w3-7bb1c2-kserve-c549df844-zt595"
Oct 30 22:12:33.858692 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:33.855305    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6k57f\" (UniqueName: \"kubernetes.io/projected/1916e87c-a1a4-4967-8640-8bbcd9d12903-kube-api-access-6k57f\") pod \"distinct-2-w3-7bb1c2-kserve-c549df844-zt595\" (UID: \"1916e87c-a1a4-4967-8640-8bbcd9d12903\") " pod="e2e-models-e2e-worker-w3-7bb1c2/distinct-2-w3-7bb1c2-kserve-c549df844-zt595"
Oct 30 22:12:33.858692 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:33.855457    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1916e87c-a1a4-4967-8640-8bbcd9d12903-tmp-dir\") pod \"distinct-2-w3-7bb1c2-kserve-c549df844-zt595\" (UID: \"1916e87c-a1a4-4967-8640-8bbcd9d12903\") " pod="e2e-models-e2e-worker-w3-7bb1c2/distinct-2-w3-7bb1c2-kserve-c549df844-zt595"
Oct 30 22:12:33.858692 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:33.856133    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1916e87c-a1a4-4967-8640-8bbcd9d12903-tmp-dir\") pod \"distinct-2-w3-7bb1c2-kserve-c549df844-zt595\" (UID: \"1916e87c-a1a4-4967-8640-8bbcd9d12903\") " pod="e2e-models-e2e-worker-w3-7bb1c2/distinct-2-w3-7bb1c2-kserve-c549df844-zt595"
Oct 30 22:12:33.863268 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:33.860016    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1916e87c-a1a4-4967-8640-8bbcd9d12903-model-cache\") pod \"distinct-2-w3-7bb1c2-kserve-c549df844-zt595\" (UID: \"1916e87c-a1a4-4967-8640-8bbcd9d12903\") " pod="e2e-models-e2e-worker-w3-7bb1c2/distinct-2-w3-7bb1c2-kserve-c549df844-zt595"
Oct 30 22:12:33.877772 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:33.874828    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1916e87c-a1a4-4967-8640-8bbcd9d12903-tls-certs\") pod \"distinct-2-w3-7bb1c2-kserve-c549df844-zt595\" (UID: \"1916e87c-a1a4-4967-8640-8bbcd9d12903\") " pod="e2e-models-e2e-worker-w3-7bb1c2/distinct-2-w3-7bb1c2-kserve-c549df844-zt595"
Oct 30 22:12:33.891556 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:33.891392    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1916e87c-a1a4-4967-8640-8bbcd9d12903-dshm\") pod \"distinct-2-w3-7bb1c2-kserve-c549df844-zt595\" (UID: \"1916e87c-a1a4-4967-8640-8bbcd9d12903\") " pod="e2e-models-e2e-worker-w3-7bb1c2/distinct-2-w3-7bb1c2-kserve-c549df844-zt595"
Oct 30 22:12:33.925046 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:33.924975    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6k57f\" (UniqueName: \"kubernetes.io/projected/1916e87c-a1a4-4967-8640-8bbcd9d12903-kube-api-access-6k57f\") pod \"distinct-2-w3-7bb1c2-kserve-c549df844-zt595\" (UID: \"1916e87c-a1a4-4967-8640-8bbcd9d12903\") " pod="e2e-models-e2e-worker-w3-7bb1c2/distinct-2-w3-7bb1c2-kserve-c549df844-zt595"
Oct 30 22:12:33.978369 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:33.978331    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-7bb1c2/distinct-2-w3-7bb1c2-kserve-c549df844-zt595"
Oct 30 22:12:34.518517 ip-10-0-136-210 kubenswrapper[5510]: W1030 22:12:34.518483    5510 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1916e87c_a1a4_4967_8640_8bbcd9d12903.slice/crio-4a3643dba5c1c2fbbf23c448892fe922ba8f446d0bc057c19c31b7f2842769f8 WatchSource:0}: Error finding container 4a3643dba5c1c2fbbf23c448892fe922ba8f446d0bc057c19c31b7f2842769f8: Status 404 returned error can't find the container with id 4a3643dba5c1c2fbbf23c448892fe922ba8f446d0bc057c19c31b7f2842769f8
Oct 30 22:12:34.522137 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:34.522110    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-7bb1c2/distinct-2-w3-7bb1c2-kserve-c549df844-zt595"]
Oct 30 22:12:35.380542 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:35.380501    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-7bb1c2/distinct-2-w3-7bb1c2-kserve-c549df844-zt595" event={"ID":"1916e87c-a1a4-4967-8640-8bbcd9d12903","Type":"ContainerStarted","Data":"f5eb22bd766f1da34b151a1610d37933dbce4a9bd1b928491eac465a30021bea"}
Oct 30 22:12:35.380941 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:35.380550    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-7bb1c2/distinct-2-w3-7bb1c2-kserve-c549df844-zt595" event={"ID":"1916e87c-a1a4-4967-8640-8bbcd9d12903","Type":"ContainerStarted","Data":"4a3643dba5c1c2fbbf23c448892fe922ba8f446d0bc057c19c31b7f2842769f8"}
Oct 30 22:12:35.502845 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:35.502776    5510 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w3-7bb1c2/distinct-2-w3-7bb1c2-kserve-c549df844-zt595" podStartSLOduration=2.502762106 podStartE2EDuration="2.502762106s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:12:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:12:35.500050459 +0000 UTC m=+1336.060860477" watchObservedRunningTime="2026-10-30 22:12:35.502762106 +0000 UTC m=+1336.063572110"
Oct 30 22:12:35.827347 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:35.816285    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:12:35.979029 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:35.978997    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-7bb1c2/distinct-2-w3-7bb1c2-kserve-c549df844-zt595"
Oct 30 22:12:38.420606 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:12:38.420563    5510 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6gjbf" containerName="extractor"
Oct 30 22:12:40.420402 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:40.420354    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:12:42.660480 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:42.660341    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:12:43.979126 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:43.979095    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w3-7bb1c2/distinct-2-w3-7bb1c2-kserve-c549df844-zt595"
Oct 30 22:12:43.992927 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:43.992901    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w3-7bb1c2/distinct-2-w3-7bb1c2-kserve-c549df844-zt595"
Oct 30 22:12:44.504707 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:44.504648    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-7bb1c2/distinct-2-w3-7bb1c2-kserve-c549df844-zt595"
Oct 30 22:12:45.464705 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:45.458552    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:12:47.825680 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:47.825005    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:12:50.107684 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:50.098300    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:12:53.381924 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:53.381887    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:12:54.726723 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:54.726352    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:12:56.418284 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:56.418252    5510 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-779f32/distinct-2-w0-779f32-kserve-857ffdf778-gk2h9"]
Oct 30 22:12:56.422668 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:56.422633    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-779f32/distinct-2-w0-779f32-kserve-857ffdf778-gk2h9"
Oct 30 22:12:56.487062 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:56.487028    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-779f32/distinct-2-w0-779f32-kserve-857ffdf778-gk2h9"]
Oct 30 22:12:56.532819 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:56.532791    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-779f32\"/\"distinct-2-w0-779f32-kserve-self-signed-certs\""
Oct 30 22:12:56.585720 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:56.581919    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ef500cdd-cc52-45b0-8dd7-1896897acc46-tmp-dir\") pod \"distinct-2-w0-779f32-kserve-857ffdf778-gk2h9\" (UID: \"ef500cdd-cc52-45b0-8dd7-1896897acc46\") " pod="e2e-models-e2e-worker-w0-779f32/distinct-2-w0-779f32-kserve-857ffdf778-gk2h9"
Oct 30 22:12:56.586061 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:56.586027    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ef500cdd-cc52-45b0-8dd7-1896897acc46-home\") pod \"distinct-2-w0-779f32-kserve-857ffdf778-gk2h9\" (UID: \"ef500cdd-cc52-45b0-8dd7-1896897acc46\") " pod="e2e-models-e2e-worker-w0-779f32/distinct-2-w0-779f32-kserve-857ffdf778-gk2h9"
Oct 30 22:12:56.586252 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:56.586236    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ef500cdd-cc52-45b0-8dd7-1896897acc46-dshm\") pod \"distinct-2-w0-779f32-kserve-857ffdf778-gk2h9\" (UID: \"ef500cdd-cc52-45b0-8dd7-1896897acc46\") " pod="e2e-models-e2e-worker-w0-779f32/distinct-2-w0-779f32-kserve-857ffdf778-gk2h9"
Oct 30 22:12:56.586626 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:56.586596    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ef500cdd-cc52-45b0-8dd7-1896897acc46-model-cache\") pod \"distinct-2-w0-779f32-kserve-857ffdf778-gk2h9\" (UID: \"ef500cdd-cc52-45b0-8dd7-1896897acc46\") " pod="e2e-models-e2e-worker-w0-779f32/distinct-2-w0-779f32-kserve-857ffdf778-gk2h9"
Oct 30 22:12:56.587967 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:56.587933    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ef500cdd-cc52-45b0-8dd7-1896897acc46-tls-certs\") pod \"distinct-2-w0-779f32-kserve-857ffdf778-gk2h9\" (UID: \"ef500cdd-cc52-45b0-8dd7-1896897acc46\") " pod="e2e-models-e2e-worker-w0-779f32/distinct-2-w0-779f32-kserve-857ffdf778-gk2h9"
Oct 30 22:12:56.588181 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:56.588159    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q8dnw\" (UniqueName: \"kubernetes.io/projected/ef500cdd-cc52-45b0-8dd7-1896897acc46-kube-api-access-q8dnw\") pod \"distinct-2-w0-779f32-kserve-857ffdf778-gk2h9\" (UID: \"ef500cdd-cc52-45b0-8dd7-1896897acc46\") " pod="e2e-models-e2e-worker-w0-779f32/distinct-2-w0-779f32-kserve-857ffdf778-gk2h9"
Oct 30 22:12:56.632379 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:56.624974    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:12:56.690069 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:56.689992    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ef500cdd-cc52-45b0-8dd7-1896897acc46-model-cache\") pod \"distinct-2-w0-779f32-kserve-857ffdf778-gk2h9\" (UID: \"ef500cdd-cc52-45b0-8dd7-1896897acc46\") " pod="e2e-models-e2e-worker-w0-779f32/distinct-2-w0-779f32-kserve-857ffdf778-gk2h9"
Oct 30 22:12:56.690224 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:56.690126    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ef500cdd-cc52-45b0-8dd7-1896897acc46-tls-certs\") pod \"distinct-2-w0-779f32-kserve-857ffdf778-gk2h9\" (UID: \"ef500cdd-cc52-45b0-8dd7-1896897acc46\") " pod="e2e-models-e2e-worker-w0-779f32/distinct-2-w0-779f32-kserve-857ffdf778-gk2h9"
Oct 30 22:12:56.690335 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:56.690312    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q8dnw\" (UniqueName: \"kubernetes.io/projected/ef500cdd-cc52-45b0-8dd7-1896897acc46-kube-api-access-q8dnw\") pod \"distinct-2-w0-779f32-kserve-857ffdf778-gk2h9\" (UID: \"ef500cdd-cc52-45b0-8dd7-1896897acc46\") " pod="e2e-models-e2e-worker-w0-779f32/distinct-2-w0-779f32-kserve-857ffdf778-gk2h9"
Oct 30 22:12:56.690381 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:56.690359    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ef500cdd-cc52-45b0-8dd7-1896897acc46-tmp-dir\") pod \"distinct-2-w0-779f32-kserve-857ffdf778-gk2h9\" (UID: \"ef500cdd-cc52-45b0-8dd7-1896897acc46\") " pod="e2e-models-e2e-worker-w0-779f32/distinct-2-w0-779f32-kserve-857ffdf778-gk2h9"
Oct 30 22:12:56.690434 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:56.690382    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ef500cdd-cc52-45b0-8dd7-1896897acc46-model-cache\") pod \"distinct-2-w0-779f32-kserve-857ffdf778-gk2h9\" (UID: \"ef500cdd-cc52-45b0-8dd7-1896897acc46\") " pod="e2e-models-e2e-worker-w0-779f32/distinct-2-w0-779f32-kserve-857ffdf778-gk2h9"
Oct 30 22:12:56.690434 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:56.690411    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ef500cdd-cc52-45b0-8dd7-1896897acc46-home\") pod \"distinct-2-w0-779f32-kserve-857ffdf778-gk2h9\" (UID: \"ef500cdd-cc52-45b0-8dd7-1896897acc46\") " pod="e2e-models-e2e-worker-w0-779f32/distinct-2-w0-779f32-kserve-857ffdf778-gk2h9"
Oct 30 22:12:56.690509 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:56.690457    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ef500cdd-cc52-45b0-8dd7-1896897acc46-dshm\") pod \"distinct-2-w0-779f32-kserve-857ffdf778-gk2h9\" (UID: \"ef500cdd-cc52-45b0-8dd7-1896897acc46\") " pod="e2e-models-e2e-worker-w0-779f32/distinct-2-w0-779f32-kserve-857ffdf778-gk2h9"
Oct 30 22:12:56.690639 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:56.690616    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ef500cdd-cc52-45b0-8dd7-1896897acc46-tmp-dir\") pod \"distinct-2-w0-779f32-kserve-857ffdf778-gk2h9\" (UID: \"ef500cdd-cc52-45b0-8dd7-1896897acc46\") " pod="e2e-models-e2e-worker-w0-779f32/distinct-2-w0-779f32-kserve-857ffdf778-gk2h9"
Oct 30 22:12:56.690832 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:56.690757    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ef500cdd-cc52-45b0-8dd7-1896897acc46-home\") pod \"distinct-2-w0-779f32-kserve-857ffdf778-gk2h9\" (UID: \"ef500cdd-cc52-45b0-8dd7-1896897acc46\") " pod="e2e-models-e2e-worker-w0-779f32/distinct-2-w0-779f32-kserve-857ffdf778-gk2h9"
Oct 30 22:12:56.692644 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:56.692622    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ef500cdd-cc52-45b0-8dd7-1896897acc46-dshm\") pod \"distinct-2-w0-779f32-kserve-857ffdf778-gk2h9\" (UID: \"ef500cdd-cc52-45b0-8dd7-1896897acc46\") " pod="e2e-models-e2e-worker-w0-779f32/distinct-2-w0-779f32-kserve-857ffdf778-gk2h9"
Oct 30 22:12:56.692837 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:56.692821    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ef500cdd-cc52-45b0-8dd7-1896897acc46-tls-certs\") pod \"distinct-2-w0-779f32-kserve-857ffdf778-gk2h9\" (UID: \"ef500cdd-cc52-45b0-8dd7-1896897acc46\") " pod="e2e-models-e2e-worker-w0-779f32/distinct-2-w0-779f32-kserve-857ffdf778-gk2h9"
Oct 30 22:12:56.756723 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:56.756636    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q8dnw\" (UniqueName: \"kubernetes.io/projected/ef500cdd-cc52-45b0-8dd7-1896897acc46-kube-api-access-q8dnw\") pod \"distinct-2-w0-779f32-kserve-857ffdf778-gk2h9\" (UID: \"ef500cdd-cc52-45b0-8dd7-1896897acc46\") " pod="e2e-models-e2e-worker-w0-779f32/distinct-2-w0-779f32-kserve-857ffdf778-gk2h9"
Oct 30 22:12:57.047956 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:57.047916    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-779f32/distinct-2-w0-779f32-kserve-857ffdf778-gk2h9"
Oct 30 22:12:57.425755 ip-10-0-136-210 kubenswrapper[5510]: W1030 22:12:57.425723    5510 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podef500cdd_cc52_45b0_8dd7_1896897acc46.slice/crio-a919144f88571a79b3353edeeb74e17fe00197b109946f8c4aea2a6eddabe57f WatchSource:0}: Error finding container a919144f88571a79b3353edeeb74e17fe00197b109946f8c4aea2a6eddabe57f: Status 404 returned error can't find the container with id a919144f88571a79b3353edeeb74e17fe00197b109946f8c4aea2a6eddabe57f
Oct 30 22:12:57.429041 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:57.429015    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-779f32/distinct-2-w0-779f32-kserve-857ffdf778-gk2h9"]
Oct 30 22:12:57.634336 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:57.634290    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-779f32/distinct-2-w0-779f32-kserve-857ffdf778-gk2h9" event={"ID":"ef500cdd-cc52-45b0-8dd7-1896897acc46","Type":"ContainerStarted","Data":"9767c12917b25a2f77eb31d59b982a4c5b1b851c6d51350581ef02f47249de27"}
Oct 30 22:12:57.634509 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:57.634344    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-779f32/distinct-2-w0-779f32-kserve-857ffdf778-gk2h9" event={"ID":"ef500cdd-cc52-45b0-8dd7-1896897acc46","Type":"ContainerStarted","Data":"a919144f88571a79b3353edeeb74e17fe00197b109946f8c4aea2a6eddabe57f"}
Oct 30 22:12:57.758342 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:57.758229    5510 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-779f32/distinct-2-w0-779f32-kserve-857ffdf778-gk2h9" podStartSLOduration=1.758212399 podStartE2EDuration="1.758212399s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:12:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:12:57.755027795 +0000 UTC m=+1358.315837801" watchObservedRunningTime="2026-10-30 22:12:57.758212399 +0000 UTC m=+1358.319022412"
Oct 30 22:12:58.048237 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:58.048200    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-779f32/distinct-2-w0-779f32-kserve-857ffdf778-gk2h9"
Oct 30 22:12:59.522754 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:12:59.522719    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:13:00.431056 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:00.431017    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:13:01.551679 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:01.550763    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-c9dbe6/distinct-w5-c9dbe6-kserve-7599fd86d9-bdhxd" podUID="e903db90-f2cd-4244-b00b-f99b21873f6e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:13:02.018823 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:02.018779    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-bc714d/distinct-w2-bc714d-kserve-66cfb968dd-jvgqj" podUID="9e9b11b8-1cc9-4fc8-8097-1afcffa8c422" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:13:02.112111 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:02.112066    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-779f32/distinct-w0-779f32-kserve-6dcbc948c7-2vzdf" podUID="e17cedd4-7d48-42b3-b127-b6192c1b3795" containerName="main" probeResult="failure" output="Get \"https://10.133.0.65:8000/ready\": context deadline exceeded"
Oct 30 22:13:02.178943 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:02.177822    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-fb0b93/premium-w6-fb0b93-kserve-5cb9948c54-w7pnk" podUID="290ae7a2-de37-4af3-82ae-a8e855b83bb9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:13:02.411790 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:02.411687    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-fb0b93/distinct-w6-fb0b93-kserve-58db6b6757-6mww8" podUID="8ee6579b-acea-4492-862b-9883f70d00c9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:13:02.497682 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:02.469003    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-7bb1c2/distinct-2-w3-7bb1c2-kserve-c549df844-zt595" podUID="1916e87c-a1a4-4967-8640-8bbcd9d12903" containerName="main" probeResult="failure" output="Get \"https://10.133.0.73:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:13:02.552053 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:02.552009    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-c9dbe6/distinct-w5-c9dbe6-kserve-7599fd86d9-bdhxd" podUID="e903db90-f2cd-4244-b00b-f99b21873f6e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:13:02.553053 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:02.553031    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-c9dbe6/distinct-w5-c9dbe6-kserve-7599fd86d9-bdhxd"
Oct 30 22:13:03.026889 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:03.026831    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-bc714d/distinct-w2-bc714d-kserve-66cfb968dd-jvgqj" podUID="9e9b11b8-1cc9-4fc8-8097-1afcffa8c422" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": context deadline exceeded"
Oct 30 22:13:03.030693 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:03.027806    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-bc714d/distinct-w2-bc714d-kserve-66cfb968dd-jvgqj"
Oct 30 22:13:03.114176 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:03.114126    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-779f32/distinct-w0-779f32-kserve-6dcbc948c7-2vzdf" podUID="e17cedd4-7d48-42b3-b127-b6192c1b3795" containerName="main" probeResult="failure" output="Get \"https://10.133.0.65:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:13:03.133407 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:03.115105    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-779f32/distinct-w0-779f32-kserve-6dcbc948c7-2vzdf"
Oct 30 22:13:03.356739 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:03.187356    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-fb0b93/premium-w6-fb0b93-kserve-5cb9948c54-w7pnk" podUID="290ae7a2-de37-4af3-82ae-a8e855b83bb9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:13:03.378479 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:03.378445    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-fb0b93/premium-w6-fb0b93-kserve-5cb9948c54-w7pnk"
Oct 30 22:13:03.412920 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:03.412878    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-fb0b93/distinct-w6-fb0b93-kserve-58db6b6757-6mww8" podUID="8ee6579b-acea-4492-862b-9883f70d00c9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.59:8000/ready\": context deadline exceeded"
Oct 30 22:13:03.413728 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:03.413705    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-fb0b93/distinct-w6-fb0b93-kserve-58db6b6757-6mww8"
Oct 30 22:13:03.595308 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:03.595264    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-c9dbe6/distinct-w5-c9dbe6-kserve-7599fd86d9-bdhxd" podUID="e903db90-f2cd-4244-b00b-f99b21873f6e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:13:03.890186 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:03.867849    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-fb0b93/distinct-w6-fb0b93-kserve-58db6b6757-6mww8"
Oct 30 22:13:04.028755 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:04.028717    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-fb0b93/premium-w6-fb0b93-kserve-5cb9948c54-w7pnk"
Oct 30 22:13:04.030590 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:04.030561    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-779f32/distinct-w0-779f32-kserve-6dcbc948c7-2vzdf"
Oct 30 22:13:04.030878 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:04.030859    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-c9dbe6/distinct-w5-c9dbe6-kserve-7599fd86d9-bdhxd"
Oct 30 22:13:04.031247 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:04.029863    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-bc714d/distinct-w2-bc714d-kserve-66cfb968dd-jvgqj" podUID="9e9b11b8-1cc9-4fc8-8097-1afcffa8c422" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:13:04.088149 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:04.088121    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-bc714d/distinct-w2-bc714d-kserve-66cfb968dd-jvgqj"
Oct 30 22:13:05.304677 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:05.303927    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:13:05.560611 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:05.560519    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:13:07.049115 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:07.049077    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-779f32/distinct-2-w0-779f32-kserve-857ffdf778-gk2h9"
Oct 30 22:13:07.061808 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:07.061779    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-779f32/distinct-2-w0-779f32-kserve-857ffdf778-gk2h9"
Oct 30 22:13:07.205779 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:07.205749    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-779f32/distinct-2-w0-779f32-kserve-857ffdf778-gk2h9"
Oct 30 22:13:08.420230 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:13:08.420197    5510 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6gjbf" containerName="extractor"
Oct 30 22:13:11.366414 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:11.366373    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:13:14.676717 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:14.658980    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:13:17.822974 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:17.822941    5510 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-q6kh7"]
Oct 30 22:13:17.827301 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:17.827285    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-q6kh7"
Oct 30 22:13:17.930179 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:17.930141    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3c82a381-05ad-436f-89dd-0985c3dbad33-istio-podinfo\") pod \"e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-q6kh7\" (UID: \"3c82a381-05ad-436f-89dd-0985c3dbad33\") " pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-q6kh7"
Oct 30 22:13:17.930370 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:17.930215    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3c82a381-05ad-436f-89dd-0985c3dbad33-istio-token\") pod \"e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-q6kh7\" (UID: \"3c82a381-05ad-436f-89dd-0985c3dbad33\") " pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-q6kh7"
Oct 30 22:13:17.930370 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:17.930267    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3c82a381-05ad-436f-89dd-0985c3dbad33-istio-envoy\") pod \"e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-q6kh7\" (UID: \"3c82a381-05ad-436f-89dd-0985c3dbad33\") " pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-q6kh7"
Oct 30 22:13:17.930496 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:17.930415    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c82a381-05ad-436f-89dd-0985c3dbad33-credential-socket\") pod \"e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-q6kh7\" (UID: \"3c82a381-05ad-436f-89dd-0985c3dbad33\") " pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-q6kh7"
Oct 30 22:13:17.930496 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:17.930463    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zl2qk\" (UniqueName: \"kubernetes.io/projected/3c82a381-05ad-436f-89dd-0985c3dbad33-kube-api-access-zl2qk\") pod \"e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-q6kh7\" (UID: \"3c82a381-05ad-436f-89dd-0985c3dbad33\") " pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-q6kh7"
Oct 30 22:13:17.930608 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:17.930575    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3c82a381-05ad-436f-89dd-0985c3dbad33-workload-certs\") pod \"e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-q6kh7\" (UID: \"3c82a381-05ad-436f-89dd-0985c3dbad33\") " pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-q6kh7"
Oct 30 22:13:17.930741 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:17.930726    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c82a381-05ad-436f-89dd-0985c3dbad33-workload-socket\") pod \"e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-q6kh7\" (UID: \"3c82a381-05ad-436f-89dd-0985c3dbad33\") " pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-q6kh7"
Oct 30 22:13:17.930785 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:17.930763    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3c82a381-05ad-436f-89dd-0985c3dbad33-istiod-ca-cert\") pod \"e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-q6kh7\" (UID: \"3c82a381-05ad-436f-89dd-0985c3dbad33\") " pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-q6kh7"
Oct 30 22:13:17.930898 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:17.930882    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3c82a381-05ad-436f-89dd-0985c3dbad33-istio-data\") pod \"e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-q6kh7\" (UID: \"3c82a381-05ad-436f-89dd-0985c3dbad33\") " pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-q6kh7"
Oct 30 22:13:18.033623 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:18.033588    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c82a381-05ad-436f-89dd-0985c3dbad33-workload-socket\") pod \"e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-q6kh7\" (UID: \"3c82a381-05ad-436f-89dd-0985c3dbad33\") " pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-q6kh7"
Oct 30 22:13:18.034004 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:18.033976    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3c82a381-05ad-436f-89dd-0985c3dbad33-istiod-ca-cert\") pod \"e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-q6kh7\" (UID: \"3c82a381-05ad-436f-89dd-0985c3dbad33\") " pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-q6kh7"
Oct 30 22:13:18.034216 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:18.034196    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3c82a381-05ad-436f-89dd-0985c3dbad33-istio-data\") pod \"e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-q6kh7\" (UID: \"3c82a381-05ad-436f-89dd-0985c3dbad33\") " pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-q6kh7"
Oct 30 22:13:18.034318 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:18.034257    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3c82a381-05ad-436f-89dd-0985c3dbad33-istio-podinfo\") pod \"e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-q6kh7\" (UID: \"3c82a381-05ad-436f-89dd-0985c3dbad33\") " pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-q6kh7"
Oct 30 22:13:18.034404 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:18.034348    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3c82a381-05ad-436f-89dd-0985c3dbad33-istio-token\") pod \"e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-q6kh7\" (UID: \"3c82a381-05ad-436f-89dd-0985c3dbad33\") " pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-q6kh7"
Oct 30 22:13:18.034404 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:18.034374    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3c82a381-05ad-436f-89dd-0985c3dbad33-istio-envoy\") pod \"e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-q6kh7\" (UID: \"3c82a381-05ad-436f-89dd-0985c3dbad33\") " pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-q6kh7"
Oct 30 22:13:18.034533 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:18.034468    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c82a381-05ad-436f-89dd-0985c3dbad33-credential-socket\") pod \"e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-q6kh7\" (UID: \"3c82a381-05ad-436f-89dd-0985c3dbad33\") " pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-q6kh7"
Oct 30 22:13:18.034533 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:18.034509    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zl2qk\" (UniqueName: \"kubernetes.io/projected/3c82a381-05ad-436f-89dd-0985c3dbad33-kube-api-access-zl2qk\") pod \"e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-q6kh7\" (UID: \"3c82a381-05ad-436f-89dd-0985c3dbad33\") " pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-q6kh7"
Oct 30 22:13:18.034632 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:18.034582    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3c82a381-05ad-436f-89dd-0985c3dbad33-workload-certs\") pod \"e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-q6kh7\" (UID: \"3c82a381-05ad-436f-89dd-0985c3dbad33\") " pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-q6kh7"
Oct 30 22:13:18.035069 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:18.035045    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3c82a381-05ad-436f-89dd-0985c3dbad33-workload-certs\") pod \"e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-q6kh7\" (UID: \"3c82a381-05ad-436f-89dd-0985c3dbad33\") " pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-q6kh7"
Oct 30 22:13:18.035336 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:18.035315    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3c82a381-05ad-436f-89dd-0985c3dbad33-istio-data\") pod \"e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-q6kh7\" (UID: \"3c82a381-05ad-436f-89dd-0985c3dbad33\") " pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-q6kh7"
Oct 30 22:13:18.037040 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:18.036988    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c82a381-05ad-436f-89dd-0985c3dbad33-credential-socket\") pod \"e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-q6kh7\" (UID: \"3c82a381-05ad-436f-89dd-0985c3dbad33\") " pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-q6kh7"
Oct 30 22:13:18.037322 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:18.037295    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c82a381-05ad-436f-89dd-0985c3dbad33-workload-socket\") pod \"e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-q6kh7\" (UID: \"3c82a381-05ad-436f-89dd-0985c3dbad33\") " pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-q6kh7"
Oct 30 22:13:18.043431 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:18.043407    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3c82a381-05ad-436f-89dd-0985c3dbad33-istiod-ca-cert\") pod \"e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-q6kh7\" (UID: \"3c82a381-05ad-436f-89dd-0985c3dbad33\") " pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-q6kh7"
Oct 30 22:13:18.045435 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:18.045402    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3c82a381-05ad-436f-89dd-0985c3dbad33-istio-podinfo\") pod \"e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-q6kh7\" (UID: \"3c82a381-05ad-436f-89dd-0985c3dbad33\") " pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-q6kh7"
Oct 30 22:13:18.047191 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:18.047167    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3c82a381-05ad-436f-89dd-0985c3dbad33-istio-envoy\") pod \"e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-q6kh7\" (UID: \"3c82a381-05ad-436f-89dd-0985c3dbad33\") " pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-q6kh7"
Oct 30 22:13:18.723538 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:18.723502    5510 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-48cnj"]
Oct 30 22:13:18.730011 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:18.729983    5510 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6mvrk"]
Oct 30 22:13:18.730185 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:18.730124    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-48cnj"
Oct 30 22:13:18.734505 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:18.734485    5510 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-k5dpm"]
Oct 30 22:13:18.734620 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:18.734594    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6mvrk"
Oct 30 22:13:18.738639 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:18.738622    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-k5dpm"
Oct 30 22:13:18.782981 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:18.782949    5510 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-l5ndh"]
Oct 30 22:13:18.792683 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:18.790224    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-q6kh7"]
Oct 30 22:13:18.792683 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:18.790372    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-l5ndh"
Oct 30 22:13:18.795350 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:18.795326    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-48cnj"]
Oct 30 22:13:18.798395 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:18.798366    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6mvrk"]
Oct 30 22:13:18.802067 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:18.802044    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-k5dpm"]
Oct 30 22:13:18.804685 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:18.804637    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-l5ndh"]
Oct 30 22:13:18.838082 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:18.838048    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3c82a381-05ad-436f-89dd-0985c3dbad33-istio-token\") pod \"e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-q6kh7\" (UID: \"3c82a381-05ad-436f-89dd-0985c3dbad33\") " pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-q6kh7"
Oct 30 22:13:18.838430 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:18.838167    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zl2qk\" (UniqueName: \"kubernetes.io/projected/3c82a381-05ad-436f-89dd-0985c3dbad33-kube-api-access-zl2qk\") pod \"e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-q6kh7\" (UID: \"3c82a381-05ad-436f-89dd-0985c3dbad33\") " pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-q6kh7"
Oct 30 22:13:18.847247 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:18.847217    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6aa08ff6-6f78-4a6e-83bd-07cf5d43b7b0-istio-token\") pod \"e2e-worker-w4-65e59f-openshift-default-596d74bb4f-48cnj\" (UID: \"6aa08ff6-6f78-4a6e-83bd-07cf5d43b7b0\") " pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-48cnj"
Oct 30 22:13:18.847380 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:18.847252    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8724086f-a5f7-4c63-87da-cb9228b03a0c-workload-certs\") pod \"e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-k5dpm\" (UID: \"8724086f-a5f7-4c63-87da-cb9228b03a0c\") " pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-k5dpm"
Oct 30 22:13:18.847380 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:18.847271    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8724086f-a5f7-4c63-87da-cb9228b03a0c-istio-token\") pod \"e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-k5dpm\" (UID: \"8724086f-a5f7-4c63-87da-cb9228b03a0c\") " pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-k5dpm"
Oct 30 22:13:18.847465 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:18.847410    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6cbfd1c6-ca09-4bee-a6d3-6461eac2c2d5-istio-podinfo\") pod \"e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6mvrk\" (UID: \"6cbfd1c6-ca09-4bee-a6d3-6461eac2c2d5\") " pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6mvrk"
Oct 30 22:13:18.847499 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:18.847467    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8724086f-a5f7-4c63-87da-cb9228b03a0c-workload-socket\") pod \"e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-k5dpm\" (UID: \"8724086f-a5f7-4c63-87da-cb9228b03a0c\") " pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-k5dpm"
Oct 30 22:13:18.847533 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:18.847515    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8724086f-a5f7-4c63-87da-cb9228b03a0c-credential-socket\") pod \"e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-k5dpm\" (UID: \"8724086f-a5f7-4c63-87da-cb9228b03a0c\") " pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-k5dpm"
Oct 30 22:13:18.847566 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:18.847546    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6cbfd1c6-ca09-4bee-a6d3-6461eac2c2d5-workload-certs\") pod \"e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6mvrk\" (UID: \"6cbfd1c6-ca09-4bee-a6d3-6461eac2c2d5\") " pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6mvrk"
Oct 30 22:13:18.847627 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:18.847613    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jz85b\" (UniqueName: \"kubernetes.io/projected/8724086f-a5f7-4c63-87da-cb9228b03a0c-kube-api-access-jz85b\") pod \"e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-k5dpm\" (UID: \"8724086f-a5f7-4c63-87da-cb9228b03a0c\") " pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-k5dpm"
Oct 30 22:13:18.847686 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:18.847670    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6cbfd1c6-ca09-4bee-a6d3-6461eac2c2d5-istio-token\") pod \"e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6mvrk\" (UID: \"6cbfd1c6-ca09-4bee-a6d3-6461eac2c2d5\") " pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6mvrk"
Oct 30 22:13:18.847764 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:18.847744    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5jwbx\" (UniqueName: \"kubernetes.io/projected/6aa08ff6-6f78-4a6e-83bd-07cf5d43b7b0-kube-api-access-5jwbx\") pod \"e2e-worker-w4-65e59f-openshift-default-596d74bb4f-48cnj\" (UID: \"6aa08ff6-6f78-4a6e-83bd-07cf5d43b7b0\") " pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-48cnj"
Oct 30 22:13:18.847796 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:18.847781    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6aa08ff6-6f78-4a6e-83bd-07cf5d43b7b0-workload-certs\") pod \"e2e-worker-w4-65e59f-openshift-default-596d74bb4f-48cnj\" (UID: \"6aa08ff6-6f78-4a6e-83bd-07cf5d43b7b0\") " pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-48cnj"
Oct 30 22:13:18.847829 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:18.847811    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v6nv7\" (UniqueName: \"kubernetes.io/projected/6cbfd1c6-ca09-4bee-a6d3-6461eac2c2d5-kube-api-access-v6nv7\") pod \"e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6mvrk\" (UID: \"6cbfd1c6-ca09-4bee-a6d3-6461eac2c2d5\") " pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6mvrk"
Oct 30 22:13:18.847944 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:18.847930    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6cbfd1c6-ca09-4bee-a6d3-6461eac2c2d5-credential-socket\") pod \"e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6mvrk\" (UID: \"6cbfd1c6-ca09-4bee-a6d3-6461eac2c2d5\") " pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6mvrk"
Oct 30 22:13:18.847978 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:18.847955    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6cbfd1c6-ca09-4bee-a6d3-6461eac2c2d5-workload-socket\") pod \"e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6mvrk\" (UID: \"6cbfd1c6-ca09-4bee-a6d3-6461eac2c2d5\") " pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6mvrk"
Oct 30 22:13:18.848012 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:18.847978    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6aa08ff6-6f78-4a6e-83bd-07cf5d43b7b0-istio-data\") pod \"e2e-worker-w4-65e59f-openshift-default-596d74bb4f-48cnj\" (UID: \"6aa08ff6-6f78-4a6e-83bd-07cf5d43b7b0\") " pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-48cnj"
Oct 30 22:13:18.848049 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:18.848025    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8724086f-a5f7-4c63-87da-cb9228b03a0c-istiod-ca-cert\") pod \"e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-k5dpm\" (UID: \"8724086f-a5f7-4c63-87da-cb9228b03a0c\") " pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-k5dpm"
Oct 30 22:13:18.848140 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:18.848128    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8724086f-a5f7-4c63-87da-cb9228b03a0c-istio-data\") pod \"e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-k5dpm\" (UID: \"8724086f-a5f7-4c63-87da-cb9228b03a0c\") " pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-k5dpm"
Oct 30 22:13:18.848180 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:18.848154    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6aa08ff6-6f78-4a6e-83bd-07cf5d43b7b0-istio-envoy\") pod \"e2e-worker-w4-65e59f-openshift-default-596d74bb4f-48cnj\" (UID: \"6aa08ff6-6f78-4a6e-83bd-07cf5d43b7b0\") " pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-48cnj"
Oct 30 22:13:18.848180 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:18.848175    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6aa08ff6-6f78-4a6e-83bd-07cf5d43b7b0-credential-socket\") pod \"e2e-worker-w4-65e59f-openshift-default-596d74bb4f-48cnj\" (UID: \"6aa08ff6-6f78-4a6e-83bd-07cf5d43b7b0\") " pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-48cnj"
Oct 30 22:13:18.848243 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:18.848191    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6aa08ff6-6f78-4a6e-83bd-07cf5d43b7b0-istio-podinfo\") pod \"e2e-worker-w4-65e59f-openshift-default-596d74bb4f-48cnj\" (UID: \"6aa08ff6-6f78-4a6e-83bd-07cf5d43b7b0\") " pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-48cnj"
Oct 30 22:13:18.848299 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:18.848283    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8724086f-a5f7-4c63-87da-cb9228b03a0c-istio-podinfo\") pod \"e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-k5dpm\" (UID: \"8724086f-a5f7-4c63-87da-cb9228b03a0c\") " pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-k5dpm"
Oct 30 22:13:18.848358 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:18.848344    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6cbfd1c6-ca09-4bee-a6d3-6461eac2c2d5-istiod-ca-cert\") pod \"e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6mvrk\" (UID: \"6cbfd1c6-ca09-4bee-a6d3-6461eac2c2d5\") " pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6mvrk"
Oct 30 22:13:18.848393 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:18.848383    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8724086f-a5f7-4c63-87da-cb9228b03a0c-istio-envoy\") pod \"e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-k5dpm\" (UID: \"8724086f-a5f7-4c63-87da-cb9228b03a0c\") " pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-k5dpm"
Oct 30 22:13:18.848468 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:18.848456    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6aa08ff6-6f78-4a6e-83bd-07cf5d43b7b0-istiod-ca-cert\") pod \"e2e-worker-w4-65e59f-openshift-default-596d74bb4f-48cnj\" (UID: \"6aa08ff6-6f78-4a6e-83bd-07cf5d43b7b0\") " pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-48cnj"
Oct 30 22:13:18.848502 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:18.848480    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6cbfd1c6-ca09-4bee-a6d3-6461eac2c2d5-istio-envoy\") pod \"e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6mvrk\" (UID: \"6cbfd1c6-ca09-4bee-a6d3-6461eac2c2d5\") " pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6mvrk"
Oct 30 22:13:18.848537 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:18.848509    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6cbfd1c6-ca09-4bee-a6d3-6461eac2c2d5-istio-data\") pod \"e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6mvrk\" (UID: \"6cbfd1c6-ca09-4bee-a6d3-6461eac2c2d5\") " pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6mvrk"
Oct 30 22:13:18.848572 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:18.848539    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6aa08ff6-6f78-4a6e-83bd-07cf5d43b7b0-workload-socket\") pod \"e2e-worker-w4-65e59f-openshift-default-596d74bb4f-48cnj\" (UID: \"6aa08ff6-6f78-4a6e-83bd-07cf5d43b7b0\") " pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-48cnj"
Oct 30 22:13:18.950271 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:18.950231    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6aa08ff6-6f78-4a6e-83bd-07cf5d43b7b0-istio-envoy\") pod \"e2e-worker-w4-65e59f-openshift-default-596d74bb4f-48cnj\" (UID: \"6aa08ff6-6f78-4a6e-83bd-07cf5d43b7b0\") " pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-48cnj"
Oct 30 22:13:18.950476 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:18.950289    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2344709a-1a48-417c-ae98-2b462bf30516-istiod-ca-cert\") pod \"e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-l5ndh\" (UID: \"2344709a-1a48-417c-ae98-2b462bf30516\") " pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-l5ndh"
Oct 30 22:13:18.950476 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:18.950316    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6aa08ff6-6f78-4a6e-83bd-07cf5d43b7b0-credential-socket\") pod \"e2e-worker-w4-65e59f-openshift-default-596d74bb4f-48cnj\" (UID: \"6aa08ff6-6f78-4a6e-83bd-07cf5d43b7b0\") " pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-48cnj"
Oct 30 22:13:18.950476 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:18.950338    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6aa08ff6-6f78-4a6e-83bd-07cf5d43b7b0-istio-podinfo\") pod \"e2e-worker-w4-65e59f-openshift-default-596d74bb4f-48cnj\" (UID: \"6aa08ff6-6f78-4a6e-83bd-07cf5d43b7b0\") " pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-48cnj"
Oct 30 22:13:18.950476 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:18.950367    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8724086f-a5f7-4c63-87da-cb9228b03a0c-istio-podinfo\") pod \"e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-k5dpm\" (UID: \"8724086f-a5f7-4c63-87da-cb9228b03a0c\") " pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-k5dpm"
Oct 30 22:13:18.950731 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:18.950478    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4k7rz\" (UniqueName: \"kubernetes.io/projected/2344709a-1a48-417c-ae98-2b462bf30516-kube-api-access-4k7rz\") pod \"e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-l5ndh\" (UID: \"2344709a-1a48-417c-ae98-2b462bf30516\") " pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-l5ndh"
Oct 30 22:13:18.950731 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:18.950541    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6cbfd1c6-ca09-4bee-a6d3-6461eac2c2d5-istiod-ca-cert\") pod \"e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6mvrk\" (UID: \"6cbfd1c6-ca09-4bee-a6d3-6461eac2c2d5\") " pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6mvrk"
Oct 30 22:13:18.950731 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:18.950586    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8724086f-a5f7-4c63-87da-cb9228b03a0c-istio-envoy\") pod \"e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-k5dpm\" (UID: \"8724086f-a5f7-4c63-87da-cb9228b03a0c\") " pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-k5dpm"
Oct 30 22:13:18.950897 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:18.950779    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6aa08ff6-6f78-4a6e-83bd-07cf5d43b7b0-credential-socket\") pod \"e2e-worker-w4-65e59f-openshift-default-596d74bb4f-48cnj\" (UID: \"6aa08ff6-6f78-4a6e-83bd-07cf5d43b7b0\") " pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-48cnj"
Oct 30 22:13:18.951482 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:18.951380    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6cbfd1c6-ca09-4bee-a6d3-6461eac2c2d5-istiod-ca-cert\") pod \"e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6mvrk\" (UID: \"6cbfd1c6-ca09-4bee-a6d3-6461eac2c2d5\") " pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6mvrk"
Oct 30 22:13:18.951482 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:18.951448    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2344709a-1a48-417c-ae98-2b462bf30516-istio-podinfo\") pod \"e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-l5ndh\" (UID: \"2344709a-1a48-417c-ae98-2b462bf30516\") " pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-l5ndh"
Oct 30 22:13:18.951740 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:18.951532    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6aa08ff6-6f78-4a6e-83bd-07cf5d43b7b0-istiod-ca-cert\") pod \"e2e-worker-w4-65e59f-openshift-default-596d74bb4f-48cnj\" (UID: \"6aa08ff6-6f78-4a6e-83bd-07cf5d43b7b0\") " pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-48cnj"
Oct 30 22:13:18.951740 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:18.951580    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6cbfd1c6-ca09-4bee-a6d3-6461eac2c2d5-istio-envoy\") pod \"e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6mvrk\" (UID: \"6cbfd1c6-ca09-4bee-a6d3-6461eac2c2d5\") " pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6mvrk"
Oct 30 22:13:18.951740 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:18.951631    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6cbfd1c6-ca09-4bee-a6d3-6461eac2c2d5-istio-data\") pod \"e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6mvrk\" (UID: \"6cbfd1c6-ca09-4bee-a6d3-6461eac2c2d5\") " pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6mvrk"
Oct 30 22:13:18.951740 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:18.951684    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6aa08ff6-6f78-4a6e-83bd-07cf5d43b7b0-workload-socket\") pod \"e2e-worker-w4-65e59f-openshift-default-596d74bb4f-48cnj\" (UID: \"6aa08ff6-6f78-4a6e-83bd-07cf5d43b7b0\") " pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-48cnj"
Oct 30 22:13:18.951740 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:18.951717    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2344709a-1a48-417c-ae98-2b462bf30516-workload-certs\") pod \"e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-l5ndh\" (UID: \"2344709a-1a48-417c-ae98-2b462bf30516\") " pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-l5ndh"
Oct 30 22:13:18.952025 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:18.951746    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6aa08ff6-6f78-4a6e-83bd-07cf5d43b7b0-istio-token\") pod \"e2e-worker-w4-65e59f-openshift-default-596d74bb4f-48cnj\" (UID: \"6aa08ff6-6f78-4a6e-83bd-07cf5d43b7b0\") " pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-48cnj"
Oct 30 22:13:18.952025 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:18.951773    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8724086f-a5f7-4c63-87da-cb9228b03a0c-workload-certs\") pod \"e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-k5dpm\" (UID: \"8724086f-a5f7-4c63-87da-cb9228b03a0c\") " pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-k5dpm"
Oct 30 22:13:18.952025 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:18.951803    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8724086f-a5f7-4c63-87da-cb9228b03a0c-istio-token\") pod \"e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-k5dpm\" (UID: \"8724086f-a5f7-4c63-87da-cb9228b03a0c\") " pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-k5dpm"
Oct 30 22:13:18.952025 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:18.951845    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2344709a-1a48-417c-ae98-2b462bf30516-workload-socket\") pod \"e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-l5ndh\" (UID: \"2344709a-1a48-417c-ae98-2b462bf30516\") " pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-l5ndh"
Oct 30 22:13:18.952025 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:18.951883    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6cbfd1c6-ca09-4bee-a6d3-6461eac2c2d5-istio-podinfo\") pod \"e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6mvrk\" (UID: \"6cbfd1c6-ca09-4bee-a6d3-6461eac2c2d5\") " pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6mvrk"
Oct 30 22:13:18.952025 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:18.951933    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8724086f-a5f7-4c63-87da-cb9228b03a0c-workload-socket\") pod \"e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-k5dpm\" (UID: \"8724086f-a5f7-4c63-87da-cb9228b03a0c\") " pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-k5dpm"
Oct 30 22:13:18.952025 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:18.951959    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8724086f-a5f7-4c63-87da-cb9228b03a0c-credential-socket\") pod \"e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-k5dpm\" (UID: \"8724086f-a5f7-4c63-87da-cb9228b03a0c\") " pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-k5dpm"
Oct 30 22:13:18.952025 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:18.951986    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6cbfd1c6-ca09-4bee-a6d3-6461eac2c2d5-workload-certs\") pod \"e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6mvrk\" (UID: \"6cbfd1c6-ca09-4bee-a6d3-6461eac2c2d5\") " pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6mvrk"
Oct 30 22:13:18.952439 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:18.952071    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jz85b\" (UniqueName: \"kubernetes.io/projected/8724086f-a5f7-4c63-87da-cb9228b03a0c-kube-api-access-jz85b\") pod \"e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-k5dpm\" (UID: \"8724086f-a5f7-4c63-87da-cb9228b03a0c\") " pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-k5dpm"
Oct 30 22:13:18.952439 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:18.952112    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6cbfd1c6-ca09-4bee-a6d3-6461eac2c2d5-istio-token\") pod \"e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6mvrk\" (UID: \"6cbfd1c6-ca09-4bee-a6d3-6461eac2c2d5\") " pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6mvrk"
Oct 30 22:13:18.952439 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:18.952223    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2344709a-1a48-417c-ae98-2b462bf30516-istio-data\") pod \"e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-l5ndh\" (UID: \"2344709a-1a48-417c-ae98-2b462bf30516\") " pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-l5ndh"
Oct 30 22:13:18.952439 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:18.952262    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5jwbx\" (UniqueName: \"kubernetes.io/projected/6aa08ff6-6f78-4a6e-83bd-07cf5d43b7b0-kube-api-access-5jwbx\") pod \"e2e-worker-w4-65e59f-openshift-default-596d74bb4f-48cnj\" (UID: \"6aa08ff6-6f78-4a6e-83bd-07cf5d43b7b0\") " pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-48cnj"
Oct 30 22:13:18.952439 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:18.952304    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6aa08ff6-6f78-4a6e-83bd-07cf5d43b7b0-workload-certs\") pod \"e2e-worker-w4-65e59f-openshift-default-596d74bb4f-48cnj\" (UID: \"6aa08ff6-6f78-4a6e-83bd-07cf5d43b7b0\") " pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-48cnj"
Oct 30 22:13:18.952439 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:18.952345    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2344709a-1a48-417c-ae98-2b462bf30516-istio-token\") pod \"e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-l5ndh\" (UID: \"2344709a-1a48-417c-ae98-2b462bf30516\") " pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-l5ndh"
Oct 30 22:13:18.952439 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:18.952386    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v6nv7\" (UniqueName: \"kubernetes.io/projected/6cbfd1c6-ca09-4bee-a6d3-6461eac2c2d5-kube-api-access-v6nv7\") pod \"e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6mvrk\" (UID: \"6cbfd1c6-ca09-4bee-a6d3-6461eac2c2d5\") " pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6mvrk"
Oct 30 22:13:18.952439 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:18.952429    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2344709a-1a48-417c-ae98-2b462bf30516-credential-socket\") pod \"e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-l5ndh\" (UID: \"2344709a-1a48-417c-ae98-2b462bf30516\") " pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-l5ndh"
Oct 30 22:13:18.952987 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:18.952498    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6cbfd1c6-ca09-4bee-a6d3-6461eac2c2d5-credential-socket\") pod \"e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6mvrk\" (UID: \"6cbfd1c6-ca09-4bee-a6d3-6461eac2c2d5\") " pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6mvrk"
Oct 30 22:13:18.952987 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:18.952536    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6cbfd1c6-ca09-4bee-a6d3-6461eac2c2d5-workload-socket\") pod \"e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6mvrk\" (UID: \"6cbfd1c6-ca09-4bee-a6d3-6461eac2c2d5\") " pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6mvrk"
Oct 30 22:13:18.952987 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:18.952572    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6aa08ff6-6f78-4a6e-83bd-07cf5d43b7b0-istio-data\") pod \"e2e-worker-w4-65e59f-openshift-default-596d74bb4f-48cnj\" (UID: \"6aa08ff6-6f78-4a6e-83bd-07cf5d43b7b0\") " pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-48cnj"
Oct 30 22:13:18.952987 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:18.952602    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8724086f-a5f7-4c63-87da-cb9228b03a0c-istiod-ca-cert\") pod \"e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-k5dpm\" (UID: \"8724086f-a5f7-4c63-87da-cb9228b03a0c\") " pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-k5dpm"
Oct 30 22:13:18.952987 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:18.952636    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2344709a-1a48-417c-ae98-2b462bf30516-istio-envoy\") pod \"e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-l5ndh\" (UID: \"2344709a-1a48-417c-ae98-2b462bf30516\") " pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-l5ndh"
Oct 30 22:13:18.952987 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:18.952724    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8724086f-a5f7-4c63-87da-cb9228b03a0c-istio-data\") pod \"e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-k5dpm\" (UID: \"8724086f-a5f7-4c63-87da-cb9228b03a0c\") " pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-k5dpm"
Oct 30 22:13:18.953342 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:18.953171    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8724086f-a5f7-4c63-87da-cb9228b03a0c-istio-data\") pod \"e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-k5dpm\" (UID: \"8724086f-a5f7-4c63-87da-cb9228b03a0c\") " pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-k5dpm"
Oct 30 22:13:18.971835 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:18.953433    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6cbfd1c6-ca09-4bee-a6d3-6461eac2c2d5-istio-data\") pod \"e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6mvrk\" (UID: \"6cbfd1c6-ca09-4bee-a6d3-6461eac2c2d5\") " pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6mvrk"
Oct 30 22:13:18.971835 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:18.953707    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6aa08ff6-6f78-4a6e-83bd-07cf5d43b7b0-workload-socket\") pod \"e2e-worker-w4-65e59f-openshift-default-596d74bb4f-48cnj\" (UID: \"6aa08ff6-6f78-4a6e-83bd-07cf5d43b7b0\") " pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-48cnj"
Oct 30 22:13:18.971835 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:18.953950    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6aa08ff6-6f78-4a6e-83bd-07cf5d43b7b0-istio-envoy\") pod \"e2e-worker-w4-65e59f-openshift-default-596d74bb4f-48cnj\" (UID: \"6aa08ff6-6f78-4a6e-83bd-07cf5d43b7b0\") " pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-48cnj"
Oct 30 22:13:18.971835 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:18.953976    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6cbfd1c6-ca09-4bee-a6d3-6461eac2c2d5-credential-socket\") pod \"e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6mvrk\" (UID: \"6cbfd1c6-ca09-4bee-a6d3-6461eac2c2d5\") " pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6mvrk"
Oct 30 22:13:18.971835 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:18.954138    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8724086f-a5f7-4c63-87da-cb9228b03a0c-workload-certs\") pod \"e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-k5dpm\" (UID: \"8724086f-a5f7-4c63-87da-cb9228b03a0c\") " pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-k5dpm"
Oct 30 22:13:18.971835 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:18.954561    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6aa08ff6-6f78-4a6e-83bd-07cf5d43b7b0-istio-data\") pod \"e2e-worker-w4-65e59f-openshift-default-596d74bb4f-48cnj\" (UID: \"6aa08ff6-6f78-4a6e-83bd-07cf5d43b7b0\") " pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-48cnj"
Oct 30 22:13:18.971835 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:18.954632    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6aa08ff6-6f78-4a6e-83bd-07cf5d43b7b0-istiod-ca-cert\") pod \"e2e-worker-w4-65e59f-openshift-default-596d74bb4f-48cnj\" (UID: \"6aa08ff6-6f78-4a6e-83bd-07cf5d43b7b0\") " pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-48cnj"
Oct 30 22:13:18.971835 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:18.954747    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6aa08ff6-6f78-4a6e-83bd-07cf5d43b7b0-workload-certs\") pod \"e2e-worker-w4-65e59f-openshift-default-596d74bb4f-48cnj\" (UID: \"6aa08ff6-6f78-4a6e-83bd-07cf5d43b7b0\") " pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-48cnj"
Oct 30 22:13:18.971835 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:18.954841    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6cbfd1c6-ca09-4bee-a6d3-6461eac2c2d5-workload-socket\") pod \"e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6mvrk\" (UID: \"6cbfd1c6-ca09-4bee-a6d3-6461eac2c2d5\") " pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6mvrk"
Oct 30 22:13:18.971835 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:18.955083    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8724086f-a5f7-4c63-87da-cb9228b03a0c-credential-socket\") pod \"e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-k5dpm\" (UID: \"8724086f-a5f7-4c63-87da-cb9228b03a0c\") " pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-k5dpm"
Oct 30 22:13:18.971835 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:18.955176    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8724086f-a5f7-4c63-87da-cb9228b03a0c-workload-socket\") pod \"e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-k5dpm\" (UID: \"8724086f-a5f7-4c63-87da-cb9228b03a0c\") " pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-k5dpm"
Oct 30 22:13:18.971835 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:18.955352    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6cbfd1c6-ca09-4bee-a6d3-6461eac2c2d5-workload-certs\") pod \"e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6mvrk\" (UID: \"6cbfd1c6-ca09-4bee-a6d3-6461eac2c2d5\") " pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6mvrk"
Oct 30 22:13:18.971835 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:18.955385    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8724086f-a5f7-4c63-87da-cb9228b03a0c-istiod-ca-cert\") pod \"e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-k5dpm\" (UID: \"8724086f-a5f7-4c63-87da-cb9228b03a0c\") " pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-k5dpm"
Oct 30 22:13:18.971835 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:18.955825    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6cbfd1c6-ca09-4bee-a6d3-6461eac2c2d5-istio-envoy\") pod \"e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6mvrk\" (UID: \"6cbfd1c6-ca09-4bee-a6d3-6461eac2c2d5\") " pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6mvrk"
Oct 30 22:13:18.971835 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:18.958916    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8724086f-a5f7-4c63-87da-cb9228b03a0c-istio-podinfo\") pod \"e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-k5dpm\" (UID: \"8724086f-a5f7-4c63-87da-cb9228b03a0c\") " pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-k5dpm"
Oct 30 22:13:18.971835 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:18.959111    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8724086f-a5f7-4c63-87da-cb9228b03a0c-istio-envoy\") pod \"e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-k5dpm\" (UID: \"8724086f-a5f7-4c63-87da-cb9228b03a0c\") " pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-k5dpm"
Oct 30 22:13:18.971835 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:18.959151    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6cbfd1c6-ca09-4bee-a6d3-6461eac2c2d5-istio-podinfo\") pod \"e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6mvrk\" (UID: \"6cbfd1c6-ca09-4bee-a6d3-6461eac2c2d5\") " pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6mvrk"
Oct 30 22:13:18.971835 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:18.959445    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6aa08ff6-6f78-4a6e-83bd-07cf5d43b7b0-istio-podinfo\") pod \"e2e-worker-w4-65e59f-openshift-default-596d74bb4f-48cnj\" (UID: \"6aa08ff6-6f78-4a6e-83bd-07cf5d43b7b0\") " pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-48cnj"
Oct 30 22:13:19.022422 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:19.022384    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jz85b\" (UniqueName: \"kubernetes.io/projected/8724086f-a5f7-4c63-87da-cb9228b03a0c-kube-api-access-jz85b\") pod \"e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-k5dpm\" (UID: \"8724086f-a5f7-4c63-87da-cb9228b03a0c\") " pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-k5dpm"
Oct 30 22:13:19.041793 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:19.041764    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-q6kh7"
Oct 30 22:13:19.047117 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:19.047094    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8724086f-a5f7-4c63-87da-cb9228b03a0c-istio-token\") pod \"e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-k5dpm\" (UID: \"8724086f-a5f7-4c63-87da-cb9228b03a0c\") " pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-k5dpm"
Oct 30 22:13:19.047284 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:19.047256    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v6nv7\" (UniqueName: \"kubernetes.io/projected/6cbfd1c6-ca09-4bee-a6d3-6461eac2c2d5-kube-api-access-v6nv7\") pod \"e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6mvrk\" (UID: \"6cbfd1c6-ca09-4bee-a6d3-6461eac2c2d5\") " pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6mvrk"
Oct 30 22:13:19.047386 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:19.047313    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6aa08ff6-6f78-4a6e-83bd-07cf5d43b7b0-istio-token\") pod \"e2e-worker-w4-65e59f-openshift-default-596d74bb4f-48cnj\" (UID: \"6aa08ff6-6f78-4a6e-83bd-07cf5d43b7b0\") " pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-48cnj"
Oct 30 22:13:19.047572 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:19.047556    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6cbfd1c6-ca09-4bee-a6d3-6461eac2c2d5-istio-token\") pod \"e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6mvrk\" (UID: \"6cbfd1c6-ca09-4bee-a6d3-6461eac2c2d5\") " pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6mvrk"
Oct 30 22:13:19.047728 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:19.047710    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5jwbx\" (UniqueName: \"kubernetes.io/projected/6aa08ff6-6f78-4a6e-83bd-07cf5d43b7b0-kube-api-access-5jwbx\") pod \"e2e-worker-w4-65e59f-openshift-default-596d74bb4f-48cnj\" (UID: \"6aa08ff6-6f78-4a6e-83bd-07cf5d43b7b0\") " pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-48cnj"
Oct 30 22:13:19.054012 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:19.053990    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2344709a-1a48-417c-ae98-2b462bf30516-istiod-ca-cert\") pod \"e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-l5ndh\" (UID: \"2344709a-1a48-417c-ae98-2b462bf30516\") " pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-l5ndh"
Oct 30 22:13:19.054110 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:19.054039    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4k7rz\" (UniqueName: \"kubernetes.io/projected/2344709a-1a48-417c-ae98-2b462bf30516-kube-api-access-4k7rz\") pod \"e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-l5ndh\" (UID: \"2344709a-1a48-417c-ae98-2b462bf30516\") " pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-l5ndh"
Oct 30 22:13:19.054110 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:19.054081    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2344709a-1a48-417c-ae98-2b462bf30516-istio-podinfo\") pod \"e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-l5ndh\" (UID: \"2344709a-1a48-417c-ae98-2b462bf30516\") " pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-l5ndh"
Oct 30 22:13:19.054216 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:19.054133    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2344709a-1a48-417c-ae98-2b462bf30516-workload-certs\") pod \"e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-l5ndh\" (UID: \"2344709a-1a48-417c-ae98-2b462bf30516\") " pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-l5ndh"
Oct 30 22:13:19.054216 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:19.054176    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2344709a-1a48-417c-ae98-2b462bf30516-workload-socket\") pod \"e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-l5ndh\" (UID: \"2344709a-1a48-417c-ae98-2b462bf30516\") " pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-l5ndh"
Oct 30 22:13:19.054311 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:19.054253    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6mvrk"
Oct 30 22:13:19.054311 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:19.054300    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2344709a-1a48-417c-ae98-2b462bf30516-istio-data\") pod \"e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-l5ndh\" (UID: \"2344709a-1a48-417c-ae98-2b462bf30516\") " pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-l5ndh"
Oct 30 22:13:19.054413 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:19.054351    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2344709a-1a48-417c-ae98-2b462bf30516-istio-token\") pod \"e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-l5ndh\" (UID: \"2344709a-1a48-417c-ae98-2b462bf30516\") " pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-l5ndh"
Oct 30 22:13:19.054413 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:19.054397    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2344709a-1a48-417c-ae98-2b462bf30516-credential-socket\") pod \"e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-l5ndh\" (UID: \"2344709a-1a48-417c-ae98-2b462bf30516\") " pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-l5ndh"
Oct 30 22:13:19.054513 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:19.054462    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2344709a-1a48-417c-ae98-2b462bf30516-istio-envoy\") pod \"e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-l5ndh\" (UID: \"2344709a-1a48-417c-ae98-2b462bf30516\") " pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-l5ndh"
Oct 30 22:13:19.054744 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:19.054725    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2344709a-1a48-417c-ae98-2b462bf30516-workload-socket\") pod \"e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-l5ndh\" (UID: \"2344709a-1a48-417c-ae98-2b462bf30516\") " pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-l5ndh"
Oct 30 22:13:19.054869 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:19.054806    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2344709a-1a48-417c-ae98-2b462bf30516-istiod-ca-cert\") pod \"e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-l5ndh\" (UID: \"2344709a-1a48-417c-ae98-2b462bf30516\") " pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-l5ndh"
Oct 30 22:13:19.054869 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:19.054829    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2344709a-1a48-417c-ae98-2b462bf30516-credential-socket\") pod \"e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-l5ndh\" (UID: \"2344709a-1a48-417c-ae98-2b462bf30516\") " pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-l5ndh"
Oct 30 22:13:19.054999 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:19.054956    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2344709a-1a48-417c-ae98-2b462bf30516-istio-data\") pod \"e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-l5ndh\" (UID: \"2344709a-1a48-417c-ae98-2b462bf30516\") " pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-l5ndh"
Oct 30 22:13:19.055054 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:19.055014    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2344709a-1a48-417c-ae98-2b462bf30516-workload-certs\") pod \"e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-l5ndh\" (UID: \"2344709a-1a48-417c-ae98-2b462bf30516\") " pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-l5ndh"
Oct 30 22:13:19.056774 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:19.056750    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2344709a-1a48-417c-ae98-2b462bf30516-istio-envoy\") pod \"e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-l5ndh\" (UID: \"2344709a-1a48-417c-ae98-2b462bf30516\") " pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-l5ndh"
Oct 30 22:13:19.056858 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:19.056793    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2344709a-1a48-417c-ae98-2b462bf30516-istio-podinfo\") pod \"e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-l5ndh\" (UID: \"2344709a-1a48-417c-ae98-2b462bf30516\") " pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-l5ndh"
Oct 30 22:13:19.060423 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:19.060400    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-k5dpm"
Oct 30 22:13:19.125964 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:19.125898    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2344709a-1a48-417c-ae98-2b462bf30516-istio-token\") pod \"e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-l5ndh\" (UID: \"2344709a-1a48-417c-ae98-2b462bf30516\") " pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-l5ndh"
Oct 30 22:13:19.131700 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:19.128859    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4k7rz\" (UniqueName: \"kubernetes.io/projected/2344709a-1a48-417c-ae98-2b462bf30516-kube-api-access-4k7rz\") pod \"e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-l5ndh\" (UID: \"2344709a-1a48-417c-ae98-2b462bf30516\") " pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-l5ndh"
Oct 30 22:13:19.346259 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:19.346171    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-48cnj"
Oct 30 22:13:19.414806 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:19.412997    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-l5ndh"
Oct 30 22:13:19.519092 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:19.516859    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:13:19.579008 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:19.573588    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-q6kh7"]
Oct 30 22:13:19.589587 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:19.589544    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6mvrk"]
Oct 30 22:13:19.595800 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:19.595765    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:13:19.595911 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:19.595844    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:13:19.595911 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:19.595889    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:13:19.596984 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:19.596922    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-k5dpm"]
Oct 30 22:13:19.597296 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:19.597271    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:13:19.597371 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:19.597347    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:13:19.597430 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:19.597392    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:13:19.608215 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:19.608189    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:13:19.608352 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:19.608338    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:13:19.608441 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:19.608429    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:13:19.845839 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:19.845528    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-48cnj"]
Oct 30 22:13:19.848857 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:19.848774    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:13:19.849072 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:19.849051    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:13:19.849194 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:19.849179    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:13:20.012967 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:20.012929    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-l5ndh"]
Oct 30 22:13:20.016076 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:20.015547    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:13:20.016076 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:20.015635    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:13:20.016076 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:20.015693    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:13:20.287391 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:20.287350    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6mvrk" event={"ID":"6cbfd1c6-ca09-4bee-a6d3-6461eac2c2d5","Type":"ContainerStarted","Data":"d55780c704afcc7e3d56c850cc0477a6aba36d95498a131e84b5479e7674d50d"}
Oct 30 22:13:20.287695 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:20.287632    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6mvrk" event={"ID":"6cbfd1c6-ca09-4bee-a6d3-6461eac2c2d5","Type":"ContainerStarted","Data":"2ab0440c18ffc1afbbe1fc60dd29a65ab4a1b5c41bb2d12bc1949287a581f8b6"}
Oct 30 22:13:20.289224 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:20.289191    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-k5dpm" event={"ID":"8724086f-a5f7-4c63-87da-cb9228b03a0c","Type":"ContainerStarted","Data":"c8299b28bfa098ae0dbbb915292d8ab452f1ad709248e8fc8ea6090ab39f5a99"}
Oct 30 22:13:20.289351 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:20.289240    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-k5dpm" event={"ID":"8724086f-a5f7-4c63-87da-cb9228b03a0c","Type":"ContainerStarted","Data":"2052ea746aa6d4c54ce3e90f3f2b33773210d397996be5f9410ef944da3de8b1"}
Oct 30 22:13:20.291114 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:20.291088    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-48cnj" event={"ID":"6aa08ff6-6f78-4a6e-83bd-07cf5d43b7b0","Type":"ContainerStarted","Data":"457dc830f7bf2bcddeeb03600608586d37a9d994491941e3ba957f653556657b"}
Oct 30 22:13:20.291229 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:20.291135    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-48cnj" event={"ID":"6aa08ff6-6f78-4a6e-83bd-07cf5d43b7b0","Type":"ContainerStarted","Data":"eb723c1060732baa900e4166970417222a52e782db181f6bbe678c2401900a57"}
Oct 30 22:13:20.292862 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:20.292842    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-l5ndh" event={"ID":"2344709a-1a48-417c-ae98-2b462bf30516","Type":"ContainerStarted","Data":"7c524cc3c61f0a271fdc614f3ce555c8653983b2f00bc3c5273fbc29d0e93e2c"}
Oct 30 22:13:20.292974 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:20.292881    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-l5ndh" event={"ID":"2344709a-1a48-417c-ae98-2b462bf30516","Type":"ContainerStarted","Data":"73b33eaac8def512e1877ae3ae20913170663d58911665bc47ebef15b60528bf"}
Oct 30 22:13:20.294739 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:20.294716    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-q6kh7" event={"ID":"3c82a381-05ad-436f-89dd-0985c3dbad33","Type":"ContainerStarted","Data":"958d8521ac74ff55fa78bffb7e2167e3fe59f664dd1035b5ed908a7a39c28c13"}
Oct 30 22:13:20.294827 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:20.294772    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-q6kh7" event={"ID":"3c82a381-05ad-436f-89dd-0985c3dbad33","Type":"ContainerStarted","Data":"8d87a7d2790df605e880d86f67b79d041e72c178bc2fbf2ce2eb7ba50b57e0aa"}
Oct 30 22:13:20.347411 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:20.347372    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-48cnj"
Oct 30 22:13:20.348992 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:20.348963    5510 patch_prober.go:29] interesting pod/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-48cnj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused" start-of-body=
Oct 30 22:13:20.349125 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:20.349022    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-48cnj" podUID="6aa08ff6-6f78-4a6e-83bd-07cf5d43b7b0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused"
Oct 30 22:13:20.414349 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:20.414261    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-l5ndh"
Oct 30 22:13:20.428754 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:20.428705    5510 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6mvrk" podStartSLOduration=3.428688229 podStartE2EDuration="3.428688229s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:13:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:13:20.426191794 +0000 UTC m=+1380.987001843" watchObservedRunningTime="2026-10-30 22:13:20.428688229 +0000 UTC m=+1380.989498238"
Oct 30 22:13:20.567356 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:20.567216    5510 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-l5ndh" podStartSLOduration=3.56719445 podStartE2EDuration="3.56719445s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:13:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:13:20.565315522 +0000 UTC m=+1381.126125527" watchObservedRunningTime="2026-10-30 22:13:20.56719445 +0000 UTC m=+1381.128004456"
Oct 30 22:13:20.873680 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:20.871136    5510 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-k5dpm" podStartSLOduration=3.871112601 podStartE2EDuration="3.871112601s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:13:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:13:20.720324167 +0000 UTC m=+1381.281134187" watchObservedRunningTime="2026-10-30 22:13:20.871112601 +0000 UTC m=+1381.431922607"
Oct 30 22:13:21.048097 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:21.045990    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-q6kh7"
Oct 30 22:13:21.051776 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:21.048822    5510 patch_prober.go:29] interesting pod/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-q6kh7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body=
Oct 30 22:13:21.051776 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:21.048883    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-q6kh7" podUID="3c82a381-05ad-436f-89dd-0985c3dbad33" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused"
Oct 30 22:13:21.064108 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:21.057013    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6mvrk"
Oct 30 22:13:21.064108 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:21.062016    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-k5dpm"
Oct 30 22:13:21.071681 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:21.064771    5510 patch_prober.go:29] interesting pod/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-k5dpm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused" start-of-body=
Oct 30 22:13:21.071681 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:21.064828    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-k5dpm" podUID="8724086f-a5f7-4c63-87da-cb9228b03a0c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused"
Oct 30 22:13:21.071681 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:21.064912    5510 patch_prober.go:29] interesting pod/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6mvrk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" start-of-body=
Oct 30 22:13:21.071681 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:21.064935    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6mvrk" podUID="6cbfd1c6-ca09-4bee-a6d3-6461eac2c2d5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused"
Oct 30 22:13:21.335082 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:21.335018    5510 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-48cnj" podStartSLOduration=4.334995206 podStartE2EDuration="4.334995206s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:13:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:13:20.87744006 +0000 UTC m=+1381.438250066" watchObservedRunningTime="2026-10-30 22:13:21.334995206 +0000 UTC m=+1381.895805211"
Oct 30 22:13:21.350542 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:21.350407    5510 patch_prober.go:29] interesting pod/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-48cnj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused" start-of-body=
Oct 30 22:13:21.350542 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:21.350469    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-48cnj" podUID="6aa08ff6-6f78-4a6e-83bd-07cf5d43b7b0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused"
Oct 30 22:13:21.365802 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:21.355703    5510 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-q6kh7" podStartSLOduration=4.355683492 podStartE2EDuration="4.355683492s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:13:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:13:21.350049959 +0000 UTC m=+1381.910860012" watchObservedRunningTime="2026-10-30 22:13:21.355683492 +0000 UTC m=+1381.916493500"
Oct 30 22:13:21.468185 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:21.468146    5510 patch_prober.go:29] interesting pod/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-l5ndh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused" start-of-body=
Oct 30 22:13:21.468360 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:21.468229    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-l5ndh" podUID="2344709a-1a48-417c-ae98-2b462bf30516" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused"
Oct 30 22:13:21.912038 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:21.911986    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-779f32/distinct-w0-779f32-kserve-6dcbc948c7-2vzdf" podUID="e17cedd4-7d48-42b3-b127-b6192c1b3795" containerName="main" probeResult="failure" output="Get \"https://10.133.0.65:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:13:21.914037 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:21.913722    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-fb0b93/premium-w6-fb0b93-kserve-5cb9948c54-w7pnk" podUID="290ae7a2-de37-4af3-82ae-a8e855b83bb9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:13:22.058391 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:22.058346    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-bc714d/distinct-w2-bc714d-kserve-66cfb968dd-jvgqj" podUID="9e9b11b8-1cc9-4fc8-8097-1afcffa8c422" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:13:22.060415 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:22.060386    5510 patch_prober.go:29] interesting pod/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6mvrk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" start-of-body=
Oct 30 22:13:22.060611 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:22.060590    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6mvrk" podUID="6cbfd1c6-ca09-4bee-a6d3-6461eac2c2d5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused"
Oct 30 22:13:22.061022 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:22.061002    5510 patch_prober.go:29] interesting pod/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-k5dpm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused" start-of-body=
Oct 30 22:13:22.061150 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:22.061130    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-k5dpm" podUID="8724086f-a5f7-4c63-87da-cb9228b03a0c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused"
Oct 30 22:13:22.061915 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:22.061895    5510 patch_prober.go:29] interesting pod/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-q6kh7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body=
Oct 30 22:13:22.062051 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:22.062033    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-q6kh7" podUID="3c82a381-05ad-436f-89dd-0985c3dbad33" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused"
Oct 30 22:13:22.198713 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:22.197766    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-779f32/distinct-2-w0-779f32-kserve-857ffdf778-gk2h9" podUID="ef500cdd-cc52-45b0-8dd7-1896897acc46" containerName="main" probeResult="failure" output="Get \"https://10.133.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:13:22.371728 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:22.371689    5510 patch_prober.go:29] interesting pod/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-48cnj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused" start-of-body=
Oct 30 22:13:22.371987 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:22.371959    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-48cnj" podUID="6aa08ff6-6f78-4a6e-83bd-07cf5d43b7b0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused"
Oct 30 22:13:22.418090 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:22.418051    5510 patch_prober.go:29] interesting pod/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-l5ndh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused" start-of-body=
Oct 30 22:13:22.418393 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:22.418362    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-l5ndh" podUID="2344709a-1a48-417c-ae98-2b462bf30516" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused"
Oct 30 22:13:22.482787 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:22.475743    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-7bb1c2/distinct-2-w3-7bb1c2-kserve-c549df844-zt595" podUID="1916e87c-a1a4-4967-8640-8bbcd9d12903" containerName="main" probeResult="failure" output="Get \"https://10.133.0.73:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:13:22.625806 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:22.625757    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-c9dbe6/distinct-w5-c9dbe6-kserve-7599fd86d9-bdhxd" podUID="e903db90-f2cd-4244-b00b-f99b21873f6e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:13:22.649803 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:22.649750    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-fb0b93/distinct-w6-fb0b93-kserve-58db6b6757-6mww8" podUID="8ee6579b-acea-4492-862b-9883f70d00c9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:13:23.043683 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:22.994487    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-779f32/distinct-w0-779f32-kserve-6dcbc948c7-2vzdf" podUID="e17cedd4-7d48-42b3-b127-b6192c1b3795" containerName="main" probeResult="failure" output="Get \"https://10.133.0.65:8000/ready\": context deadline exceeded"
Oct 30 22:13:23.043683 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:22.995038    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-779f32/distinct-w0-779f32-kserve-6dcbc948c7-2vzdf"
Oct 30 22:13:23.043683 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:23.017679    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-fb0b93/premium-w6-fb0b93-kserve-5cb9948c54-w7pnk" podUID="290ae7a2-de37-4af3-82ae-a8e855b83bb9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:13:23.043683 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:23.018109    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-fb0b93/premium-w6-fb0b93-kserve-5cb9948c54-w7pnk"
Oct 30 22:13:23.061680 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:23.056071    5510 patch_prober.go:29] interesting pod/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6mvrk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" start-of-body=
Oct 30 22:13:23.061680 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:23.056129    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6mvrk" podUID="6cbfd1c6-ca09-4bee-a6d3-6461eac2c2d5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused"
Oct 30 22:13:23.061680 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:23.057077    5510 patch_prober.go:29] interesting pod/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-q6kh7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body=
Oct 30 22:13:23.061680 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:23.057113    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-q6kh7" podUID="3c82a381-05ad-436f-89dd-0985c3dbad33" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused"
Oct 30 22:13:23.061680 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:23.059767    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-bc714d/distinct-w2-bc714d-kserve-66cfb968dd-jvgqj" podUID="9e9b11b8-1cc9-4fc8-8097-1afcffa8c422" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:13:23.061680 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:23.060754    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-bc714d/distinct-w2-bc714d-kserve-66cfb968dd-jvgqj"
Oct 30 22:13:23.169692 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:23.149947    5510 patch_prober.go:29] interesting pod/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-k5dpm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused" start-of-body=
Oct 30 22:13:23.169692 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:23.150003    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-k5dpm" podUID="8724086f-a5f7-4c63-87da-cb9228b03a0c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused"
Oct 30 22:13:23.398961 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:23.391606    5510 patch_prober.go:29] interesting pod/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-48cnj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused" start-of-body=
Oct 30 22:13:23.398961 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:23.391693    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-48cnj" podUID="6aa08ff6-6f78-4a6e-83bd-07cf5d43b7b0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused"
Oct 30 22:13:23.490688 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:23.487835    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-7bb1c2/distinct-2-w3-7bb1c2-kserve-c549df844-zt595" podUID="1916e87c-a1a4-4967-8640-8bbcd9d12903" containerName="main" probeResult="failure" output="Get \"https://10.133.0.73:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:13:23.490688 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:23.488438    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-7bb1c2/distinct-2-w3-7bb1c2-kserve-c549df844-zt595"
Oct 30 22:13:23.501688 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:23.496143    5510 patch_prober.go:29] interesting pod/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-l5ndh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused" start-of-body=
Oct 30 22:13:23.501688 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:23.496193    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-l5ndh" podUID="2344709a-1a48-417c-ae98-2b462bf30516" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused"
Oct 30 22:13:23.635683 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:23.631273    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-c9dbe6/distinct-w5-c9dbe6-kserve-7599fd86d9-bdhxd" podUID="e903db90-f2cd-4244-b00b-f99b21873f6e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:13:23.730526 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:23.712805    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-fb0b93/distinct-w6-fb0b93-kserve-58db6b6757-6mww8" podUID="8ee6579b-acea-4492-862b-9883f70d00c9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:13:23.764520 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:23.745250    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-c9dbe6/distinct-w5-c9dbe6-kserve-7599fd86d9-bdhxd"
Oct 30 22:13:24.125702 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:24.123477    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-fb0b93/premium-w6-fb0b93-kserve-5cb9948c54-w7pnk" podUID="290ae7a2-de37-4af3-82ae-a8e855b83bb9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:13:24.160995 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:24.160947    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-bc714d/distinct-w2-bc714d-kserve-66cfb968dd-jvgqj" podUID="9e9b11b8-1cc9-4fc8-8097-1afcffa8c422" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": context deadline exceeded"
Oct 30 22:13:24.193589 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:24.193546    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-fb0b93/distinct-w6-fb0b93-kserve-58db6b6757-6mww8"
Oct 30 22:13:24.236848 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:24.236792    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-779f32/distinct-2-w0-779f32-kserve-857ffdf778-gk2h9" podUID="ef500cdd-cc52-45b0-8dd7-1896897acc46" containerName="main" probeResult="failure" output="Get \"https://10.133.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:13:24.240965 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:24.209003    5510 patch_prober.go:29] interesting pod/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-q6kh7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body=
Oct 30 22:13:24.241230 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:24.241200    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-q6kh7" podUID="3c82a381-05ad-436f-89dd-0985c3dbad33" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused"
Oct 30 22:13:24.246205 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:24.246173    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-779f32/distinct-w0-779f32-kserve-6dcbc948c7-2vzdf" podUID="e17cedd4-7d48-42b3-b127-b6192c1b3795" containerName="main" probeResult="failure" output="Get \"https://10.133.0.65:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:13:24.296870 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:24.296831    5510 patch_prober.go:29] interesting pod/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-k5dpm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused" start-of-body=
Oct 30 22:13:24.379014 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:24.378908    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-k5dpm" podUID="8724086f-a5f7-4c63-87da-cb9228b03a0c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused"
Oct 30 22:13:24.379685 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:24.363336    5510 patch_prober.go:29] interesting pod/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6mvrk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" start-of-body=
Oct 30 22:13:24.379685 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:24.379303    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6mvrk" podUID="6cbfd1c6-ca09-4bee-a6d3-6461eac2c2d5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused"
Oct 30 22:13:24.387924 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:24.387888    5510 patch_prober.go:29] interesting pod/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-48cnj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused" start-of-body=
Oct 30 22:13:24.388145 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:24.388117    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-48cnj" podUID="6aa08ff6-6f78-4a6e-83bd-07cf5d43b7b0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused"
Oct 30 22:13:24.526846 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:24.526806    5510 patch_prober.go:29] interesting pod/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-l5ndh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused" start-of-body=
Oct 30 22:13:24.527125 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:24.527088    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-l5ndh" podUID="2344709a-1a48-417c-ae98-2b462bf30516" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused"
Oct 30 22:13:24.527905 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:24.488264    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-7bb1c2/distinct-2-w3-7bb1c2-kserve-c549df844-zt595" podUID="1916e87c-a1a4-4967-8640-8bbcd9d12903" containerName="main" probeResult="failure" output="Get \"https://10.133.0.73:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:13:24.643025 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:24.642901    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-c9dbe6/distinct-w5-c9dbe6-kserve-7599fd86d9-bdhxd" podUID="e903db90-f2cd-4244-b00b-f99b21873f6e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:13:24.719165 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:24.719116    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-fb0b93/distinct-w6-fb0b93-kserve-58db6b6757-6mww8" podUID="8ee6579b-acea-4492-862b-9883f70d00c9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:13:24.826109 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:24.826071    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:13:25.087358 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:25.087309    5510 patch_prober.go:29] interesting pod/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-k5dpm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused" start-of-body=
Oct 30 22:13:25.087547 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:25.087371    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-k5dpm" podUID="8724086f-a5f7-4c63-87da-cb9228b03a0c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused"
Oct 30 22:13:25.088777 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:25.088744    5510 patch_prober.go:29] interesting pod/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6mvrk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" start-of-body=
Oct 30 22:13:25.088990 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:25.088964    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6mvrk" podUID="6cbfd1c6-ca09-4bee-a6d3-6461eac2c2d5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused"
Oct 30 22:13:25.089315 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:25.089297    5510 patch_prober.go:29] interesting pod/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-q6kh7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body=
Oct 30 22:13:25.089441 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:25.089421    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-q6kh7" podUID="3c82a381-05ad-436f-89dd-0985c3dbad33" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused"
Oct 30 22:13:25.165681 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:25.159539    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-fb0b93/premium-w6-fb0b93-kserve-5cb9948c54-w7pnk" podUID="290ae7a2-de37-4af3-82ae-a8e855b83bb9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": context deadline exceeded"
Oct 30 22:13:25.187693 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:25.186691    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-bc714d/distinct-w2-bc714d-kserve-66cfb968dd-jvgqj" podUID="9e9b11b8-1cc9-4fc8-8097-1afcffa8c422" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:13:25.267035 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:25.247723    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-779f32/distinct-w0-779f32-kserve-6dcbc948c7-2vzdf" podUID="e17cedd4-7d48-42b3-b127-b6192c1b3795" containerName="main" probeResult="failure" output="Get \"https://10.133.0.65:8000/ready\": context deadline exceeded"
Oct 30 22:13:25.291827 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:25.291773    5510 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-7bb1c2/distinct-2-w3-7bb1c2-kserve-c549df844-zt595" podUID="1916e87c-a1a4-4967-8640-8bbcd9d12903" containerName="main" probeResult="failure" output="Get \"https://10.133.0.73:8000/health\": context deadline exceeded"
Oct 30 22:13:25.359879 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:25.359782    5510 patch_prober.go:29] interesting pod/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-48cnj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused" start-of-body=
Oct 30 22:13:25.360125 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:25.360096    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-48cnj" podUID="6aa08ff6-6f78-4a6e-83bd-07cf5d43b7b0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused"
Oct 30 22:13:25.402363 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:25.402312    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-779f32/distinct-2-w0-779f32-kserve-857ffdf778-gk2h9" podUID="ef500cdd-cc52-45b0-8dd7-1896897acc46" containerName="main" probeResult="failure" output="Get \"https://10.133.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:13:25.406687 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:25.403249    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-779f32/distinct-2-w0-779f32-kserve-857ffdf778-gk2h9"
Oct 30 22:13:25.445702 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:25.439079    5510 patch_prober.go:29] interesting pod/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-l5ndh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused" start-of-body=
Oct 30 22:13:25.445983 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:25.445957    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-l5ndh" podUID="2344709a-1a48-417c-ae98-2b462bf30516" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused"
Oct 30 22:13:25.544681 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:25.533764    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-7bb1c2/distinct-2-w3-7bb1c2-kserve-c549df844-zt595" podUID="1916e87c-a1a4-4967-8640-8bbcd9d12903" containerName="main" probeResult="failure" output="Get \"https://10.133.0.73:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:13:25.654745 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:25.649905    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-c9dbe6/distinct-w5-c9dbe6-kserve-7599fd86d9-bdhxd" podUID="e903db90-f2cd-4244-b00b-f99b21873f6e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:13:25.735010 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:25.734969    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-fb0b93/distinct-w6-fb0b93-kserve-58db6b6757-6mww8" podUID="8ee6579b-acea-4492-862b-9883f70d00c9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.59:8000/ready\": context deadline exceeded"
Oct 30 22:13:25.767411 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:25.767359    5510 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-bc714d/distinct-w2-bc714d-kserve-66cfb968dd-jvgqj" podUID="9e9b11b8-1cc9-4fc8-8097-1afcffa8c422" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:13:25.942000 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:25.941908    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:13:26.045108 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:26.045070    5510 patch_prober.go:29] interesting pod/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-q6kh7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body=
Oct 30 22:13:26.045402 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:26.045369    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-q6kh7" podUID="3c82a381-05ad-436f-89dd-0985c3dbad33" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused"
Oct 30 22:13:26.073682 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:26.073055    5510 patch_prober.go:29] interesting pod/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6mvrk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" start-of-body=
Oct 30 22:13:26.073682 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:26.073116    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6mvrk" podUID="6cbfd1c6-ca09-4bee-a6d3-6461eac2c2d5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused"
Oct 30 22:13:26.073939 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:26.073890    5510 patch_prober.go:29] interesting pod/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-k5dpm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused" start-of-body=
Oct 30 22:13:26.074006 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:26.073932    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-k5dpm" podUID="8724086f-a5f7-4c63-87da-cb9228b03a0c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused"
Oct 30 22:13:26.169101 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:26.168764    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-fb0b93/premium-w6-fb0b93-kserve-5cb9948c54-w7pnk" podUID="290ae7a2-de37-4af3-82ae-a8e855b83bb9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:13:26.191827 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:26.185534    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:13:26.191827 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:26.191400    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-bc714d/distinct-w2-bc714d-kserve-66cfb968dd-jvgqj" podUID="9e9b11b8-1cc9-4fc8-8097-1afcffa8c422" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:13:26.258747 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:26.254981    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-779f32/distinct-w0-779f32-kserve-6dcbc948c7-2vzdf" podUID="e17cedd4-7d48-42b3-b127-b6192c1b3795" containerName="main" probeResult="failure" output="Get \"https://10.133.0.65:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:13:26.379626 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:26.379589    5510 patch_prober.go:29] interesting pod/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-48cnj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused" start-of-body=
Oct 30 22:13:26.379913 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:26.379885    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-48cnj" podUID="6aa08ff6-6f78-4a6e-83bd-07cf5d43b7b0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused"
Oct 30 22:13:26.464228 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:26.464170    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-779f32/distinct-2-w0-779f32-kserve-857ffdf778-gk2h9" podUID="ef500cdd-cc52-45b0-8dd7-1896897acc46" containerName="main" probeResult="failure" output="Get \"https://10.133.0.74:8000/ready\": context deadline exceeded"
Oct 30 22:13:26.488136 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:26.464820    5510 patch_prober.go:29] interesting pod/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-l5ndh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused" start-of-body=
Oct 30 22:13:26.488136 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:26.488073    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-l5ndh" podUID="2344709a-1a48-417c-ae98-2b462bf30516" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused"
Oct 30 22:13:26.534510 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:26.534473    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-7bb1c2/distinct-2-w3-7bb1c2-kserve-c549df844-zt595" podUID="1916e87c-a1a4-4967-8640-8bbcd9d12903" containerName="main" probeResult="failure" output="Get \"https://10.133.0.73:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:13:26.653480 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:26.652730    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-c9dbe6/distinct-w5-c9dbe6-kserve-7599fd86d9-bdhxd" podUID="e903db90-f2cd-4244-b00b-f99b21873f6e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:13:26.767685 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:26.757036    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-fb0b93/distinct-w6-fb0b93-kserve-58db6b6757-6mww8" podUID="8ee6579b-acea-4492-862b-9883f70d00c9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:13:27.044808 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:27.044769    5510 patch_prober.go:29] interesting pod/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-q6kh7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body=
Oct 30 22:13:27.045082 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:27.045051    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-q6kh7" podUID="3c82a381-05ad-436f-89dd-0985c3dbad33" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused"
Oct 30 22:13:27.061683 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:27.061630    5510 patch_prober.go:29] interesting pod/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-k5dpm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused" start-of-body=
Oct 30 22:13:27.061968 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:27.061934    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-k5dpm" podUID="8724086f-a5f7-4c63-87da-cb9228b03a0c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused"
Oct 30 22:13:27.065602 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:27.065573    5510 patch_prober.go:29] interesting pod/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6mvrk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" start-of-body=
Oct 30 22:13:27.065820 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:27.065795    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6mvrk" podUID="6cbfd1c6-ca09-4bee-a6d3-6461eac2c2d5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused"
Oct 30 22:13:27.174077 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:27.173999    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-fb0b93/premium-w6-fb0b93-kserve-5cb9948c54-w7pnk" podUID="290ae7a2-de37-4af3-82ae-a8e855b83bb9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": context deadline exceeded"
Oct 30 22:13:27.193803 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:27.193754    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-bc714d/distinct-w2-bc714d-kserve-66cfb968dd-jvgqj" podUID="9e9b11b8-1cc9-4fc8-8097-1afcffa8c422" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:13:27.257820 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:27.257768    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-779f32/distinct-w0-779f32-kserve-6dcbc948c7-2vzdf" podUID="e17cedd4-7d48-42b3-b127-b6192c1b3795" containerName="main" probeResult="failure" output="Get \"https://10.133.0.65:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:13:27.392738 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:27.385347    5510 patch_prober.go:29] interesting pod/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-48cnj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused" start-of-body=
Oct 30 22:13:27.392738 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:27.385424    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-48cnj" podUID="6aa08ff6-6f78-4a6e-83bd-07cf5d43b7b0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused"
Oct 30 22:13:27.447960 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:27.447923    5510 patch_prober.go:29] interesting pod/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-l5ndh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused" start-of-body=
Oct 30 22:13:27.448266 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:27.448236    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-l5ndh" podUID="2344709a-1a48-417c-ae98-2b462bf30516" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused"
Oct 30 22:13:27.540645 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:27.540599    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-779f32/distinct-2-w0-779f32-kserve-857ffdf778-gk2h9" podUID="ef500cdd-cc52-45b0-8dd7-1896897acc46" containerName="main" probeResult="failure" output="Get \"https://10.133.0.74:8000/ready\": context deadline exceeded"
Oct 30 22:13:27.553679 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:27.542072    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-7bb1c2/distinct-2-w3-7bb1c2-kserve-c549df844-zt595" podUID="1916e87c-a1a4-4967-8640-8bbcd9d12903" containerName="main" probeResult="failure" output="Get \"https://10.133.0.73:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:13:27.687752 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:27.668118    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-c9dbe6/distinct-w5-c9dbe6-kserve-7599fd86d9-bdhxd" podUID="e903db90-f2cd-4244-b00b-f99b21873f6e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:13:27.739044 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:27.739014    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-7bb1c2/distinct-2-w3-7bb1c2-kserve-c549df844-zt595"
Oct 30 22:13:27.771795 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:27.771745    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-fb0b93/distinct-w6-fb0b93-kserve-58db6b6757-6mww8" podUID="8ee6579b-acea-4492-862b-9883f70d00c9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.59:8000/ready\": context deadline exceeded"
Oct 30 22:13:27.895682 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:27.894925    5510 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w6-fb0b93/distinct-w6-fb0b93-kserve-58db6b6757-6mww8" podUID="8ee6579b-acea-4492-862b-9883f70d00c9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.59:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:13:27.961645 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:27.961540    5510 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w6-fb0b93/premium-w6-fb0b93-kserve-5cb9948c54-w7pnk" podUID="290ae7a2-de37-4af3-82ae-a8e855b83bb9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/health\": context deadline exceeded"
Oct 30 22:13:28.092682 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:28.092624    5510 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-779f32/distinct-2-w0-779f32-kserve-857ffdf778-gk2h9" podUID="ef500cdd-cc52-45b0-8dd7-1896897acc46" containerName="main" probeResult="failure" output="Get \"https://10.133.0.74:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:13:28.093272 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:28.093246    5510 patch_prober.go:29] interesting pod/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6mvrk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" start-of-body=
Oct 30 22:13:28.119728 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:28.119686    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6mvrk" podUID="6cbfd1c6-ca09-4bee-a6d3-6461eac2c2d5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused"
Oct 30 22:13:28.127240 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:28.097877    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-bc714d/distinct-w2-bc714d-kserve-66cfb968dd-jvgqj"
Oct 30 22:13:28.130155 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:28.096727    5510 patch_prober.go:29] interesting pod/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-k5dpm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused" start-of-body=
Oct 30 22:13:28.130375 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:28.130351    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-k5dpm" podUID="8724086f-a5f7-4c63-87da-cb9228b03a0c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused"
Oct 30 22:13:28.130507 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:28.117154    5510 patch_prober.go:29] interesting pod/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-q6kh7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body=
Oct 30 22:13:28.130621 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:28.130603    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-q6kh7" podUID="3c82a381-05ad-436f-89dd-0985c3dbad33" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused"
Oct 30 22:13:28.234191 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:28.234091    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-fb0b93/premium-w6-fb0b93-kserve-5cb9948c54-w7pnk" podUID="290ae7a2-de37-4af3-82ae-a8e855b83bb9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:13:28.235774 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:28.235197    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-fb0b93/distinct-w6-fb0b93-kserve-58db6b6757-6mww8"
Oct 30 22:13:28.265698 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:28.263990    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-779f32/distinct-w0-779f32-kserve-6dcbc948c7-2vzdf" podUID="e17cedd4-7d48-42b3-b127-b6192c1b3795" containerName="main" probeResult="failure" output="Get \"https://10.133.0.65:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:13:28.353683 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:28.350130    5510 patch_prober.go:29] interesting pod/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-48cnj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused" start-of-body=
Oct 30 22:13:28.353683 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:28.350212    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-48cnj" podUID="6aa08ff6-6f78-4a6e-83bd-07cf5d43b7b0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused"
Oct 30 22:13:28.464682 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:28.443750    5510 patch_prober.go:29] interesting pod/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-l5ndh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused" start-of-body=
Oct 30 22:13:28.464682 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:28.443826    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-l5ndh" podUID="2344709a-1a48-417c-ae98-2b462bf30516" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused"
Oct 30 22:13:28.588814 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:28.588749    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-779f32/distinct-2-w0-779f32-kserve-857ffdf778-gk2h9" podUID="ef500cdd-cc52-45b0-8dd7-1896897acc46" containerName="main" probeResult="failure" output="Get \"https://10.133.0.74:8000/ready\": context deadline exceeded"
Oct 30 22:13:28.683763 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:28.683719    5510 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-779f32/distinct-w0-779f32-kserve-6dcbc948c7-2vzdf" podUID="e17cedd4-7d48-42b3-b127-b6192c1b3795" containerName="main" probeResult="failure" output="Get \"https://10.133.0.65:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:13:28.959889 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:28.711815    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-c9dbe6/distinct-w5-c9dbe6-kserve-7599fd86d9-bdhxd" podUID="e903db90-f2cd-4244-b00b-f99b21873f6e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.67:8000/ready\": context deadline exceeded"
Oct 30 22:13:29.082689 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:29.044683    5510 patch_prober.go:29] interesting pod/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-q6kh7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body=
Oct 30 22:13:29.100682 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:29.082959    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-q6kh7" podUID="3c82a381-05ad-436f-89dd-0985c3dbad33" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused"
Oct 30 22:13:29.125725 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:29.055105    5510 patch_prober.go:29] interesting pod/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6mvrk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" start-of-body=
Oct 30 22:13:29.129682 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:29.126000    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6mvrk" podUID="6cbfd1c6-ca09-4bee-a6d3-6461eac2c2d5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused"
Oct 30 22:13:29.141799 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:29.071153    5510 patch_prober.go:29] interesting pod/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-k5dpm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused" start-of-body=
Oct 30 22:13:29.142084 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:29.142030    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-k5dpm" podUID="8724086f-a5f7-4c63-87da-cb9228b03a0c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused"
Oct 30 22:13:29.261876 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:29.261764    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-fb0b93/premium-w6-fb0b93-kserve-5cb9948c54-w7pnk" podUID="290ae7a2-de37-4af3-82ae-a8e855b83bb9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:13:29.364690 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:29.345863    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-779f32/distinct-w0-779f32-kserve-6dcbc948c7-2vzdf" podUID="e17cedd4-7d48-42b3-b127-b6192c1b3795" containerName="main" probeResult="failure" output="Get \"https://10.133.0.65:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:13:29.390186 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:29.366302    5510 patch_prober.go:29] interesting pod/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-48cnj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused" start-of-body=
Oct 30 22:13:29.390537 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:29.390500    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-48cnj" podUID="6aa08ff6-6f78-4a6e-83bd-07cf5d43b7b0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused"
Oct 30 22:13:29.514422 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:29.492415    5510 patch_prober.go:29] interesting pod/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-l5ndh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused" start-of-body=
Oct 30 22:13:29.514422 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:29.492499    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-l5ndh" podUID="2344709a-1a48-417c-ae98-2b462bf30516" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused"
Oct 30 22:13:29.588120 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:29.583207    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-7bb1c2/distinct-2-w3-7bb1c2-kserve-c549df844-zt595" podUID="1916e87c-a1a4-4967-8640-8bbcd9d12903" containerName="main" probeResult="failure" output="Get \"https://10.133.0.73:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:13:29.605684 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:29.589436    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-779f32/distinct-2-w0-779f32-kserve-857ffdf778-gk2h9" podUID="ef500cdd-cc52-45b0-8dd7-1896897acc46" containerName="main" probeResult="failure" output="Get \"https://10.133.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:13:29.725854 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:29.725820    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-779f32/distinct-2-w0-779f32-kserve-857ffdf778-gk2h9"
Oct 30 22:13:30.006193 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:30.003895    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-c9dbe6/distinct-w5-c9dbe6-kserve-7599fd86d9-bdhxd" podUID="e903db90-f2cd-4244-b00b-f99b21873f6e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.67:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:13:30.034259 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:30.034228    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-c9dbe6/distinct-w5-c9dbe6-kserve-7599fd86d9-bdhxd"
Oct 30 22:13:30.034528 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:30.034512    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-779f32/distinct-w0-779f32-kserve-6dcbc948c7-2vzdf"
Oct 30 22:13:30.041571 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:30.041547    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-fb0b93/premium-w6-fb0b93-kserve-5cb9948c54-w7pnk"
Oct 30 22:13:30.042819 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:30.042792    5510 patch_prober.go:29] interesting pod/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-q6kh7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body=
Oct 30 22:13:30.043014 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:30.042988    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-q6kh7" podUID="3c82a381-05ad-436f-89dd-0985c3dbad33" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused"
Oct 30 22:13:30.055030 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:30.055004    5510 patch_prober.go:29] interesting pod/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6mvrk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" start-of-body=
Oct 30 22:13:30.056775 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:30.055418    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6mvrk" podUID="6cbfd1c6-ca09-4bee-a6d3-6461eac2c2d5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused"
Oct 30 22:13:30.061748 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:30.061726    5510 patch_prober.go:29] interesting pod/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-k5dpm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused" start-of-body=
Oct 30 22:13:30.061926 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:30.061903    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-k5dpm" podUID="8724086f-a5f7-4c63-87da-cb9228b03a0c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused"
Oct 30 22:13:30.393002 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:30.392967    5510 patch_prober.go:29] interesting pod/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-48cnj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused" start-of-body=
Oct 30 22:13:30.395082 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:30.393556    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-48cnj" podUID="6aa08ff6-6f78-4a6e-83bd-07cf5d43b7b0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused"
Oct 30 22:13:30.418945 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:30.417828    5510 patch_prober.go:29] interesting pod/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-l5ndh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused" start-of-body=
Oct 30 22:13:30.418945 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:30.417914    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-l5ndh" podUID="2344709a-1a48-417c-ae98-2b462bf30516" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused"
Oct 30 22:13:30.796794 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:30.778565    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:13:31.071549 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:31.071377    5510 patch_prober.go:29] interesting pod/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-k5dpm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused" start-of-body=
Oct 30 22:13:31.071549 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:31.071431    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-k5dpm" podUID="8724086f-a5f7-4c63-87da-cb9228b03a0c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused"
Oct 30 22:13:31.183995 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:31.182574    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6mvrk"
Oct 30 22:13:31.183995 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:31.183874    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6mvrk"
Oct 30 22:13:31.186405 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:31.186382    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6mvrk"
Oct 30 22:13:31.366745 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:31.356712    5510 patch_prober.go:29] interesting pod/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-48cnj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused" start-of-body=
Oct 30 22:13:31.366745 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:31.356859    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-48cnj" podUID="6aa08ff6-6f78-4a6e-83bd-07cf5d43b7b0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused"
Oct 30 22:13:32.049963 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:32.049925    5510 patch_prober.go:29] interesting pod/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-q6kh7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 22:13:32.050550 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:32.050519    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-q6kh7" podUID="3c82a381-05ad-436f-89dd-0985c3dbad33" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:13:32.156040 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:32.156000    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-q6kh7"
Oct 30 22:13:32.158029 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:32.158005    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-q6kh7"
Oct 30 22:13:32.187717 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:32.179840    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-q6kh7"
Oct 30 22:13:32.722685 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:32.691729    5510 patch_prober.go:29] interesting pod/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-l5ndh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.79:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 22:13:32.722685 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:32.691830    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-l5ndh" podUID="2344709a-1a48-417c-ae98-2b462bf30516" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.79:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:13:33.060737 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:33.010519    5510 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-f969x"]
Oct 30 22:13:33.093681 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:33.076752    5510 patch_prober.go:29] interesting pod/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-k5dpm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.78:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 22:13:33.093681 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:33.076823    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-k5dpm" podUID="8724086f-a5f7-4c63-87da-cb9228b03a0c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.78:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:13:33.120627 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:33.120599    5510 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-v4cxn"]
Oct 30 22:13:33.134734 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:33.131290    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-fb0b93/premium-w6-fb0b93-kserve-5cb9948c54-w7pnk" podUID="290ae7a2-de37-4af3-82ae-a8e855b83bb9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": context deadline exceeded"
Oct 30 22:13:33.135285 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:33.131412    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-f969x"
Oct 30 22:13:33.144024 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:33.143995    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-v4cxn"
Oct 30 22:13:33.203943 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:33.203911    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-f969x"]
Oct 30 22:13:33.306687 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:33.294427    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5542b62f-bc73-47d7-8615-180f43324911-istio-podinfo\") pod \"e2e-worker-w2-bc714d-openshift-default-67558b95cf-v4cxn\" (UID: \"5542b62f-bc73-47d7-8615-180f43324911\") " pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-v4cxn"
Oct 30 22:13:33.306687 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:33.294510    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b0d37b4a-b986-438a-a811-9037505f0e6a-workload-certs\") pod \"e2e-worker-w2-bc714d-openshift-default-67558b95cf-f969x\" (UID: \"b0d37b4a-b986-438a-a811-9037505f0e6a\") " pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-f969x"
Oct 30 22:13:33.306687 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:33.294553    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5542b62f-bc73-47d7-8615-180f43324911-istio-envoy\") pod \"e2e-worker-w2-bc714d-openshift-default-67558b95cf-v4cxn\" (UID: \"5542b62f-bc73-47d7-8615-180f43324911\") " pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-v4cxn"
Oct 30 22:13:33.306687 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:33.294635    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b0d37b4a-b986-438a-a811-9037505f0e6a-istio-podinfo\") pod \"e2e-worker-w2-bc714d-openshift-default-67558b95cf-f969x\" (UID: \"b0d37b4a-b986-438a-a811-9037505f0e6a\") " pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-f969x"
Oct 30 22:13:33.306687 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:33.294717    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5542b62f-bc73-47d7-8615-180f43324911-istiod-ca-cert\") pod \"e2e-worker-w2-bc714d-openshift-default-67558b95cf-v4cxn\" (UID: \"5542b62f-bc73-47d7-8615-180f43324911\") " pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-v4cxn"
Oct 30 22:13:33.306687 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:33.294746    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b0d37b4a-b986-438a-a811-9037505f0e6a-istio-envoy\") pod \"e2e-worker-w2-bc714d-openshift-default-67558b95cf-f969x\" (UID: \"b0d37b4a-b986-438a-a811-9037505f0e6a\") " pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-f969x"
Oct 30 22:13:33.306687 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:33.294789    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b0d37b4a-b986-438a-a811-9037505f0e6a-istio-token\") pod \"e2e-worker-w2-bc714d-openshift-default-67558b95cf-f969x\" (UID: \"b0d37b4a-b986-438a-a811-9037505f0e6a\") " pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-f969x"
Oct 30 22:13:33.306687 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:33.294862    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b0d37b4a-b986-438a-a811-9037505f0e6a-istiod-ca-cert\") pod \"e2e-worker-w2-bc714d-openshift-default-67558b95cf-f969x\" (UID: \"b0d37b4a-b986-438a-a811-9037505f0e6a\") " pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-f969x"
Oct 30 22:13:33.306687 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:33.294889    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0d37b4a-b986-438a-a811-9037505f0e6a-credential-socket\") pod \"e2e-worker-w2-bc714d-openshift-default-67558b95cf-f969x\" (UID: \"b0d37b4a-b986-438a-a811-9037505f0e6a\") " pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-f969x"
Oct 30 22:13:33.306687 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:33.294931    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5542b62f-bc73-47d7-8615-180f43324911-istio-data\") pod \"e2e-worker-w2-bc714d-openshift-default-67558b95cf-v4cxn\" (UID: \"5542b62f-bc73-47d7-8615-180f43324911\") " pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-v4cxn"
Oct 30 22:13:33.306687 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:33.294994    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5542b62f-bc73-47d7-8615-180f43324911-workload-socket\") pod \"e2e-worker-w2-bc714d-openshift-default-67558b95cf-v4cxn\" (UID: \"5542b62f-bc73-47d7-8615-180f43324911\") " pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-v4cxn"
Oct 30 22:13:33.306687 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:33.295033    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5542b62f-bc73-47d7-8615-180f43324911-istio-token\") pod \"e2e-worker-w2-bc714d-openshift-default-67558b95cf-v4cxn\" (UID: \"5542b62f-bc73-47d7-8615-180f43324911\") " pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-v4cxn"
Oct 30 22:13:33.306687 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:33.295113    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5542b62f-bc73-47d7-8615-180f43324911-workload-certs\") pod \"e2e-worker-w2-bc714d-openshift-default-67558b95cf-v4cxn\" (UID: \"5542b62f-bc73-47d7-8615-180f43324911\") " pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-v4cxn"
Oct 30 22:13:33.306687 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:33.295137    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xhh86\" (UniqueName: \"kubernetes.io/projected/5542b62f-bc73-47d7-8615-180f43324911-kube-api-access-xhh86\") pod \"e2e-worker-w2-bc714d-openshift-default-67558b95cf-v4cxn\" (UID: \"5542b62f-bc73-47d7-8615-180f43324911\") " pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-v4cxn"
Oct 30 22:13:33.306687 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:33.295182    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0d37b4a-b986-438a-a811-9037505f0e6a-workload-socket\") pod \"e2e-worker-w2-bc714d-openshift-default-67558b95cf-f969x\" (UID: \"b0d37b4a-b986-438a-a811-9037505f0e6a\") " pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-f969x"
Oct 30 22:13:33.306687 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:33.295208    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b0d37b4a-b986-438a-a811-9037505f0e6a-istio-data\") pod \"e2e-worker-w2-bc714d-openshift-default-67558b95cf-f969x\" (UID: \"b0d37b4a-b986-438a-a811-9037505f0e6a\") " pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-f969x"
Oct 30 22:13:33.306687 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:33.295259    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5542b62f-bc73-47d7-8615-180f43324911-credential-socket\") pod \"e2e-worker-w2-bc714d-openshift-default-67558b95cf-v4cxn\" (UID: \"5542b62f-bc73-47d7-8615-180f43324911\") " pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-v4cxn"
Oct 30 22:13:33.306687 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:33.295343    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6nvw5\" (UniqueName: \"kubernetes.io/projected/b0d37b4a-b986-438a-a811-9037505f0e6a-kube-api-access-6nvw5\") pod \"e2e-worker-w2-bc714d-openshift-default-67558b95cf-f969x\" (UID: \"b0d37b4a-b986-438a-a811-9037505f0e6a\") " pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-f969x"
Oct 30 22:13:33.381741 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:33.324626    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-l5ndh"
Oct 30 22:13:33.443208 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:33.354736    5510 patch_prober.go:29] interesting pod/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-48cnj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.76:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 22:13:33.443469 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:33.443440    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-48cnj" podUID="6aa08ff6-6f78-4a6e-83bd-07cf5d43b7b0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.76:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:13:33.443934 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:33.442217    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b0d37b4a-b986-438a-a811-9037505f0e6a-istiod-ca-cert\") pod \"e2e-worker-w2-bc714d-openshift-default-67558b95cf-f969x\" (UID: \"b0d37b4a-b986-438a-a811-9037505f0e6a\") " pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-f969x"
Oct 30 22:13:33.444092 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:33.444074    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0d37b4a-b986-438a-a811-9037505f0e6a-credential-socket\") pod \"e2e-worker-w2-bc714d-openshift-default-67558b95cf-f969x\" (UID: \"b0d37b4a-b986-438a-a811-9037505f0e6a\") " pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-f969x"
Oct 30 22:13:33.444228 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:33.444213    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5542b62f-bc73-47d7-8615-180f43324911-istio-data\") pod \"e2e-worker-w2-bc714d-openshift-default-67558b95cf-v4cxn\" (UID: \"5542b62f-bc73-47d7-8615-180f43324911\") " pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-v4cxn"
Oct 30 22:13:33.444496 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:33.444475    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5542b62f-bc73-47d7-8615-180f43324911-workload-socket\") pod \"e2e-worker-w2-bc714d-openshift-default-67558b95cf-v4cxn\" (UID: \"5542b62f-bc73-47d7-8615-180f43324911\") " pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-v4cxn"
Oct 30 22:13:33.444637 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:33.444622    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5542b62f-bc73-47d7-8615-180f43324911-istio-token\") pod \"e2e-worker-w2-bc714d-openshift-default-67558b95cf-v4cxn\" (UID: \"5542b62f-bc73-47d7-8615-180f43324911\") " pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-v4cxn"
Oct 30 22:13:33.446480 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:33.446456    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5542b62f-bc73-47d7-8615-180f43324911-workload-certs\") pod \"e2e-worker-w2-bc714d-openshift-default-67558b95cf-v4cxn\" (UID: \"5542b62f-bc73-47d7-8615-180f43324911\") " pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-v4cxn"
Oct 30 22:13:33.446706 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:33.446677    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xhh86\" (UniqueName: \"kubernetes.io/projected/5542b62f-bc73-47d7-8615-180f43324911-kube-api-access-xhh86\") pod \"e2e-worker-w2-bc714d-openshift-default-67558b95cf-v4cxn\" (UID: \"5542b62f-bc73-47d7-8615-180f43324911\") " pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-v4cxn"
Oct 30 22:13:33.446871 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:33.446854    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0d37b4a-b986-438a-a811-9037505f0e6a-workload-socket\") pod \"e2e-worker-w2-bc714d-openshift-default-67558b95cf-f969x\" (UID: \"b0d37b4a-b986-438a-a811-9037505f0e6a\") " pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-f969x"
Oct 30 22:13:33.446991 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:33.446975    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b0d37b4a-b986-438a-a811-9037505f0e6a-istio-data\") pod \"e2e-worker-w2-bc714d-openshift-default-67558b95cf-f969x\" (UID: \"b0d37b4a-b986-438a-a811-9037505f0e6a\") " pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-f969x"
Oct 30 22:13:33.447128 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:33.447114    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5542b62f-bc73-47d7-8615-180f43324911-credential-socket\") pod \"e2e-worker-w2-bc714d-openshift-default-67558b95cf-v4cxn\" (UID: \"5542b62f-bc73-47d7-8615-180f43324911\") " pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-v4cxn"
Oct 30 22:13:33.447312 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:33.447294    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6nvw5\" (UniqueName: \"kubernetes.io/projected/b0d37b4a-b986-438a-a811-9037505f0e6a-kube-api-access-6nvw5\") pod \"e2e-worker-w2-bc714d-openshift-default-67558b95cf-f969x\" (UID: \"b0d37b4a-b986-438a-a811-9037505f0e6a\") " pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-f969x"
Oct 30 22:13:33.447437 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:33.447420    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5542b62f-bc73-47d7-8615-180f43324911-istio-podinfo\") pod \"e2e-worker-w2-bc714d-openshift-default-67558b95cf-v4cxn\" (UID: \"5542b62f-bc73-47d7-8615-180f43324911\") " pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-v4cxn"
Oct 30 22:13:33.447589 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:33.447574    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b0d37b4a-b986-438a-a811-9037505f0e6a-workload-certs\") pod \"e2e-worker-w2-bc714d-openshift-default-67558b95cf-f969x\" (UID: \"b0d37b4a-b986-438a-a811-9037505f0e6a\") " pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-f969x"
Oct 30 22:13:33.447748 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:33.447731    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5542b62f-bc73-47d7-8615-180f43324911-istio-envoy\") pod \"e2e-worker-w2-bc714d-openshift-default-67558b95cf-v4cxn\" (UID: \"5542b62f-bc73-47d7-8615-180f43324911\") " pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-v4cxn"
Oct 30 22:13:33.447961 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:33.447945    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b0d37b4a-b986-438a-a811-9037505f0e6a-istio-podinfo\") pod \"e2e-worker-w2-bc714d-openshift-default-67558b95cf-f969x\" (UID: \"b0d37b4a-b986-438a-a811-9037505f0e6a\") " pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-f969x"
Oct 30 22:13:33.448117 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:33.448102    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5542b62f-bc73-47d7-8615-180f43324911-istiod-ca-cert\") pod \"e2e-worker-w2-bc714d-openshift-default-67558b95cf-v4cxn\" (UID: \"5542b62f-bc73-47d7-8615-180f43324911\") " pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-v4cxn"
Oct 30 22:13:33.448230 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:33.448216    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b0d37b4a-b986-438a-a811-9037505f0e6a-istio-envoy\") pod \"e2e-worker-w2-bc714d-openshift-default-67558b95cf-f969x\" (UID: \"b0d37b4a-b986-438a-a811-9037505f0e6a\") " pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-f969x"
Oct 30 22:13:33.448366 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:33.448350    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b0d37b4a-b986-438a-a811-9037505f0e6a-istio-token\") pod \"e2e-worker-w2-bc714d-openshift-default-67558b95cf-f969x\" (UID: \"b0d37b4a-b986-438a-a811-9037505f0e6a\") " pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-f969x"
Oct 30 22:13:33.448929 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:33.443100    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b0d37b4a-b986-438a-a811-9037505f0e6a-istiod-ca-cert\") pod \"e2e-worker-w2-bc714d-openshift-default-67558b95cf-f969x\" (UID: \"b0d37b4a-b986-438a-a811-9037505f0e6a\") " pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-f969x"
Oct 30 22:13:33.449385 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:33.449363    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0d37b4a-b986-438a-a811-9037505f0e6a-credential-socket\") pod \"e2e-worker-w2-bc714d-openshift-default-67558b95cf-f969x\" (UID: \"b0d37b4a-b986-438a-a811-9037505f0e6a\") " pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-f969x"
Oct 30 22:13:33.450083 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:33.450042    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5542b62f-bc73-47d7-8615-180f43324911-istio-data\") pod \"e2e-worker-w2-bc714d-openshift-default-67558b95cf-v4cxn\" (UID: \"5542b62f-bc73-47d7-8615-180f43324911\") " pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-v4cxn"
Oct 30 22:13:33.450485 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:33.450464    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5542b62f-bc73-47d7-8615-180f43324911-workload-socket\") pod \"e2e-worker-w2-bc714d-openshift-default-67558b95cf-v4cxn\" (UID: \"5542b62f-bc73-47d7-8615-180f43324911\") " pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-v4cxn"
Oct 30 22:13:33.451128 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:33.451106    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5542b62f-bc73-47d7-8615-180f43324911-workload-certs\") pod \"e2e-worker-w2-bc714d-openshift-default-67558b95cf-v4cxn\" (UID: \"5542b62f-bc73-47d7-8615-180f43324911\") " pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-v4cxn"
Oct 30 22:13:33.451751 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:33.451730    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0d37b4a-b986-438a-a811-9037505f0e6a-workload-socket\") pod \"e2e-worker-w2-bc714d-openshift-default-67558b95cf-f969x\" (UID: \"b0d37b4a-b986-438a-a811-9037505f0e6a\") " pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-f969x"
Oct 30 22:13:33.452113 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:33.452092    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b0d37b4a-b986-438a-a811-9037505f0e6a-istio-data\") pod \"e2e-worker-w2-bc714d-openshift-default-67558b95cf-f969x\" (UID: \"b0d37b4a-b986-438a-a811-9037505f0e6a\") " pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-f969x"
Oct 30 22:13:33.452474 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:33.452451    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5542b62f-bc73-47d7-8615-180f43324911-credential-socket\") pod \"e2e-worker-w2-bc714d-openshift-default-67558b95cf-v4cxn\" (UID: \"5542b62f-bc73-47d7-8615-180f43324911\") " pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-v4cxn"
Oct 30 22:13:33.486290 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:33.485606    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b0d37b4a-b986-438a-a811-9037505f0e6a-istio-podinfo\") pod \"e2e-worker-w2-bc714d-openshift-default-67558b95cf-f969x\" (UID: \"b0d37b4a-b986-438a-a811-9037505f0e6a\") " pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-f969x"
Oct 30 22:13:33.486290 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:33.485954    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b0d37b4a-b986-438a-a811-9037505f0e6a-workload-certs\") pod \"e2e-worker-w2-bc714d-openshift-default-67558b95cf-f969x\" (UID: \"b0d37b4a-b986-438a-a811-9037505f0e6a\") " pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-f969x"
Oct 30 22:13:33.489128 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:33.488606    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5542b62f-bc73-47d7-8615-180f43324911-istiod-ca-cert\") pod \"e2e-worker-w2-bc714d-openshift-default-67558b95cf-v4cxn\" (UID: \"5542b62f-bc73-47d7-8615-180f43324911\") " pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-v4cxn"
Oct 30 22:13:33.489128 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:33.488849    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5542b62f-bc73-47d7-8615-180f43324911-istio-envoy\") pod \"e2e-worker-w2-bc714d-openshift-default-67558b95cf-v4cxn\" (UID: \"5542b62f-bc73-47d7-8615-180f43324911\") " pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-v4cxn"
Oct 30 22:13:33.491925 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:33.491542    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b0d37b4a-b986-438a-a811-9037505f0e6a-istio-envoy\") pod \"e2e-worker-w2-bc714d-openshift-default-67558b95cf-f969x\" (UID: \"b0d37b4a-b986-438a-a811-9037505f0e6a\") " pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-f969x"
Oct 30 22:13:33.552679 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:33.516237    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5542b62f-bc73-47d7-8615-180f43324911-istio-podinfo\") pod \"e2e-worker-w2-bc714d-openshift-default-67558b95cf-v4cxn\" (UID: \"5542b62f-bc73-47d7-8615-180f43324911\") " pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-v4cxn"
Oct 30 22:13:33.552679 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:33.535750    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-7bb1c2/distinct-2-w3-7bb1c2-kserve-c549df844-zt595" podUID="1916e87c-a1a4-4967-8640-8bbcd9d12903" containerName="main" probeResult="failure" output="Get \"https://10.133.0.73:8000/ready\": context deadline exceeded"
Oct 30 22:13:33.592677 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:33.567497    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-v4cxn"]
Oct 30 22:13:33.592677 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:33.578889    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-779f32/distinct-2-w0-779f32-kserve-857ffdf778-gk2h9" podUID="ef500cdd-cc52-45b0-8dd7-1896897acc46" containerName="main" probeResult="failure" output="Get \"https://10.133.0.74:8000/ready\": context deadline exceeded"
Oct 30 22:13:33.666733 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:33.662936    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xhh86\" (UniqueName: \"kubernetes.io/projected/5542b62f-bc73-47d7-8615-180f43324911-kube-api-access-xhh86\") pod \"e2e-worker-w2-bc714d-openshift-default-67558b95cf-v4cxn\" (UID: \"5542b62f-bc73-47d7-8615-180f43324911\") " pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-v4cxn"
Oct 30 22:13:33.696618 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:33.689390    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5542b62f-bc73-47d7-8615-180f43324911-istio-token\") pod \"e2e-worker-w2-bc714d-openshift-default-67558b95cf-v4cxn\" (UID: \"5542b62f-bc73-47d7-8615-180f43324911\") " pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-v4cxn"
Oct 30 22:13:33.700568 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:33.700542    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-48cnj"
Oct 30 22:13:33.704853 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:33.704829    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-l5ndh"
Oct 30 22:13:33.705049 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:33.705034    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-l5ndh"
Oct 30 22:13:33.714768 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:33.714735    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-48cnj"
Oct 30 22:13:33.743699 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:33.743373    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6nvw5\" (UniqueName: \"kubernetes.io/projected/b0d37b4a-b986-438a-a811-9037505f0e6a-kube-api-access-6nvw5\") pod \"e2e-worker-w2-bc714d-openshift-default-67558b95cf-f969x\" (UID: \"b0d37b4a-b986-438a-a811-9037505f0e6a\") " pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-f969x"
Oct 30 22:13:33.768693 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:33.767492    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-48cnj"
Oct 30 22:13:33.777081 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:33.777051    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b0d37b4a-b986-438a-a811-9037505f0e6a-istio-token\") pod \"e2e-worker-w2-bc714d-openshift-default-67558b95cf-f969x\" (UID: \"b0d37b4a-b986-438a-a811-9037505f0e6a\") " pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-f969x"
Oct 30 22:13:33.797340 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:33.797312    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-v4cxn"
Oct 30 22:13:33.962405 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:33.962311    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-f969x"
Oct 30 22:13:34.079692 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:34.075943    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-k5dpm"
Oct 30 22:13:34.214736 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:34.210462    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-bc714d/distinct-w2-bc714d-kserve-66cfb968dd-jvgqj" podUID="9e9b11b8-1cc9-4fc8-8097-1afcffa8c422" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:13:34.214736 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:34.212767    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-fb0b93/premium-w6-fb0b93-kserve-5cb9948c54-w7pnk" podUID="290ae7a2-de37-4af3-82ae-a8e855b83bb9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:13:34.214736 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:34.213089    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-fb0b93/premium-w6-fb0b93-kserve-5cb9948c54-w7pnk"
Oct 30 22:13:34.229146 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:34.228765    5510 patch_prober.go:29] interesting pod/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-k5dpm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.78:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 22:13:34.229146 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:34.228824    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-k5dpm" podUID="8724086f-a5f7-4c63-87da-cb9228b03a0c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.78:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:13:34.260682 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:34.257012    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-779f32/distinct-w0-779f32-kserve-6dcbc948c7-2vzdf" podUID="e17cedd4-7d48-42b3-b127-b6192c1b3795" containerName="main" probeResult="failure" output="Get \"https://10.133.0.65:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:13:34.543629 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:34.543584    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-7bb1c2/distinct-2-w3-7bb1c2-kserve-c549df844-zt595" podUID="1916e87c-a1a4-4967-8640-8bbcd9d12903" containerName="main" probeResult="failure" output="Get \"https://10.133.0.73:8000/ready\": context deadline exceeded"
Oct 30 22:13:34.548117 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:34.548085    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-7bb1c2/distinct-2-w3-7bb1c2-kserve-c549df844-zt595"
Oct 30 22:13:34.678870 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:34.678821    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-c9dbe6/distinct-w5-c9dbe6-kserve-7599fd86d9-bdhxd" podUID="e903db90-f2cd-4244-b00b-f99b21873f6e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:13:34.877682 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:34.765938    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-fb0b93/distinct-w6-fb0b93-kserve-58db6b6757-6mww8" podUID="8ee6579b-acea-4492-862b-9883f70d00c9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:13:34.993679 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:34.986057    5510 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-7bb1c2/distinct-2-w3-7bb1c2-kserve-c549df844-zt595" podUID="1916e87c-a1a4-4967-8640-8bbcd9d12903" containerName="main" probeResult="failure" output="Get \"https://10.133.0.73:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:13:35.222914 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:35.222817    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-bc714d/distinct-w2-bc714d-kserve-66cfb968dd-jvgqj" podUID="9e9b11b8-1cc9-4fc8-8097-1afcffa8c422" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:13:35.224197 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:35.224170    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-bc714d/distinct-w2-bc714d-kserve-66cfb968dd-jvgqj"
Oct 30 22:13:35.226274 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:35.225860    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-fb0b93/premium-w6-fb0b93-kserve-5cb9948c54-w7pnk" podUID="290ae7a2-de37-4af3-82ae-a8e855b83bb9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:13:35.283830 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:35.271573    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-779f32/distinct-w0-779f32-kserve-6dcbc948c7-2vzdf" podUID="e17cedd4-7d48-42b3-b127-b6192c1b3795" containerName="main" probeResult="failure" output="Get \"https://10.133.0.65:8000/ready\": context deadline exceeded"
Oct 30 22:13:35.283830 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:35.271791    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-779f32/distinct-w0-779f32-kserve-6dcbc948c7-2vzdf"
Oct 30 22:13:35.361788 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:35.361748    5510 patch_prober.go:29] interesting pod/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-k5dpm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.78:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 22:13:35.365866 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:35.362011    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-k5dpm" podUID="8724086f-a5f7-4c63-87da-cb9228b03a0c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.78:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:13:35.430880 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:35.430839    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-779f32/distinct-w0-779f32-kserve-6dcbc948c7-2vzdf"
Oct 30 22:13:35.653175 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:35.653118    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-7bb1c2/distinct-2-w3-7bb1c2-kserve-c549df844-zt595" podUID="1916e87c-a1a4-4967-8640-8bbcd9d12903" containerName="main" probeResult="failure" output="Get \"https://10.133.0.73:8000/ready\": context deadline exceeded"
Oct 30 22:13:35.885737 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:35.861029    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-779f32/distinct-2-w0-779f32-kserve-857ffdf778-gk2h9" podUID="ef500cdd-cc52-45b0-8dd7-1896897acc46" containerName="main" probeResult="failure" output="Get \"https://10.133.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:13:36.002326 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:36.002286    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-c9dbe6/distinct-w5-c9dbe6-kserve-7599fd86d9-bdhxd" podUID="e903db90-f2cd-4244-b00b-f99b21873f6e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:13:36.021684 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:36.021376    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-fb0b93/distinct-w6-fb0b93-kserve-58db6b6757-6mww8" podUID="8ee6579b-acea-4492-862b-9883f70d00c9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.59:8000/ready\": context deadline exceeded"
Oct 30 22:13:36.146829 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:36.146477    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:13:36.146829 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:36.146621    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:13:36.146829 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:36.146734    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:13:36.219937 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:36.219903    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-fb0b93/premium-w6-fb0b93-kserve-5cb9948c54-w7pnk"
Oct 30 22:13:36.220322 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:36.220303    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-c9dbe6/distinct-w5-c9dbe6-kserve-7599fd86d9-bdhxd"
Oct 30 22:13:36.220557 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:36.220541    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-fb0b93/distinct-w6-fb0b93-kserve-58db6b6757-6mww8"
Oct 30 22:13:36.221711 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:36.220917    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-c9dbe6/distinct-w5-c9dbe6-kserve-7599fd86d9-bdhxd"
Oct 30 22:13:36.222585 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:36.222261    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-bc714d/distinct-w2-bc714d-kserve-66cfb968dd-jvgqj"
Oct 30 22:13:36.233534 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:36.233438    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-k5dpm"
Oct 30 22:13:36.235856 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:36.235062    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-7bb1c2/distinct-2-w3-7bb1c2-kserve-c549df844-zt595"
Oct 30 22:13:36.239750 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:36.239727    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-fb0b93/distinct-w6-fb0b93-kserve-58db6b6757-6mww8"
Oct 30 22:13:36.239920 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:36.239905    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-k5dpm"
Oct 30 22:13:36.255312 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:36.255247    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-v4cxn"]
Oct 30 22:13:36.335913 ip-10-0-136-210 kubenswrapper[5510]: W1030 22:13:36.335880    5510 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb0d37b4a_b986_438a_a811_9037505f0e6a.slice/crio-3e9e71fdf38e83391ea1c00e9ae7c4f6b50abea630ea214be94b0a710d0f3cd5 WatchSource:0}: Error finding container 3e9e71fdf38e83391ea1c00e9ae7c4f6b50abea630ea214be94b0a710d0f3cd5: Status 404 returned error can't find the container with id 3e9e71fdf38e83391ea1c00e9ae7c4f6b50abea630ea214be94b0a710d0f3cd5
Oct 30 22:13:36.340814 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:36.340775    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:13:36.340937 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:36.340855    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:13:36.340937 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:36.340919    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:13:36.341554 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:36.341529    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-f969x"]
Oct 30 22:13:36.908160 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:36.908131    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:13:37.228146 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:37.227970    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-v4cxn" event={"ID":"5542b62f-bc73-47d7-8615-180f43324911","Type":"ContainerStarted","Data":"c6b85d95983fbf83ac908236d7f59308ddfbd8918e9314137276f3ecd11bfce6"}
Oct 30 22:13:37.228146 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:37.228036    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-v4cxn" event={"ID":"5542b62f-bc73-47d7-8615-180f43324911","Type":"ContainerStarted","Data":"941b827793a7cd2a808186af7148024ddd475cd61bde3ed027b95a67412d68e1"}
Oct 30 22:13:37.231575 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:37.231543    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-f969x" event={"ID":"b0d37b4a-b986-438a-a811-9037505f0e6a","Type":"ContainerStarted","Data":"ec2bae482f04cc37a8b77e7640794dd7b79c3183380a1e47d15be53d8bf7a675"}
Oct 30 22:13:37.231765 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:37.231589    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-f969x" event={"ID":"b0d37b4a-b986-438a-a811-9037505f0e6a","Type":"ContainerStarted","Data":"3e9e71fdf38e83391ea1c00e9ae7c4f6b50abea630ea214be94b0a710d0f3cd5"}
Oct 30 22:13:37.374690 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:37.373854    5510 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-v4cxn" podStartSLOduration=5.373837418 podStartE2EDuration="5.373837418s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:13:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:13:37.373440841 +0000 UTC m=+1397.934250847" watchObservedRunningTime="2026-10-30 22:13:37.373837418 +0000 UTC m=+1397.934647425"
Oct 30 22:13:37.497503 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:37.497392    5510 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-f969x" podStartSLOduration=5.497377426 podStartE2EDuration="5.497377426s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:13:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:13:37.496419714 +0000 UTC m=+1398.057229746" watchObservedRunningTime="2026-10-30 22:13:37.497377426 +0000 UTC m=+1398.058187505"
Oct 30 22:13:37.797994 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:37.797948    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-v4cxn"
Oct 30 22:13:37.799463 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:37.799435    5510 patch_prober.go:29] interesting pod/e2e-worker-w2-bc714d-openshift-default-67558b95cf-v4cxn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused" start-of-body=
Oct 30 22:13:37.799629 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:37.799490    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-v4cxn" podUID="5542b62f-bc73-47d7-8615-180f43324911" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused"
Oct 30 22:13:37.966684 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:37.964013    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-f969x"
Oct 30 22:13:37.966684 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:37.966466    5510 patch_prober.go:29] interesting pod/e2e-worker-w2-bc714d-openshift-default-67558b95cf-f969x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused" start-of-body=
Oct 30 22:13:37.966684 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:37.966521    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-f969x" podUID="b0d37b4a-b986-438a-a811-9037505f0e6a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused"
Oct 30 22:13:38.428129 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:13:38.428096    5510 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6gjbf" containerName="extractor"
Oct 30 22:13:38.798316 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:38.798273    5510 patch_prober.go:29] interesting pod/e2e-worker-w2-bc714d-openshift-default-67558b95cf-v4cxn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused" start-of-body=
Oct 30 22:13:38.798512 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:38.798338    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-v4cxn" podUID="5542b62f-bc73-47d7-8615-180f43324911" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused"
Oct 30 22:13:38.967063 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:38.967028    5510 patch_prober.go:29] interesting pod/e2e-worker-w2-bc714d-openshift-default-67558b95cf-f969x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused" start-of-body=
Oct 30 22:13:38.967346 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:38.967313    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-f969x" podUID="b0d37b4a-b986-438a-a811-9037505f0e6a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused"
Oct 30 22:13:39.799269 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:39.799228    5510 patch_prober.go:29] interesting pod/e2e-worker-w2-bc714d-openshift-default-67558b95cf-v4cxn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused" start-of-body=
Oct 30 22:13:39.799686 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:39.799288    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-v4cxn" podUID="5542b62f-bc73-47d7-8615-180f43324911" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused"
Oct 30 22:13:39.964726 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:39.964219    5510 patch_prober.go:29] interesting pod/e2e-worker-w2-bc714d-openshift-default-67558b95cf-f969x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused" start-of-body=
Oct 30 22:13:39.964726 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:39.964298    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-f969x" podUID="b0d37b4a-b986-438a-a811-9037505f0e6a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused"
Oct 30 22:13:40.801005 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:40.800512    5510 patch_prober.go:29] interesting pod/e2e-worker-w2-bc714d-openshift-default-67558b95cf-v4cxn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused" start-of-body=
Oct 30 22:13:40.801005 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:40.800594    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-v4cxn" podUID="5542b62f-bc73-47d7-8615-180f43324911" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused"
Oct 30 22:13:40.967692 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:40.966197    5510 patch_prober.go:29] interesting pod/e2e-worker-w2-bc714d-openshift-default-67558b95cf-f969x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused" start-of-body=
Oct 30 22:13:40.967692 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:40.966276    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-f969x" podUID="b0d37b4a-b986-438a-a811-9037505f0e6a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused"
Oct 30 22:13:41.798414 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:41.798378    5510 patch_prober.go:29] interesting pod/e2e-worker-w2-bc714d-openshift-default-67558b95cf-v4cxn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused" start-of-body=
Oct 30 22:13:41.798614 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:41.798433    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-v4cxn" podUID="5542b62f-bc73-47d7-8615-180f43324911" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused"
Oct 30 22:13:41.977678 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:41.972312    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:13:41.977678 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:41.972809    5510 patch_prober.go:29] interesting pod/e2e-worker-w2-bc714d-openshift-default-67558b95cf-f969x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused" start-of-body=
Oct 30 22:13:41.977678 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:41.972874    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-f969x" podUID="b0d37b4a-b986-438a-a811-9037505f0e6a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused"
Oct 30 22:13:42.798489 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:42.798441    5510 patch_prober.go:29] interesting pod/e2e-worker-w2-bc714d-openshift-default-67558b95cf-v4cxn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused" start-of-body=
Oct 30 22:13:42.798907 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:42.798519    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-v4cxn" podUID="5542b62f-bc73-47d7-8615-180f43324911" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused"
Oct 30 22:13:42.963386 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:42.963343    5510 patch_prober.go:29] interesting pod/e2e-worker-w2-bc714d-openshift-default-67558b95cf-f969x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused" start-of-body=
Oct 30 22:13:42.963616 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:42.963414    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-f969x" podUID="b0d37b4a-b986-438a-a811-9037505f0e6a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused"
Oct 30 22:13:43.804562 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:43.804531    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-v4cxn"
Oct 30 22:13:43.805053 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:43.805031    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-v4cxn"
Oct 30 22:13:43.806046 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:43.806027    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-v4cxn"
Oct 30 22:13:44.025082 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:44.025042    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-f969x"
Oct 30 22:13:44.025431 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:44.025400    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-f969x"
Oct 30 22:13:44.026592 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:44.026572    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-f969x"
Oct 30 22:13:46.591731 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:46.591693    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:13:51.222684 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:51.217010    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:13:55.800890 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:13:55.800792    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:14:00.231819 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:00.231764    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-bc714d/distinct-w2-bc714d-kserve-66cfb968dd-jvgqj" podUID="9e9b11b8-1cc9-4fc8-8097-1afcffa8c422" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:14:00.240768 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:00.240722    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-7bb1c2/distinct-2-w3-7bb1c2-kserve-c549df844-zt595" podUID="1916e87c-a1a4-4967-8640-8bbcd9d12903" containerName="main" probeResult="failure" output="Get \"https://10.133.0.73:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:14:00.261824 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:00.261758    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-fb0b93/distinct-w6-fb0b93-kserve-58db6b6757-6mww8" podUID="8ee6579b-acea-4492-862b-9883f70d00c9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:14:00.400840 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:00.400741    5510 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w5-c9dbe6/distinct-w5-c9dbe6-kserve-7599fd86d9-bdhxd" podUID="e903db90-f2cd-4244-b00b-f99b21873f6e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.67:8000/health\": context deadline exceeded"
Oct 30 22:14:00.432871 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:00.432830    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-779f32/distinct-w0-779f32-kserve-6dcbc948c7-2vzdf" podUID="e17cedd4-7d48-42b3-b127-b6192c1b3795" containerName="main" probeResult="failure" output="Get \"https://10.133.0.65:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:14:00.574495 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:00.574452    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-779f32/distinct-2-w0-779f32-kserve-857ffdf778-gk2h9" podUID="ef500cdd-cc52-45b0-8dd7-1896897acc46" containerName="main" probeResult="failure" output="Get \"https://10.133.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:14:00.726818 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:00.726672    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-c9dbe6/distinct-w5-c9dbe6-kserve-7599fd86d9-bdhxd" podUID="e903db90-f2cd-4244-b00b-f99b21873f6e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:14:00.892873 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:00.892763    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-fb0b93/premium-w6-fb0b93-kserve-5cb9948c54-w7pnk" podUID="290ae7a2-de37-4af3-82ae-a8e855b83bb9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:14:01.426898 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:01.388592    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-bc714d/distinct-w2-bc714d-kserve-66cfb968dd-jvgqj" podUID="9e9b11b8-1cc9-4fc8-8097-1afcffa8c422" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": context deadline exceeded"
Oct 30 22:14:01.426898 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:01.388935    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-7bb1c2/distinct-2-w3-7bb1c2-kserve-c549df844-zt595" podUID="1916e87c-a1a4-4967-8640-8bbcd9d12903" containerName="main" probeResult="failure" output="Get \"https://10.133.0.73:8000/ready\": context deadline exceeded"
Oct 30 22:14:01.426898 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:01.389191    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-fb0b93/distinct-w6-fb0b93-kserve-58db6b6757-6mww8" podUID="8ee6579b-acea-4492-862b-9883f70d00c9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.59:8000/ready\": context deadline exceeded"
Oct 30 22:14:01.426898 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:01.389897    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-bc714d/distinct-w2-bc714d-kserve-66cfb968dd-jvgqj"
Oct 30 22:14:01.426898 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:01.389959    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-7bb1c2/distinct-2-w3-7bb1c2-kserve-c549df844-zt595"
Oct 30 22:14:01.426898 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:01.389998    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-fb0b93/distinct-w6-fb0b93-kserve-58db6b6757-6mww8"
Oct 30 22:14:01.486008 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:01.483882    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-779f32/distinct-w0-779f32-kserve-6dcbc948c7-2vzdf" podUID="e17cedd4-7d48-42b3-b127-b6192c1b3795" containerName="main" probeResult="failure" output="Get \"https://10.133.0.65:8000/ready\": context deadline exceeded"
Oct 30 22:14:01.486008 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:01.484252    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-779f32/distinct-w0-779f32-kserve-6dcbc948c7-2vzdf"
Oct 30 22:14:01.614516 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:01.614474    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-779f32/distinct-2-w0-779f32-kserve-857ffdf778-gk2h9" podUID="ef500cdd-cc52-45b0-8dd7-1896897acc46" containerName="main" probeResult="failure" output="Get \"https://10.133.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:14:01.615327 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:01.615303    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-779f32/distinct-2-w0-779f32-kserve-857ffdf778-gk2h9"
Oct 30 22:14:01.632148 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:01.628975    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-7bb1c2/distinct-2-w3-7bb1c2-kserve-c549df844-zt595"
Oct 30 22:14:01.856253 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:01.854797    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:14:01.933371 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:01.933340    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-fb0b93/distinct-w6-fb0b93-kserve-58db6b6757-6mww8"
Oct 30 22:14:01.933705 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:01.933687    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-779f32/distinct-w0-779f32-kserve-6dcbc948c7-2vzdf"
Oct 30 22:14:01.933938 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:01.933925    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-bc714d/distinct-w2-bc714d-kserve-66cfb968dd-jvgqj"
Oct 30 22:14:01.934129 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:01.934116    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-779f32/distinct-2-w0-779f32-kserve-857ffdf778-gk2h9"
Oct 30 22:14:02.769415 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:02.769380    5510 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-86dps"]
Oct 30 22:14:02.773903 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:02.773885    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-86dps"
Oct 30 22:14:02.839330 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:02.839296    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-86dps"]
Oct 30 22:14:02.904397 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:02.904361    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/10d681c2-b6b1-40a1-8c97-1988a0e91d05-istio-envoy\") pod \"e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-86dps\" (UID: \"10d681c2-b6b1-40a1-8c97-1988a0e91d05\") " pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-86dps"
Oct 30 22:14:02.904397 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:02.904404    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/10d681c2-b6b1-40a1-8c97-1988a0e91d05-workload-certs\") pod \"e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-86dps\" (UID: \"10d681c2-b6b1-40a1-8c97-1988a0e91d05\") " pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-86dps"
Oct 30 22:14:02.904611 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:02.904484    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/10d681c2-b6b1-40a1-8c97-1988a0e91d05-istiod-ca-cert\") pod \"e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-86dps\" (UID: \"10d681c2-b6b1-40a1-8c97-1988a0e91d05\") " pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-86dps"
Oct 30 22:14:02.904664 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:02.904628    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/10d681c2-b6b1-40a1-8c97-1988a0e91d05-credential-socket\") pod \"e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-86dps\" (UID: \"10d681c2-b6b1-40a1-8c97-1988a0e91d05\") " pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-86dps"
Oct 30 22:14:02.904715 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:02.904699    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/10d681c2-b6b1-40a1-8c97-1988a0e91d05-istio-token\") pod \"e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-86dps\" (UID: \"10d681c2-b6b1-40a1-8c97-1988a0e91d05\") " pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-86dps"
Oct 30 22:14:02.904759 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:02.904734    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zl46q\" (UniqueName: \"kubernetes.io/projected/10d681c2-b6b1-40a1-8c97-1988a0e91d05-kube-api-access-zl46q\") pod \"e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-86dps\" (UID: \"10d681c2-b6b1-40a1-8c97-1988a0e91d05\") " pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-86dps"
Oct 30 22:14:02.904852 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:02.904839    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/10d681c2-b6b1-40a1-8c97-1988a0e91d05-istio-podinfo\") pod \"e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-86dps\" (UID: \"10d681c2-b6b1-40a1-8c97-1988a0e91d05\") " pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-86dps"
Oct 30 22:14:02.904929 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:02.904914    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/10d681c2-b6b1-40a1-8c97-1988a0e91d05-istio-data\") pod \"e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-86dps\" (UID: \"10d681c2-b6b1-40a1-8c97-1988a0e91d05\") " pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-86dps"
Oct 30 22:14:02.904989 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:02.904976    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/10d681c2-b6b1-40a1-8c97-1988a0e91d05-workload-socket\") pod \"e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-86dps\" (UID: \"10d681c2-b6b1-40a1-8c97-1988a0e91d05\") " pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-86dps"
Oct 30 22:14:03.006425 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:03.006388    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/10d681c2-b6b1-40a1-8c97-1988a0e91d05-istio-podinfo\") pod \"e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-86dps\" (UID: \"10d681c2-b6b1-40a1-8c97-1988a0e91d05\") " pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-86dps"
Oct 30 22:14:03.006613 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:03.006484    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/10d681c2-b6b1-40a1-8c97-1988a0e91d05-istio-data\") pod \"e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-86dps\" (UID: \"10d681c2-b6b1-40a1-8c97-1988a0e91d05\") " pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-86dps"
Oct 30 22:14:03.006613 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:03.006560    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/10d681c2-b6b1-40a1-8c97-1988a0e91d05-workload-socket\") pod \"e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-86dps\" (UID: \"10d681c2-b6b1-40a1-8c97-1988a0e91d05\") " pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-86dps"
Oct 30 22:14:03.006613 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:03.006608    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/10d681c2-b6b1-40a1-8c97-1988a0e91d05-istio-envoy\") pod \"e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-86dps\" (UID: \"10d681c2-b6b1-40a1-8c97-1988a0e91d05\") " pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-86dps"
Oct 30 22:14:03.006799 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:03.006639    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/10d681c2-b6b1-40a1-8c97-1988a0e91d05-workload-certs\") pod \"e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-86dps\" (UID: \"10d681c2-b6b1-40a1-8c97-1988a0e91d05\") " pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-86dps"
Oct 30 22:14:03.006799 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:03.006709    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/10d681c2-b6b1-40a1-8c97-1988a0e91d05-istiod-ca-cert\") pod \"e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-86dps\" (UID: \"10d681c2-b6b1-40a1-8c97-1988a0e91d05\") " pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-86dps"
Oct 30 22:14:03.006896 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:03.006837    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/10d681c2-b6b1-40a1-8c97-1988a0e91d05-credential-socket\") pod \"e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-86dps\" (UID: \"10d681c2-b6b1-40a1-8c97-1988a0e91d05\") " pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-86dps"
Oct 30 22:14:03.006948 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:03.006894    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/10d681c2-b6b1-40a1-8c97-1988a0e91d05-istio-token\") pod \"e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-86dps\" (UID: \"10d681c2-b6b1-40a1-8c97-1988a0e91d05\") " pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-86dps"
Oct 30 22:14:03.006948 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:03.006937    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zl46q\" (UniqueName: \"kubernetes.io/projected/10d681c2-b6b1-40a1-8c97-1988a0e91d05-kube-api-access-zl46q\") pod \"e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-86dps\" (UID: \"10d681c2-b6b1-40a1-8c97-1988a0e91d05\") " pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-86dps"
Oct 30 22:14:03.007127 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:03.007048    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/10d681c2-b6b1-40a1-8c97-1988a0e91d05-istio-data\") pod \"e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-86dps\" (UID: \"10d681c2-b6b1-40a1-8c97-1988a0e91d05\") " pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-86dps"
Oct 30 22:14:03.007127 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:03.007065    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/10d681c2-b6b1-40a1-8c97-1988a0e91d05-workload-certs\") pod \"e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-86dps\" (UID: \"10d681c2-b6b1-40a1-8c97-1988a0e91d05\") " pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-86dps"
Oct 30 22:14:03.007563 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:03.007532    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/10d681c2-b6b1-40a1-8c97-1988a0e91d05-istiod-ca-cert\") pod \"e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-86dps\" (UID: \"10d681c2-b6b1-40a1-8c97-1988a0e91d05\") " pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-86dps"
Oct 30 22:14:03.007759 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:03.007740    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/10d681c2-b6b1-40a1-8c97-1988a0e91d05-workload-socket\") pod \"e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-86dps\" (UID: \"10d681c2-b6b1-40a1-8c97-1988a0e91d05\") " pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-86dps"
Oct 30 22:14:03.007826 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:03.007772    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/10d681c2-b6b1-40a1-8c97-1988a0e91d05-credential-socket\") pod \"e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-86dps\" (UID: \"10d681c2-b6b1-40a1-8c97-1988a0e91d05\") " pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-86dps"
Oct 30 22:14:03.009193 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:03.009173    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/10d681c2-b6b1-40a1-8c97-1988a0e91d05-istio-podinfo\") pod \"e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-86dps\" (UID: \"10d681c2-b6b1-40a1-8c97-1988a0e91d05\") " pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-86dps"
Oct 30 22:14:03.009541 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:03.009524    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/10d681c2-b6b1-40a1-8c97-1988a0e91d05-istio-envoy\") pod \"e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-86dps\" (UID: \"10d681c2-b6b1-40a1-8c97-1988a0e91d05\") " pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-86dps"
Oct 30 22:14:03.069388 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:03.069313    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/10d681c2-b6b1-40a1-8c97-1988a0e91d05-istio-token\") pod \"e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-86dps\" (UID: \"10d681c2-b6b1-40a1-8c97-1988a0e91d05\") " pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-86dps"
Oct 30 22:14:03.069521 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:03.069495    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zl46q\" (UniqueName: \"kubernetes.io/projected/10d681c2-b6b1-40a1-8c97-1988a0e91d05-kube-api-access-zl46q\") pod \"e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-86dps\" (UID: \"10d681c2-b6b1-40a1-8c97-1988a0e91d05\") " pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-86dps"
Oct 30 22:14:03.088521 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:03.088493    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-86dps"
Oct 30 22:14:03.516648 ip-10-0-136-210 kubenswrapper[5510]: W1030 22:14:03.516620    5510 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10d681c2_b6b1_40a1_8c97_1988a0e91d05.slice/crio-e71dd109b65c5a1427d9172043c5e0bd42199d0f329ab8d348b985f2154430dc WatchSource:0}: Error finding container e71dd109b65c5a1427d9172043c5e0bd42199d0f329ab8d348b985f2154430dc: Status 404 returned error can't find the container with id e71dd109b65c5a1427d9172043c5e0bd42199d0f329ab8d348b985f2154430dc
Oct 30 22:14:03.519337 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:03.519296    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:14:03.519425 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:03.519385    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:14:03.519462 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:03.519448    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:14:03.519785 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:03.519765    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-86dps"]
Oct 30 22:14:03.932917 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:03.932884    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-86dps" event={"ID":"10d681c2-b6b1-40a1-8c97-1988a0e91d05","Type":"ContainerStarted","Data":"f791c34bc62d549080d2963539c7ef6fffcf7a4cd452bd82810ba5d43aae9b08"}
Oct 30 22:14:03.933304 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:03.932936    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-86dps" event={"ID":"10d681c2-b6b1-40a1-8c97-1988a0e91d05","Type":"ContainerStarted","Data":"e71dd109b65c5a1427d9172043c5e0bd42199d0f329ab8d348b985f2154430dc"}
Oct 30 22:14:04.089580 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:04.089545    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-86dps"
Oct 30 22:14:04.091216 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:04.091188    5510 patch_prober.go:29] interesting pod/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-86dps container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.82:15021/healthz/ready\": dial tcp 10.133.0.82:15021: connect: connection refused" start-of-body=
Oct 30 22:14:04.091353 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:04.091238    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-86dps" podUID="10d681c2-b6b1-40a1-8c97-1988a0e91d05" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.82:15021/healthz/ready\": dial tcp 10.133.0.82:15021: connect: connection refused"
Oct 30 22:14:04.106510 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:04.106451    5510 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-86dps" podStartSLOduration=2.1064349509999998 podStartE2EDuration="2.106434951s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:14:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:14:04.104963215 +0000 UTC m=+1424.665773275" watchObservedRunningTime="2026-10-30 22:14:04.106434951 +0000 UTC m=+1424.667244950"
Oct 30 22:14:05.089204 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:05.089167    5510 patch_prober.go:29] interesting pod/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-86dps container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.82:15021/healthz/ready\": dial tcp 10.133.0.82:15021: connect: connection refused" start-of-body=
Oct 30 22:14:05.089735 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:05.089224    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-86dps" podUID="10d681c2-b6b1-40a1-8c97-1988a0e91d05" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.82:15021/healthz/ready\": dial tcp 10.133.0.82:15021: connect: connection refused"
Oct 30 22:14:06.088985 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:06.088949    5510 patch_prober.go:29] interesting pod/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-86dps container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.82:15021/healthz/ready\": dial tcp 10.133.0.82:15021: connect: connection refused" start-of-body=
Oct 30 22:14:06.089187 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:06.089007    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-86dps" podUID="10d681c2-b6b1-40a1-8c97-1988a0e91d05" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.82:15021/healthz/ready\": dial tcp 10.133.0.82:15021: connect: connection refused"
Oct 30 22:14:06.469299 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:06.469211    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:14:07.089353 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:07.089314    5510 patch_prober.go:29] interesting pod/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-86dps container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.82:15021/healthz/ready\": dial tcp 10.133.0.82:15021: connect: connection refused" start-of-body=
Oct 30 22:14:07.089544 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:07.089371    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-86dps" podUID="10d681c2-b6b1-40a1-8c97-1988a0e91d05" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.82:15021/healthz/ready\": dial tcp 10.133.0.82:15021: connect: connection refused"
Oct 30 22:14:08.089309 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:08.089274    5510 patch_prober.go:29] interesting pod/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-86dps container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.82:15021/healthz/ready\": dial tcp 10.133.0.82:15021: connect: connection refused" start-of-body=
Oct 30 22:14:08.089850 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:08.089337    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-86dps" podUID="10d681c2-b6b1-40a1-8c97-1988a0e91d05" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.82:15021/healthz/ready\": dial tcp 10.133.0.82:15021: connect: connection refused"
Oct 30 22:14:08.421273 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:14:08.421182    5510 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6gjbf" containerName="extractor"
Oct 30 22:14:09.089088 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:09.089053    5510 patch_prober.go:29] interesting pod/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-86dps container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.82:15021/healthz/ready\": dial tcp 10.133.0.82:15021: connect: connection refused" start-of-body=
Oct 30 22:14:09.089275 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:09.089112    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-86dps" podUID="10d681c2-b6b1-40a1-8c97-1988a0e91d05" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.82:15021/healthz/ready\": dial tcp 10.133.0.82:15021: connect: connection refused"
Oct 30 22:14:10.089006 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:10.088971    5510 patch_prober.go:29] interesting pod/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-86dps container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.82:15021/healthz/ready\": dial tcp 10.133.0.82:15021: connect: connection refused" start-of-body=
Oct 30 22:14:10.089534 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:10.089042    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-86dps" podUID="10d681c2-b6b1-40a1-8c97-1988a0e91d05" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.82:15021/healthz/ready\": dial tcp 10.133.0.82:15021: connect: connection refused"
Oct 30 22:14:11.146820 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:11.146781    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:14:11.165836 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:11.165506    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-86dps"
Oct 30 22:14:11.167190 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:11.167166    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-86dps"
Oct 30 22:14:11.177476 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:11.175914    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-86dps"
Oct 30 22:14:15.734679 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:15.732290    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:14:20.426678 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:20.425869    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:14:25.058520 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:25.058472    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:14:26.186170 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:26.186129    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:14:26.386601 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:26.386558    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:14:27.350668 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:27.350619    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-cmqdj"]
Oct 30 22:14:27.351058 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:27.350841    5510 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-cmqdj" podUID="f4952ed4-c979-43db-b64e-a5666ff61bed" containerName="istio-proxy" containerID="cri-o://d1b90d7e93a28b2233e3419789ca4f482604ccd85169e7a02120f220b5a4b3b9" gracePeriod=30
Oct 30 22:14:27.401633 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:27.401595    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-49rg2"]
Oct 30 22:14:27.401885 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:27.401858    5510 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-49rg2" podUID="f98cd426-fb0e-433e-97ff-3fc95c44239b" containerName="istio-proxy" containerID="cri-o://5416194ed875923b42615c7bd6625c8bfb768b5c3ebcb42640ad0352fb756fcf" gracePeriod=30
Oct 30 22:14:27.406458 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:27.406427    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-fwfzh"]
Oct 30 22:14:27.406649 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:27.406607    5510 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-fwfzh" podUID="edc8b181-2f27-423e-96c9-8057cf0d1851" containerName="istio-proxy" containerID="cri-o://da0b4a000b09a3bc94f8d8890c9f3fafc8080fde6c9e24a051634312cd58141c" gracePeriod=30
Oct 30 22:14:27.452818 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:27.452785    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-94hzz"]
Oct 30 22:14:27.453033 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:27.453009    5510 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-94hzz" podUID="fcded500-59d4-493e-bce3-fcf44d59b277" containerName="istio-proxy" containerID="cri-o://ee36ff048c63cdf014bad75038d4fbe701e8f26c644e42558ba341e38ad04c21" gracePeriod=30
Oct 30 22:14:29.704432 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:29.704327    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:14:31.381685 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:31.373726    5510 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-85544d4cd7-fwfzh container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 22:14:31.381685 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:31.373814    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-fwfzh" podUID="edc8b181-2f27-423e-96c9-8057cf0d1851" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 22:14:32.694955 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:32.636997    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-779f32/distinct-w0-779f32-kserve-6dcbc948c7-2vzdf" podUID="e17cedd4-7d48-42b3-b127-b6192c1b3795" containerName="main" probeResult="failure" output="Get \"https://10.133.0.65:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:14:32.769207 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:32.769164    5510 generic.go:356] "Generic (PLEG): container finished" podID="f4952ed4-c979-43db-b64e-a5666ff61bed" containerID="d1b90d7e93a28b2233e3419789ca4f482604ccd85169e7a02120f220b5a4b3b9" exitCode=0
Oct 30 22:14:32.781680 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:32.769479    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-cmqdj" event={"ID":"f4952ed4-c979-43db-b64e-a5666ff61bed","Type":"ContainerDied","Data":"d1b90d7e93a28b2233e3419789ca4f482604ccd85169e7a02120f220b5a4b3b9"}
Oct 30 22:14:32.857865 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:32.857798    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-779f32/distinct-2-w0-779f32-kserve-857ffdf778-gk2h9" podUID="ef500cdd-cc52-45b0-8dd7-1896897acc46" containerName="main" probeResult="failure" output="Get \"https://10.133.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:14:33.673861 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:33.673811    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-779f32/distinct-w0-779f32-kserve-6dcbc948c7-2vzdf" podUID="e17cedd4-7d48-42b3-b127-b6192c1b3795" containerName="main" probeResult="failure" output="Get \"https://10.133.0.65:8000/ready\": context deadline exceeded"
Oct 30 22:14:33.724815 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:33.676005    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-7bb1c2/distinct-2-w3-7bb1c2-kserve-c549df844-zt595" podUID="1916e87c-a1a4-4967-8640-8bbcd9d12903" containerName="main" probeResult="failure" output="Get \"https://10.133.0.73:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:14:33.725706 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:33.676036    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-bc714d/distinct-w2-bc714d-kserve-66cfb968dd-jvgqj" podUID="9e9b11b8-1cc9-4fc8-8097-1afcffa8c422" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:14:33.726140 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:33.676877    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-fb0b93/distinct-w6-fb0b93-kserve-58db6b6757-6mww8" podUID="8ee6579b-acea-4492-862b-9883f70d00c9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.59:8000/ready\": context deadline exceeded"
Oct 30 22:14:33.750890 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:33.727005    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-779f32/distinct-w0-779f32-kserve-6dcbc948c7-2vzdf"
Oct 30 22:14:33.750890 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:33.676903    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-c9dbe6/distinct-w5-c9dbe6-kserve-7599fd86d9-bdhxd" podUID="e903db90-f2cd-4244-b00b-f99b21873f6e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.67:8000/ready\": context deadline exceeded"
Oct 30 22:14:33.891856 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:33.891752    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-779f32/distinct-2-w0-779f32-kserve-857ffdf778-gk2h9" podUID="ef500cdd-cc52-45b0-8dd7-1896897acc46" containerName="main" probeResult="failure" output="Get \"https://10.133.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:14:33.892579 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:33.892553    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-779f32/distinct-2-w0-779f32-kserve-857ffdf778-gk2h9"
Oct 30 22:14:33.964683 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:33.921178    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-fb0b93/premium-w6-fb0b93-kserve-5cb9948c54-w7pnk" podUID="290ae7a2-de37-4af3-82ae-a8e855b83bb9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": context deadline exceeded"
Oct 30 22:14:34.353705 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:34.167884    5510 generic.go:356] "Generic (PLEG): container finished" podID="edc8b181-2f27-423e-96c9-8057cf0d1851" containerID="da0b4a000b09a3bc94f8d8890c9f3fafc8080fde6c9e24a051634312cd58141c" exitCode=0
Oct 30 22:14:34.379713 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:34.167930    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-fwfzh" event={"ID":"edc8b181-2f27-423e-96c9-8057cf0d1851","Type":"ContainerDied","Data":"da0b4a000b09a3bc94f8d8890c9f3fafc8080fde6c9e24a051634312cd58141c"}
Oct 30 22:14:34.379713 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:34.199839    5510 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-85544d4cd7-49rg2 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" start-of-body=
Oct 30 22:14:34.379713 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:34.354073    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-49rg2" podUID="f98cd426-fb0e-433e-97ff-3fc95c44239b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused"
Oct 30 22:14:34.379713 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:34.359954    5510 generic.go:356] "Generic (PLEG): container finished" podID="f98cd426-fb0e-433e-97ff-3fc95c44239b" containerID="5416194ed875923b42615c7bd6625c8bfb768b5c3ebcb42640ad0352fb756fcf" exitCode=0
Oct 30 22:14:34.379713 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:34.360157    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-49rg2" event={"ID":"f98cd426-fb0e-433e-97ff-3fc95c44239b","Type":"ContainerDied","Data":"5416194ed875923b42615c7bd6625c8bfb768b5c3ebcb42640ad0352fb756fcf"}
Oct 30 22:14:34.403472 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:34.394291    5510 generic.go:356] "Generic (PLEG): container finished" podID="fcded500-59d4-493e-bce3-fcf44d59b277" containerID="ee36ff048c63cdf014bad75038d4fbe701e8f26c644e42558ba341e38ad04c21" exitCode=0
Oct 30 22:14:34.403472 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:34.394409    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-94hzz" event={"ID":"fcded500-59d4-493e-bce3-fcf44d59b277","Type":"ContainerDied","Data":"ee36ff048c63cdf014bad75038d4fbe701e8f26c644e42558ba341e38ad04c21"}
Oct 30 22:14:34.744743 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:34.736769    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-fb0b93/distinct-w6-fb0b93-kserve-58db6b6757-6mww8" podUID="8ee6579b-acea-4492-862b-9883f70d00c9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.59:8000/ready\": context deadline exceeded"
Oct 30 22:14:34.744743 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:34.737085    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-7bb1c2/distinct-2-w3-7bb1c2-kserve-c549df844-zt595" podUID="1916e87c-a1a4-4967-8640-8bbcd9d12903" containerName="main" probeResult="failure" output="Get \"https://10.133.0.73:8000/ready\": context deadline exceeded"
Oct 30 22:14:34.744743 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:34.737290    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-bc714d/distinct-w2-bc714d-kserve-66cfb968dd-jvgqj" podUID="9e9b11b8-1cc9-4fc8-8097-1afcffa8c422" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": context deadline exceeded"
Oct 30 22:14:34.744743 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:34.737834    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-fb0b93/distinct-w6-fb0b93-kserve-58db6b6757-6mww8"
Oct 30 22:14:34.744743 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:34.737897    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-7bb1c2/distinct-2-w3-7bb1c2-kserve-c549df844-zt595"
Oct 30 22:14:34.744743 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:34.737935    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-bc714d/distinct-w2-bc714d-kserve-66cfb968dd-jvgqj"
Oct 30 22:14:34.767931 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:34.767063    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-c9dbe6/distinct-w5-c9dbe6-kserve-7599fd86d9-bdhxd" podUID="e903db90-f2cd-4244-b00b-f99b21873f6e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:14:34.767931 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:34.767298    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-779f32/distinct-w0-779f32-kserve-6dcbc948c7-2vzdf" podUID="e17cedd4-7d48-42b3-b127-b6192c1b3795" containerName="main" probeResult="failure" output="Get \"https://10.133.0.65:8000/ready\": context deadline exceeded"
Oct 30 22:14:34.767931 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:34.767499    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-c9dbe6/distinct-w5-c9dbe6-kserve-7599fd86d9-bdhxd"
Oct 30 22:14:34.798451 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:34.798408    5510 patch_prober.go:29] interesting pod/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-48cnj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.76:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 22:14:34.798758 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:34.798722    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-48cnj" podUID="6aa08ff6-6f78-4a6e-83bd-07cf5d43b7b0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.76:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:14:34.800821 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:34.800790    5510 patch_prober.go:29] interesting pod/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-l5ndh container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.79:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 22:14:34.812960 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:34.812747    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-l5ndh" podUID="2344709a-1a48-417c-ae98-2b462bf30516" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.79:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:14:34.898970 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:34.895908    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-779f32/distinct-2-w0-779f32-kserve-857ffdf778-gk2h9" podUID="ef500cdd-cc52-45b0-8dd7-1896897acc46" containerName="main" probeResult="failure" output="Get \"https://10.133.0.74:8000/ready\": context deadline exceeded"
Oct 30 22:14:35.018385 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:35.018341    5510 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-7bb1c2/distinct-2-w3-7bb1c2-kserve-c549df844-zt595" podUID="1916e87c-a1a4-4967-8640-8bbcd9d12903" containerName="main" probeResult="failure" output="Get \"https://10.133.0.73:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:14:35.108712 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:35.108647    5510 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-85544d4cd7-cmqdj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body=
Oct 30 22:14:35.127683 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:35.112783    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-cmqdj" podUID="f4952ed4-c979-43db-b64e-a5666ff61bed" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused"
Oct 30 22:14:35.454745 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:35.433756    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-fb0b93/premium-w6-fb0b93-kserve-5cb9948c54-w7pnk" podUID="290ae7a2-de37-4af3-82ae-a8e855b83bb9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:14:35.454745 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:35.434173    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-fb0b93/premium-w6-fb0b93-kserve-5cb9948c54-w7pnk"
Oct 30 22:14:35.730828 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:35.730716    5510 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-85544d4cd7-94hzz container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body=
Oct 30 22:14:35.731138 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:35.731088    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-94hzz" podUID="fcded500-59d4-493e-bce3-fcf44d59b277" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused"
Oct 30 22:14:35.809110 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:35.809064    5510 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-bc714d/distinct-w2-bc714d-kserve-66cfb968dd-jvgqj" podUID="9e9b11b8-1cc9-4fc8-8097-1afcffa8c422" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/health\": context deadline exceeded"
Oct 30 22:14:35.809761 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:35.809729    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-c9dbe6/distinct-w5-c9dbe6-kserve-7599fd86d9-bdhxd" podUID="e903db90-f2cd-4244-b00b-f99b21873f6e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.67:8000/ready\": context deadline exceeded"
Oct 30 22:14:35.810241 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:35.810215    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-779f32/distinct-w0-779f32-kserve-6dcbc948c7-2vzdf" podUID="e17cedd4-7d48-42b3-b127-b6192c1b3795" containerName="main" probeResult="failure" output="Get \"https://10.133.0.65:8000/ready\": context deadline exceeded"
Oct 30 22:14:35.936870 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:35.917037    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-779f32/distinct-2-w0-779f32-kserve-857ffdf778-gk2h9" podUID="ef500cdd-cc52-45b0-8dd7-1896897acc46" containerName="main" probeResult="failure" output="Get \"https://10.133.0.74:8000/ready\": context deadline exceeded"
Oct 30 22:14:36.442283 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:36.442238    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-7bb1c2/distinct-2-w3-7bb1c2-kserve-c549df844-zt595" podUID="1916e87c-a1a4-4967-8640-8bbcd9d12903" containerName="main" probeResult="failure" output="Get \"https://10.133.0.73:8000/ready\": context deadline exceeded"
Oct 30 22:14:36.484200 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:36.484155    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-42mzp" podUID="9f899391-6025-4f44-9553-a82cd5e25e69" containerName="main" probeResult="failure" output="Get \"https://10.133.0.33:8000/ready\": context deadline exceeded"
Oct 30 22:14:36.511278 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:36.445360    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-fb0b93/distinct-w6-fb0b93-kserve-58db6b6757-6mww8" podUID="8ee6579b-acea-4492-862b-9883f70d00c9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.59:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:14:36.539107 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:36.442524    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-bc714d/distinct-w2-bc714d-kserve-66cfb968dd-jvgqj" podUID="9e9b11b8-1cc9-4fc8-8097-1afcffa8c422" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": context deadline exceeded"
Oct 30 22:14:36.564546 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:36.489525    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-fb0b93/premium-w6-fb0b93-kserve-5cb9948c54-w7pnk" podUID="290ae7a2-de37-4af3-82ae-a8e855b83bb9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": context deadline exceeded"
Oct 30 22:14:36.913535 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:36.884142    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-c9dbe6/distinct-w5-c9dbe6-kserve-7599fd86d9-bdhxd" podUID="e903db90-f2cd-4244-b00b-f99b21873f6e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:14:36.913535 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:36.885800    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-779f32/distinct-w0-779f32-kserve-6dcbc948c7-2vzdf" podUID="e17cedd4-7d48-42b3-b127-b6192c1b3795" containerName="main" probeResult="failure" output="Get \"https://10.133.0.65:8000/ready\": context deadline exceeded"
Oct 30 22:14:36.922725 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:36.919182    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-779f32/distinct-2-w0-779f32-kserve-857ffdf778-gk2h9" podUID="ef500cdd-cc52-45b0-8dd7-1896897acc46" containerName="main" probeResult="failure" output="Get \"https://10.133.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:14:37.393376 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:37.393233    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-cmqdj"
Oct 30 22:14:37.527044 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:37.489673    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-7bb1c2/distinct-2-w3-7bb1c2-kserve-c549df844-zt595" podUID="1916e87c-a1a4-4967-8640-8bbcd9d12903" containerName="main" probeResult="failure" output="Get \"https://10.133.0.73:8000/ready\": context deadline exceeded"
Oct 30 22:14:37.529470 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:37.527313    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-fb0b93/distinct-w6-fb0b93-kserve-58db6b6757-6mww8" podUID="8ee6579b-acea-4492-862b-9883f70d00c9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.59:8000/ready\": context deadline exceeded"
Oct 30 22:14:37.553697 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:37.532764    5510 patch_prober.go:29] interesting pod/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-k5dpm container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.78:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 22:14:37.553697 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:37.532825    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-k5dpm" podUID="8724086f-a5f7-4c63-87da-cb9228b03a0c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.78:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:14:37.553697 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:37.542512    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-bc714d/distinct-w2-bc714d-kserve-66cfb968dd-jvgqj" podUID="9e9b11b8-1cc9-4fc8-8097-1afcffa8c422" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:14:37.636717 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:37.628210    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f4952ed4-c979-43db-b64e-a5666ff61bed-istio-token\" (UniqueName: \"kubernetes.io/projected/f4952ed4-c979-43db-b64e-a5666ff61bed-istio-token\") pod \"f4952ed4-c979-43db-b64e-a5666ff61bed\" (UID: \"f4952ed4-c979-43db-b64e-a5666ff61bed\") "
Oct 30 22:14:37.636717 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:37.628306    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f4952ed4-c979-43db-b64e-a5666ff61bed-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f4952ed4-c979-43db-b64e-a5666ff61bed-istiod-ca-cert\") pod \"f4952ed4-c979-43db-b64e-a5666ff61bed\" (UID: \"f4952ed4-c979-43db-b64e-a5666ff61bed\") "
Oct 30 22:14:37.636717 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:37.628377    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f4952ed4-c979-43db-b64e-a5666ff61bed-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f4952ed4-c979-43db-b64e-a5666ff61bed-workload-certs\") pod \"f4952ed4-c979-43db-b64e-a5666ff61bed\" (UID: \"f4952ed4-c979-43db-b64e-a5666ff61bed\") "
Oct 30 22:14:37.691739 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:37.689732    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-fb0b93/premium-w6-fb0b93-kserve-5cb9948c54-w7pnk" podUID="290ae7a2-de37-4af3-82ae-a8e855b83bb9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:14:37.691739 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:37.628434    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f4952ed4-c979-43db-b64e-a5666ff61bed-kube-api-access-rr5lz\" (UniqueName: \"kubernetes.io/projected/f4952ed4-c979-43db-b64e-a5666ff61bed-kube-api-access-rr5lz\") pod \"f4952ed4-c979-43db-b64e-a5666ff61bed\" (UID: \"f4952ed4-c979-43db-b64e-a5666ff61bed\") "
Oct 30 22:14:37.691739 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:37.690723    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f4952ed4-c979-43db-b64e-a5666ff61bed-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f4952ed4-c979-43db-b64e-a5666ff61bed-istio-data\") pod \"f4952ed4-c979-43db-b64e-a5666ff61bed\" (UID: \"f4952ed4-c979-43db-b64e-a5666ff61bed\") "
Oct 30 22:14:37.691739 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:37.690888    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f4952ed4-c979-43db-b64e-a5666ff61bed-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4952ed4-c979-43db-b64e-a5666ff61bed-credential-socket\") pod \"f4952ed4-c979-43db-b64e-a5666ff61bed\" (UID: \"f4952ed4-c979-43db-b64e-a5666ff61bed\") "
Oct 30 22:14:37.691739 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:37.690939    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/f4952ed4-c979-43db-b64e-a5666ff61bed-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f4952ed4-c979-43db-b64e-a5666ff61bed-istio-podinfo\") pod \"f4952ed4-c979-43db-b64e-a5666ff61bed\" (UID: \"f4952ed4-c979-43db-b64e-a5666ff61bed\") "
Oct 30 22:14:37.691739 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:37.690992    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f4952ed4-c979-43db-b64e-a5666ff61bed-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f4952ed4-c979-43db-b64e-a5666ff61bed-istio-envoy\") pod \"f4952ed4-c979-43db-b64e-a5666ff61bed\" (UID: \"f4952ed4-c979-43db-b64e-a5666ff61bed\") "
Oct 30 22:14:37.691739 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:37.691039    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f4952ed4-c979-43db-b64e-a5666ff61bed-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4952ed4-c979-43db-b64e-a5666ff61bed-workload-socket\") pod \"f4952ed4-c979-43db-b64e-a5666ff61bed\" (UID: \"f4952ed4-c979-43db-b64e-a5666ff61bed\") "
Oct 30 22:14:37.781681 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:37.754725    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-94hzz"
Oct 30 22:14:37.876279 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:37.876242    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f4952ed4-c979-43db-b64e-a5666ff61bed-credential-socket" pod "f4952ed4-c979-43db-b64e-a5666ff61bed" (UID: "f4952ed4-c979-43db-b64e-a5666ff61bed"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:14:38.009769 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:37.990931    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-c9dbe6/distinct-w5-c9dbe6-kserve-7599fd86d9-bdhxd" podUID="e903db90-f2cd-4244-b00b-f99b21873f6e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:14:38.066422 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:38.038111    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-779f32/distinct-w0-779f32-kserve-6dcbc948c7-2vzdf" podUID="e17cedd4-7d48-42b3-b127-b6192c1b3795" containerName="main" probeResult="failure" output="Get \"https://10.133.0.65:8000/ready\": context deadline exceeded"
Oct 30 22:14:38.066422 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:38.038528    5510 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w6-fb0b93/distinct-w6-fb0b93-kserve-58db6b6757-6mww8" podUID="8ee6579b-acea-4492-862b-9883f70d00c9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.59:8000/health\": context deadline exceeded"
Oct 30 22:14:38.066422 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:38.038708    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f4952ed4-c979-43db-b64e-a5666ff61bed-istio-data" pod "f4952ed4-c979-43db-b64e-a5666ff61bed" (UID: "f4952ed4-c979-43db-b64e-a5666ff61bed"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:14:38.115686 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:38.101438    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f4952ed4-c979-43db-b64e-a5666ff61bed-istio-envoy" pod "f4952ed4-c979-43db-b64e-a5666ff61bed" (UID: "f4952ed4-c979-43db-b64e-a5666ff61bed"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:14:38.133980 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:38.133933    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f4952ed4-c979-43db-b64e-a5666ff61bed-istiod-ca-cert" pod "f4952ed4-c979-43db-b64e-a5666ff61bed" (UID: "f4952ed4-c979-43db-b64e-a5666ff61bed"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:14:38.487754 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:38.384893    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f4952ed4-c979-43db-b64e-a5666ff61bed-workload-certs" pod "f4952ed4-c979-43db-b64e-a5666ff61bed" (UID: "f4952ed4-c979-43db-b64e-a5666ff61bed"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:14:38.487754 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:38.262925    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-779f32/distinct-2-w0-779f32-kserve-857ffdf778-gk2h9" podUID="ef500cdd-cc52-45b0-8dd7-1896897acc46" containerName="main" probeResult="failure" output="Get \"https://10.133.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:14:38.487754 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:38.262856    5510 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w6-fb0b93/premium-w6-fb0b93-kserve-5cb9948c54-w7pnk" podUID="290ae7a2-de37-4af3-82ae-a8e855b83bb9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/health\": context deadline exceeded"
Oct 30 22:14:38.487754 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:38.446891    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fcded500-59d4-493e-bce3-fcf44d59b277-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fcded500-59d4-493e-bce3-fcf44d59b277-istiod-ca-cert\") pod \"fcded500-59d4-493e-bce3-fcf44d59b277\" (UID: \"fcded500-59d4-493e-bce3-fcf44d59b277\") "
Oct 30 22:14:38.487754 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:38.447061    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fcded500-59d4-493e-bce3-fcf44d59b277-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fcded500-59d4-493e-bce3-fcf44d59b277-istio-data\") pod \"fcded500-59d4-493e-bce3-fcf44d59b277\" (UID: \"fcded500-59d4-493e-bce3-fcf44d59b277\") "
Oct 30 22:14:38.487754 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:38.447209    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fcded500-59d4-493e-bce3-fcf44d59b277-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fcded500-59d4-493e-bce3-fcf44d59b277-istio-envoy\") pod \"fcded500-59d4-493e-bce3-fcf44d59b277\" (UID: \"fcded500-59d4-493e-bce3-fcf44d59b277\") "
Oct 30 22:14:38.585850 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:38.585801    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f4952ed4-c979-43db-b64e-a5666ff61bed-workload-socket" pod "f4952ed4-c979-43db-b64e-a5666ff61bed" (UID: "f4952ed4-c979-43db-b64e-a5666ff61bed"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:14:38.597863 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:38.597793    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f4952ed4-c979-43db-b64e-a5666ff61bed-istio-token" pod "f4952ed4-c979-43db-b64e-a5666ff61bed" (UID: "f4952ed4-c979-43db-b64e-a5666ff61bed"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:14:38.656968 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:38.447297    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fcded500-59d4-493e-bce3-fcf44d59b277-kube-api-access-kfskv\" (UniqueName: \"kubernetes.io/projected/fcded500-59d4-493e-bce3-fcf44d59b277-kube-api-access-kfskv\") pod \"fcded500-59d4-493e-bce3-fcf44d59b277\" (UID: \"fcded500-59d4-493e-bce3-fcf44d59b277\") "
Oct 30 22:14:38.973165 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:38.942557    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/fcded500-59d4-493e-bce3-fcf44d59b277-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fcded500-59d4-493e-bce3-fcf44d59b277-istio-podinfo\") pod \"fcded500-59d4-493e-bce3-fcf44d59b277\" (UID: \"fcded500-59d4-493e-bce3-fcf44d59b277\") "
Oct 30 22:14:38.973165 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:38.942703    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fcded500-59d4-493e-bce3-fcf44d59b277-istio-token\" (UniqueName: \"kubernetes.io/projected/fcded500-59d4-493e-bce3-fcf44d59b277-istio-token\") pod \"fcded500-59d4-493e-bce3-fcf44d59b277\" (UID: \"fcded500-59d4-493e-bce3-fcf44d59b277\") "
Oct 30 22:14:38.973165 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:38.942784    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fcded500-59d4-493e-bce3-fcf44d59b277-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fcded500-59d4-493e-bce3-fcf44d59b277-credential-socket\") pod \"fcded500-59d4-493e-bce3-fcf44d59b277\" (UID: \"fcded500-59d4-493e-bce3-fcf44d59b277\") "
Oct 30 22:14:38.973165 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:38.942884    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fcded500-59d4-493e-bce3-fcf44d59b277-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fcded500-59d4-493e-bce3-fcf44d59b277-workload-socket\") pod \"fcded500-59d4-493e-bce3-fcf44d59b277\" (UID: \"fcded500-59d4-493e-bce3-fcf44d59b277\") "
Oct 30 22:14:38.973165 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:38.943011    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fcded500-59d4-493e-bce3-fcf44d59b277-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fcded500-59d4-493e-bce3-fcf44d59b277-workload-certs\") pod \"fcded500-59d4-493e-bce3-fcf44d59b277\" (UID: \"fcded500-59d4-493e-bce3-fcf44d59b277\") "
Oct 30 22:14:38.973165 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:38.682919    5510 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-779f32/distinct-w0-779f32-kserve-6dcbc948c7-2vzdf" podUID="e17cedd4-7d48-42b3-b127-b6192c1b3795" containerName="main" probeResult="failure" output="Get \"https://10.133.0.65:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:14:39.095704 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:38.707805    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-bc714d/distinct-w2-bc714d-kserve-66cfb968dd-jvgqj" podUID="9e9b11b8-1cc9-4fc8-8097-1afcffa8c422" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:14:39.096164 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:39.095770    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fcded500-59d4-493e-bce3-fcf44d59b277-istiod-ca-cert" pod "fcded500-59d4-493e-bce3-fcf44d59b277" (UID: "fcded500-59d4-493e-bce3-fcf44d59b277"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:14:39.097095 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:39.096310    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fcded500-59d4-493e-bce3-fcf44d59b277-istio-data" pod "fcded500-59d4-493e-bce3-fcf44d59b277" (UID: "fcded500-59d4-493e-bce3-fcf44d59b277"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:14:39.114355 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:39.114172    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:14:39.153033 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:39.152992    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-c9dbe6/distinct-w5-c9dbe6-kserve-7599fd86d9-bdhxd" podUID="e903db90-f2cd-4244-b00b-f99b21873f6e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.67:8000/ready\": context deadline exceeded"
Oct 30 22:14:39.166093 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:39.154757    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-779f32/distinct-w0-779f32-kserve-6dcbc948c7-2vzdf" podUID="e17cedd4-7d48-42b3-b127-b6192c1b3795" containerName="main" probeResult="failure" output="Get \"https://10.133.0.65:8000/ready\": context deadline exceeded"
Oct 30 22:14:39.189913 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:39.189866    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f4952ed4-c979-43db-b64e-a5666ff61bed-istio-podinfo" pod "f4952ed4-c979-43db-b64e-a5666ff61bed" (UID: "f4952ed4-c979-43db-b64e-a5666ff61bed"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:14:39.227190 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:39.207751    5510 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-779f32/distinct-2-w0-779f32-kserve-857ffdf778-gk2h9" podUID="ef500cdd-cc52-45b0-8dd7-1896897acc46" containerName="main" probeResult="failure" output="Get \"https://10.133.0.74:8000/health\": context deadline exceeded"
Oct 30 22:14:39.252073 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:39.233027    5510 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f4952ed4-c979-43db-b64e-a5666ff61bed-workload-certs\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:14:39.252073 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:39.233065    5510 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f4952ed4-c979-43db-b64e-a5666ff61bed-istio-data\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:14:39.252073 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:39.233100    5510 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fcded500-59d4-493e-bce3-fcf44d59b277-istiod-ca-cert\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:14:39.252073 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:39.233116    5510 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4952ed4-c979-43db-b64e-a5666ff61bed-credential-socket\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:14:39.252073 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:39.233135    5510 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f4952ed4-c979-43db-b64e-a5666ff61bed-istio-podinfo\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:14:39.252073 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:39.233151    5510 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f4952ed4-c979-43db-b64e-a5666ff61bed-istio-envoy\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:14:39.252073 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:39.233170    5510 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4952ed4-c979-43db-b64e-a5666ff61bed-workload-socket\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:14:39.252073 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:39.233186    5510 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fcded500-59d4-493e-bce3-fcf44d59b277-istio-data\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:14:39.252073 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:39.233205    5510 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f4952ed4-c979-43db-b64e-a5666ff61bed-istio-token\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:14:39.252073 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:39.233219    5510 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f4952ed4-c979-43db-b64e-a5666ff61bed-istiod-ca-cert\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:14:39.252073 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:14:39.239335    5510 kubelet.go:2711] "Housekeeping took longer than expected" err="housekeeping took too long" expected="1s" actual="1.693s"
Oct 30 22:14:39.252073 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:39.240263    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-fb0b93/premium-w6-fb0b93-kserve-5cb9948c54-w7pnk"
Oct 30 22:14:39.252073 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:39.240364    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-779f32/distinct-2-w0-779f32-kserve-857ffdf778-gk2h9"
Oct 30 22:14:39.252073 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:38.708938    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-fb0b93/distinct-w6-fb0b93-kserve-58db6b6757-6mww8" podUID="8ee6579b-acea-4492-862b-9883f70d00c9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:14:39.252073 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:39.249330    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fcded500-59d4-493e-bce3-fcf44d59b277-credential-socket" pod "fcded500-59d4-493e-bce3-fcf44d59b277" (UID: "fcded500-59d4-493e-bce3-fcf44d59b277"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:14:39.252073 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:38.745785    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-7bb1c2/distinct-2-w3-7bb1c2-kserve-c549df844-zt595" podUID="1916e87c-a1a4-4967-8640-8bbcd9d12903" containerName="main" probeResult="failure" output="Get \"https://10.133.0.73:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:14:39.252073 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:39.249631    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fcded500-59d4-493e-bce3-fcf44d59b277-workload-socket" pod "fcded500-59d4-493e-bce3-fcf44d59b277" (UID: "fcded500-59d4-493e-bce3-fcf44d59b277"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:14:39.252073 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:39.249913    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fcded500-59d4-493e-bce3-fcf44d59b277-workload-certs" pod "fcded500-59d4-493e-bce3-fcf44d59b277" (UID: "fcded500-59d4-493e-bce3-fcf44d59b277"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:14:39.268401 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:39.268162    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/fcded500-59d4-493e-bce3-fcf44d59b277-istio-podinfo" pod "fcded500-59d4-493e-bce3-fcf44d59b277" (UID: "fcded500-59d4-493e-bce3-fcf44d59b277"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:14:39.269022 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:39.268987    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fcded500-59d4-493e-bce3-fcf44d59b277-kube-api-access-kfskv" pod "fcded500-59d4-493e-bce3-fcf44d59b277" (UID: "fcded500-59d4-493e-bce3-fcf44d59b277"). InnerVolumeSpecName "kube-api-access-kfskv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:14:39.273263 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:39.272930    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f4952ed4-c979-43db-b64e-a5666ff61bed-kube-api-access-rr5lz" pod "f4952ed4-c979-43db-b64e-a5666ff61bed" (UID: "f4952ed4-c979-43db-b64e-a5666ff61bed"). InnerVolumeSpecName "kube-api-access-rr5lz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:14:39.274780 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:39.274287    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-49rg2"
Oct 30 22:14:39.286718 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:39.284242    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-fb0b93/distinct-w6-fb0b93-kserve-58db6b6757-6mww8"
Oct 30 22:14:39.286718 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:39.286235    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fcded500-59d4-493e-bce3-fcf44d59b277-istio-envoy" pod "fcded500-59d4-493e-bce3-fcf44d59b277" (UID: "fcded500-59d4-493e-bce3-fcf44d59b277"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:14:39.286718 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:39.286358    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fcded500-59d4-493e-bce3-fcf44d59b277-istio-token" pod "fcded500-59d4-493e-bce3-fcf44d59b277" (UID: "fcded500-59d4-493e-bce3-fcf44d59b277"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:14:39.286718 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:39.286597    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-cmqdj" event={"ID":"f4952ed4-c979-43db-b64e-a5666ff61bed","Type":"ContainerDied","Data":"bb00e9ced4ca50e14707c8dc534d10165f1a8872dcf84fb3c7d175f6d3b54199"}
Oct 30 22:14:39.286718 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:39.286649    5510 scope.go:122] "RemoveContainer" containerID="d1b90d7e93a28b2233e3419789ca4f482604ccd85169e7a02120f220b5a4b3b9"
Oct 30 22:14:39.287039 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:39.286910    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-bc714d/distinct-w2-bc714d-kserve-66cfb968dd-jvgqj"
Oct 30 22:14:39.288199 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:39.288156    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-cmqdj"
Oct 30 22:14:39.292639 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:39.292620    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-fwfzh"
Oct 30 22:14:39.298514 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:39.294433    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-94hzz"
Oct 30 22:14:39.298514 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:39.296683    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-94hzz" event={"ID":"fcded500-59d4-493e-bce3-fcf44d59b277","Type":"ContainerDied","Data":"82f086a26602e45df84749ef406f8e07078c6874220ead0bf78908f4a66c6f6f"}
Oct 30 22:14:39.298514 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:39.296768    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-c9dbe6/distinct-w5-c9dbe6-kserve-7599fd86d9-bdhxd"
Oct 30 22:14:39.298514 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:39.296806    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-779f32/distinct-w0-779f32-kserve-6dcbc948c7-2vzdf"
Oct 30 22:14:39.301152 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:39.300598    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-7bb1c2/distinct-2-w3-7bb1c2-kserve-c549df844-zt595"
Oct 30 22:14:39.306199 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:39.306061    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-fwfzh" event={"ID":"edc8b181-2f27-423e-96c9-8057cf0d1851","Type":"ContainerDied","Data":"9a4431fc0bee1bbbb52099c5fcb76a369b0cacb9b0ece04006be5ee0e6242d1b"}
Oct 30 22:14:39.306313 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:39.306287    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-fwfzh"
Oct 30 22:14:39.308790 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:39.308755    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-49rg2" event={"ID":"f98cd426-fb0e-433e-97ff-3fc95c44239b","Type":"ContainerDied","Data":"47ffca3a29d53f35b833eb9755f14d620a884722760e3cab1787abc0a191c603"}
Oct 30 22:14:39.309105 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:39.309087    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-49rg2"
Oct 30 22:14:39.323202 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:39.323179    5510 scope.go:122] "RemoveContainer" containerID="ee36ff048c63cdf014bad75038d4fbe701e8f26c644e42558ba341e38ad04c21"
Oct 30 22:14:39.323839 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:14:39.323805    5510 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6gjbf" containerName="extractor"
Oct 30 22:14:39.334813 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:39.334784    5510 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fcded500-59d4-493e-bce3-fcf44d59b277-workload-socket\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:14:39.334925 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:39.334818    5510 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fcded500-59d4-493e-bce3-fcf44d59b277-workload-certs\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:14:39.334925 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:39.334837    5510 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rr5lz\" (UniqueName: \"kubernetes.io/projected/f4952ed4-c979-43db-b64e-a5666ff61bed-kube-api-access-rr5lz\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:14:39.334925 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:39.334854    5510 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fcded500-59d4-493e-bce3-fcf44d59b277-istio-envoy\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:14:39.334925 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:39.334868    5510 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kfskv\" (UniqueName: \"kubernetes.io/projected/fcded500-59d4-493e-bce3-fcf44d59b277-kube-api-access-kfskv\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:14:39.334925 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:39.334889    5510 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fcded500-59d4-493e-bce3-fcf44d59b277-istio-podinfo\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:14:39.334925 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:39.334901    5510 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fcded500-59d4-493e-bce3-fcf44d59b277-istio-token\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:14:39.334925 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:39.334914    5510 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fcded500-59d4-493e-bce3-fcf44d59b277-credential-socket\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:14:39.337282 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:14:39.337241    5510 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4952ed4_c979_43db_b64e_a5666ff61bed.slice\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfcded500_59d4_493e_bce3_fcf44d59b277.slice/crio-82f086a26602e45df84749ef406f8e07078c6874220ead0bf78908f4a66c6f6f\": RecentStats: unable to find data in memory cache]"
Oct 30 22:14:39.340811 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:39.340786    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:14:39.341504 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:39.341482    5510 scope.go:122] "RemoveContainer" containerID="da0b4a000b09a3bc94f8d8890c9f3fafc8080fde6c9e24a051634312cd58141c"
Oct 30 22:14:39.351578 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:39.351557    5510 scope.go:122] "RemoveContainer" containerID="5416194ed875923b42615c7bd6625c8bfb768b5c3ebcb42640ad0352fb756fcf"
Oct 30 22:14:39.444050 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:39.441261    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f98cd426-fb0e-433e-97ff-3fc95c44239b-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f98cd426-fb0e-433e-97ff-3fc95c44239b-istiod-ca-cert\") pod \"f98cd426-fb0e-433e-97ff-3fc95c44239b\" (UID: \"f98cd426-fb0e-433e-97ff-3fc95c44239b\") "
Oct 30 22:14:39.444050 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:39.441308    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f98cd426-fb0e-433e-97ff-3fc95c44239b-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f98cd426-fb0e-433e-97ff-3fc95c44239b-workload-socket\") pod \"f98cd426-fb0e-433e-97ff-3fc95c44239b\" (UID: \"f98cd426-fb0e-433e-97ff-3fc95c44239b\") "
Oct 30 22:14:39.444050 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:39.441356    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/edc8b181-2f27-423e-96c9-8057cf0d1851-kube-api-access-6jw8w\" (UniqueName: \"kubernetes.io/projected/edc8b181-2f27-423e-96c9-8057cf0d1851-kube-api-access-6jw8w\") pod \"edc8b181-2f27-423e-96c9-8057cf0d1851\" (UID: \"edc8b181-2f27-423e-96c9-8057cf0d1851\") "
Oct 30 22:14:39.444050 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:39.441387    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/edc8b181-2f27-423e-96c9-8057cf0d1851-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/edc8b181-2f27-423e-96c9-8057cf0d1851-istio-podinfo\") pod \"edc8b181-2f27-423e-96c9-8057cf0d1851\" (UID: \"edc8b181-2f27-423e-96c9-8057cf0d1851\") "
Oct 30 22:14:39.444050 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:39.441447    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/edc8b181-2f27-423e-96c9-8057cf0d1851-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/edc8b181-2f27-423e-96c9-8057cf0d1851-workload-certs\") pod \"edc8b181-2f27-423e-96c9-8057cf0d1851\" (UID: \"edc8b181-2f27-423e-96c9-8057cf0d1851\") "
Oct 30 22:14:39.444050 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:39.441498    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f98cd426-fb0e-433e-97ff-3fc95c44239b-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f98cd426-fb0e-433e-97ff-3fc95c44239b-credential-socket\") pod \"f98cd426-fb0e-433e-97ff-3fc95c44239b\" (UID: \"f98cd426-fb0e-433e-97ff-3fc95c44239b\") "
Oct 30 22:14:39.444050 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:39.441554    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f98cd426-fb0e-433e-97ff-3fc95c44239b-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f98cd426-fb0e-433e-97ff-3fc95c44239b-istio-envoy\") pod \"f98cd426-fb0e-433e-97ff-3fc95c44239b\" (UID: \"f98cd426-fb0e-433e-97ff-3fc95c44239b\") "
Oct 30 22:14:39.444050 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:39.441694    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/edc8b181-2f27-423e-96c9-8057cf0d1851-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/edc8b181-2f27-423e-96c9-8057cf0d1851-credential-socket\") pod \"edc8b181-2f27-423e-96c9-8057cf0d1851\" (UID: \"edc8b181-2f27-423e-96c9-8057cf0d1851\") "
Oct 30 22:14:39.444050 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:39.441770    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f98cd426-fb0e-433e-97ff-3fc95c44239b-kube-api-access-6fj9h\" (UniqueName: \"kubernetes.io/projected/f98cd426-fb0e-433e-97ff-3fc95c44239b-kube-api-access-6fj9h\") pod \"f98cd426-fb0e-433e-97ff-3fc95c44239b\" (UID: \"f98cd426-fb0e-433e-97ff-3fc95c44239b\") "
Oct 30 22:14:39.444050 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:39.441802    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/edc8b181-2f27-423e-96c9-8057cf0d1851-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/edc8b181-2f27-423e-96c9-8057cf0d1851-istio-envoy\") pod \"edc8b181-2f27-423e-96c9-8057cf0d1851\" (UID: \"edc8b181-2f27-423e-96c9-8057cf0d1851\") "
Oct 30 22:14:39.444050 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:39.441854    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f98cd426-fb0e-433e-97ff-3fc95c44239b-istio-token\" (UniqueName: \"kubernetes.io/projected/f98cd426-fb0e-433e-97ff-3fc95c44239b-istio-token\") pod \"f98cd426-fb0e-433e-97ff-3fc95c44239b\" (UID: \"f98cd426-fb0e-433e-97ff-3fc95c44239b\") "
Oct 30 22:14:39.444050 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:39.441882    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/edc8b181-2f27-423e-96c9-8057cf0d1851-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/edc8b181-2f27-423e-96c9-8057cf0d1851-istiod-ca-cert\") pod \"edc8b181-2f27-423e-96c9-8057cf0d1851\" (UID: \"edc8b181-2f27-423e-96c9-8057cf0d1851\") "
Oct 30 22:14:39.444050 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:39.441922    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/edc8b181-2f27-423e-96c9-8057cf0d1851-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/edc8b181-2f27-423e-96c9-8057cf0d1851-workload-socket\") pod \"edc8b181-2f27-423e-96c9-8057cf0d1851\" (UID: \"edc8b181-2f27-423e-96c9-8057cf0d1851\") "
Oct 30 22:14:39.444050 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:39.443079    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/edc8b181-2f27-423e-96c9-8057cf0d1851-istio-token\" (UniqueName: \"kubernetes.io/projected/edc8b181-2f27-423e-96c9-8057cf0d1851-istio-token\") pod \"edc8b181-2f27-423e-96c9-8057cf0d1851\" (UID: \"edc8b181-2f27-423e-96c9-8057cf0d1851\") "
Oct 30 22:14:39.444050 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:39.443138    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/f98cd426-fb0e-433e-97ff-3fc95c44239b-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f98cd426-fb0e-433e-97ff-3fc95c44239b-istio-podinfo\") pod \"f98cd426-fb0e-433e-97ff-3fc95c44239b\" (UID: \"f98cd426-fb0e-433e-97ff-3fc95c44239b\") "
Oct 30 22:14:39.444050 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:39.442244    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f98cd426-fb0e-433e-97ff-3fc95c44239b-credential-socket" pod "f98cd426-fb0e-433e-97ff-3fc95c44239b" (UID: "f98cd426-fb0e-433e-97ff-3fc95c44239b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:14:39.444050 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:39.442907    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f98cd426-fb0e-433e-97ff-3fc95c44239b-workload-socket" pod "f98cd426-fb0e-433e-97ff-3fc95c44239b" (UID: "f98cd426-fb0e-433e-97ff-3fc95c44239b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:14:39.444050 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:39.442955    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f98cd426-fb0e-433e-97ff-3fc95c44239b-istiod-ca-cert" pod "f98cd426-fb0e-433e-97ff-3fc95c44239b" (UID: "f98cd426-fb0e-433e-97ff-3fc95c44239b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:14:39.444050 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:39.443283    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/edc8b181-2f27-423e-96c9-8057cf0d1851-workload-certs" pod "edc8b181-2f27-423e-96c9-8057cf0d1851" (UID: "edc8b181-2f27-423e-96c9-8057cf0d1851"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:14:39.444050 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:39.443483    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/edc8b181-2f27-423e-96c9-8057cf0d1851-workload-socket" pod "edc8b181-2f27-423e-96c9-8057cf0d1851" (UID: "edc8b181-2f27-423e-96c9-8057cf0d1851"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:14:39.445180 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:39.444502    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/edc8b181-2f27-423e-96c9-8057cf0d1851-istiod-ca-cert" pod "edc8b181-2f27-423e-96c9-8057cf0d1851" (UID: "edc8b181-2f27-423e-96c9-8057cf0d1851"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:14:39.449957 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:39.449915    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f98cd426-fb0e-433e-97ff-3fc95c44239b-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f98cd426-fb0e-433e-97ff-3fc95c44239b-istio-data\") pod \"f98cd426-fb0e-433e-97ff-3fc95c44239b\" (UID: \"f98cd426-fb0e-433e-97ff-3fc95c44239b\") "
Oct 30 22:14:39.449957 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:39.449944    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/edc8b181-2f27-423e-96c9-8057cf0d1851-credential-socket" pod "edc8b181-2f27-423e-96c9-8057cf0d1851" (UID: "edc8b181-2f27-423e-96c9-8057cf0d1851"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:14:39.450165 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:39.450093    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f98cd426-fb0e-433e-97ff-3fc95c44239b-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f98cd426-fb0e-433e-97ff-3fc95c44239b-workload-certs\") pod \"f98cd426-fb0e-433e-97ff-3fc95c44239b\" (UID: \"f98cd426-fb0e-433e-97ff-3fc95c44239b\") "
Oct 30 22:14:39.450165 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:39.450122    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/edc8b181-2f27-423e-96c9-8057cf0d1851-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/edc8b181-2f27-423e-96c9-8057cf0d1851-istio-data\") pod \"edc8b181-2f27-423e-96c9-8057cf0d1851\" (UID: \"edc8b181-2f27-423e-96c9-8057cf0d1851\") "
Oct 30 22:14:39.450718 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:39.450628    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f98cd426-fb0e-433e-97ff-3fc95c44239b-istio-data" pod "f98cd426-fb0e-433e-97ff-3fc95c44239b" (UID: "f98cd426-fb0e-433e-97ff-3fc95c44239b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:14:39.451037 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:39.450871    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/edc8b181-2f27-423e-96c9-8057cf0d1851-kube-api-access-6jw8w" pod "edc8b181-2f27-423e-96c9-8057cf0d1851" (UID: "edc8b181-2f27-423e-96c9-8057cf0d1851"). InnerVolumeSpecName "kube-api-access-6jw8w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:14:39.451037 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:39.450890    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f98cd426-fb0e-433e-97ff-3fc95c44239b-workload-certs" pod "f98cd426-fb0e-433e-97ff-3fc95c44239b" (UID: "f98cd426-fb0e-433e-97ff-3fc95c44239b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:14:39.451471 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:39.451446    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/edc8b181-2f27-423e-96c9-8057cf0d1851-istio-data" pod "edc8b181-2f27-423e-96c9-8057cf0d1851" (UID: "edc8b181-2f27-423e-96c9-8057cf0d1851"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:14:39.452857 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:39.452835    5510 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/edc8b181-2f27-423e-96c9-8057cf0d1851-workload-certs\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:14:39.453000 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:39.452985    5510 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f98cd426-fb0e-433e-97ff-3fc95c44239b-credential-socket\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:14:39.453276 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:39.453238    5510 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/edc8b181-2f27-423e-96c9-8057cf0d1851-credential-socket\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:14:39.453424 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:39.453407    5510 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/edc8b181-2f27-423e-96c9-8057cf0d1851-istiod-ca-cert\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:14:39.454292 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:39.454272    5510 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/edc8b181-2f27-423e-96c9-8057cf0d1851-workload-socket\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:14:39.454413 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:39.454398    5510 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f98cd426-fb0e-433e-97ff-3fc95c44239b-istio-data\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:14:39.454505 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:39.454493    5510 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f98cd426-fb0e-433e-97ff-3fc95c44239b-workload-certs\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:14:39.454598 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:39.454587    5510 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/edc8b181-2f27-423e-96c9-8057cf0d1851-istio-data\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:14:39.454700 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:39.454680    5510 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f98cd426-fb0e-433e-97ff-3fc95c44239b-istiod-ca-cert\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:14:39.454815 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:39.454793    5510 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f98cd426-fb0e-433e-97ff-3fc95c44239b-workload-socket\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:14:39.454929 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:39.454819    5510 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6jw8w\" (UniqueName: \"kubernetes.io/projected/edc8b181-2f27-423e-96c9-8057cf0d1851-kube-api-access-6jw8w\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:14:39.454929 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:39.453761    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f98cd426-fb0e-433e-97ff-3fc95c44239b-istio-token" pod "f98cd426-fb0e-433e-97ff-3fc95c44239b" (UID: "f98cd426-fb0e-433e-97ff-3fc95c44239b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:14:39.454929 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:39.453819    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f98cd426-fb0e-433e-97ff-3fc95c44239b-kube-api-access-6fj9h" pod "f98cd426-fb0e-433e-97ff-3fc95c44239b" (UID: "f98cd426-fb0e-433e-97ff-3fc95c44239b"). InnerVolumeSpecName "kube-api-access-6fj9h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:14:39.454929 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:39.454093    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/edc8b181-2f27-423e-96c9-8057cf0d1851-istio-envoy" pod "edc8b181-2f27-423e-96c9-8057cf0d1851" (UID: "edc8b181-2f27-423e-96c9-8057cf0d1851"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:14:39.454929 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:39.454373    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/edc8b181-2f27-423e-96c9-8057cf0d1851-istio-podinfo" pod "edc8b181-2f27-423e-96c9-8057cf0d1851" (UID: "edc8b181-2f27-423e-96c9-8057cf0d1851"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:14:39.454929 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:39.454649    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f98cd426-fb0e-433e-97ff-3fc95c44239b-istio-envoy" pod "f98cd426-fb0e-433e-97ff-3fc95c44239b" (UID: "f98cd426-fb0e-433e-97ff-3fc95c44239b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:14:39.455382 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:39.455361    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f98cd426-fb0e-433e-97ff-3fc95c44239b-istio-podinfo" pod "f98cd426-fb0e-433e-97ff-3fc95c44239b" (UID: "f98cd426-fb0e-433e-97ff-3fc95c44239b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:14:39.455486 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:39.455465    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/edc8b181-2f27-423e-96c9-8057cf0d1851-istio-token" pod "edc8b181-2f27-423e-96c9-8057cf0d1851" (UID: "edc8b181-2f27-423e-96c9-8057cf0d1851"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:14:39.556274 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:39.556243    5510 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f98cd426-fb0e-433e-97ff-3fc95c44239b-istio-token\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:14:39.556274 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:39.556271    5510 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/edc8b181-2f27-423e-96c9-8057cf0d1851-istio-token\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:14:39.556274 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:39.556281    5510 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f98cd426-fb0e-433e-97ff-3fc95c44239b-istio-podinfo\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:14:39.556506 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:39.556290    5510 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/edc8b181-2f27-423e-96c9-8057cf0d1851-istio-podinfo\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:14:39.556506 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:39.556300    5510 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f98cd426-fb0e-433e-97ff-3fc95c44239b-istio-envoy\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:14:39.556506 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:39.556312    5510 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6fj9h\" (UniqueName: \"kubernetes.io/projected/f98cd426-fb0e-433e-97ff-3fc95c44239b-kube-api-access-6fj9h\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:14:39.556506 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:39.556325    5510 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/edc8b181-2f27-423e-96c9-8057cf0d1851-istio-envoy\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:14:39.597571 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:39.597533    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-94hzz"]
Oct 30 22:14:39.602376 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:39.602343    5510 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-94hzz"]
Oct 30 22:14:39.844385 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:39.844309    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-cmqdj"]
Oct 30 22:14:39.862693 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:39.862639    5510 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-cmqdj"]
Oct 30 22:14:40.987534 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:40.987505    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-49rg2"]
Oct 30 22:14:40.990888 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:40.990860    5510 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-49rg2"]
Oct 30 22:14:41.152196 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:41.152162    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-fwfzh"]
Oct 30 22:14:41.156589 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:41.156561    5510 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-fwfzh"]
Oct 30 22:14:41.557441 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:41.557407    5510 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="edc8b181-2f27-423e-96c9-8057cf0d1851" path="/var/lib/kubelet/pods/edc8b181-2f27-423e-96c9-8057cf0d1851/volumes"
Oct 30 22:14:41.558043 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:41.558026    5510 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f4952ed4-c979-43db-b64e-a5666ff61bed" path="/var/lib/kubelet/pods/f4952ed4-c979-43db-b64e-a5666ff61bed/volumes"
Oct 30 22:14:41.558477 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:41.558465    5510 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f98cd426-fb0e-433e-97ff-3fc95c44239b" path="/var/lib/kubelet/pods/f98cd426-fb0e-433e-97ff-3fc95c44239b/volumes"
Oct 30 22:14:41.559312 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:41.559296    5510 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fcded500-59d4-493e-bce3-fcf44d59b277" path="/var/lib/kubelet/pods/fcded500-59d4-493e-bce3-fcf44d59b277/volumes"
Oct 30 22:14:43.714392 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:43.714327    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:14:44.306310 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:44.306272    5510 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7403-630546"]
Oct 30 22:14:44.306955 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:44.306939    5510 cpu_manager.go:414] "containerMap: removing container" podUID="f98cd426-fb0e-433e-97ff-3fc95c44239b" containerName="istio-proxy"
Oct 30 22:14:44.307011 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:44.306955    5510 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f98cd426-fb0e-433e-97ff-3fc95c44239b" containerName="istio-proxy"
Oct 30 22:14:44.307011 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:44.306976    5510 cpu_manager.go:414] "containerMap: removing container" podUID="f4952ed4-c979-43db-b64e-a5666ff61bed" containerName="istio-proxy"
Oct 30 22:14:44.307011 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:44.306981    5510 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f4952ed4-c979-43db-b64e-a5666ff61bed" containerName="istio-proxy"
Oct 30 22:14:44.307011 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:44.307006    5510 cpu_manager.go:414] "containerMap: removing container" podUID="edc8b181-2f27-423e-96c9-8057cf0d1851" containerName="istio-proxy"
Oct 30 22:14:44.307147 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:44.307012    5510 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="edc8b181-2f27-423e-96c9-8057cf0d1851" containerName="istio-proxy"
Oct 30 22:14:44.307147 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:44.307033    5510 cpu_manager.go:414] "containerMap: removing container" podUID="fcded500-59d4-493e-bce3-fcf44d59b277" containerName="istio-proxy"
Oct 30 22:14:44.307147 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:44.307038    5510 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fcded500-59d4-493e-bce3-fcf44d59b277" containerName="istio-proxy"
Oct 30 22:14:44.307147 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:44.307144    5510 memory_manager.go:365] "RemoveStaleState removing state" podUID="f4952ed4-c979-43db-b64e-a5666ff61bed" containerName="istio-proxy"
Oct 30 22:14:44.307263 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:44.307157    5510 memory_manager.go:365] "RemoveStaleState removing state" podUID="edc8b181-2f27-423e-96c9-8057cf0d1851" containerName="istio-proxy"
Oct 30 22:14:44.307263 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:44.307166    5510 memory_manager.go:365] "RemoveStaleState removing state" podUID="fcded500-59d4-493e-bce3-fcf44d59b277" containerName="istio-proxy"
Oct 30 22:14:44.307263 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:44.307174    5510 memory_manager.go:365] "RemoveStaleState removing state" podUID="f98cd426-fb0e-433e-97ff-3fc95c44239b" containerName="istio-proxy"
Oct 30 22:14:44.312697 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:44.312673    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7403-630546"
Oct 30 22:14:44.346512 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:44.346469    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2dlsw\" (UniqueName: \"kubernetes.io/projected/2d6ad901-b630-454a-80ed-bb1531a8f9fa-kube-api-access-2dlsw\") pod \"test-curl-7403-630546\" (UID: \"2d6ad901-b630-454a-80ed-bb1531a8f9fa\") " pod="opendatahub/test-curl-7403-630546"
Oct 30 22:14:44.366867 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:44.366834    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7403-630546"]
Oct 30 22:14:44.412838 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:44.412801    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-rx2kz\""
Oct 30 22:14:44.448346 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:44.448303    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2dlsw\" (UniqueName: \"kubernetes.io/projected/2d6ad901-b630-454a-80ed-bb1531a8f9fa-kube-api-access-2dlsw\") pod \"test-curl-7403-630546\" (UID: \"2d6ad901-b630-454a-80ed-bb1531a8f9fa\") " pod="opendatahub/test-curl-7403-630546"
Oct 30 22:14:44.512916 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:44.512879    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2dlsw\" (UniqueName: \"kubernetes.io/projected/2d6ad901-b630-454a-80ed-bb1531a8f9fa-kube-api-access-2dlsw\") pod \"test-curl-7403-630546\" (UID: \"2d6ad901-b630-454a-80ed-bb1531a8f9fa\") " pod="opendatahub/test-curl-7403-630546"
Oct 30 22:14:44.625561 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:44.625463    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7403-630546"
Oct 30 22:14:45.004059 ip-10-0-136-210 kubenswrapper[5510]: W1030 22:14:45.004027    5510 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2d6ad901_b630_454a_80ed_bb1531a8f9fa.slice/crio-2fb7645b3908fbbf9924b252aa78fd8d4f14ffab0ab576e9cb3f53e37cf7e827 WatchSource:0}: Error finding container 2fb7645b3908fbbf9924b252aa78fd8d4f14ffab0ab576e9cb3f53e37cf7e827: Status 404 returned error can't find the container with id 2fb7645b3908fbbf9924b252aa78fd8d4f14ffab0ab576e9cb3f53e37cf7e827
Oct 30 22:14:45.006211 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:45.006190    5510 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 22:14:45.006604 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:45.006581    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7403-630546"]
Oct 30 22:14:45.367176 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:45.367139    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7403-630546" event={"ID":"2d6ad901-b630-454a-80ed-bb1531a8f9fa","Type":"ContainerStarted","Data":"2fb7645b3908fbbf9924b252aa78fd8d4f14ffab0ab576e9cb3f53e37cf7e827"}
Oct 30 22:14:48.294731 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:48.293641    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:14:48.411159 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:48.411023    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7403-630546" event={"ID":"2d6ad901-b630-454a-80ed-bb1531a8f9fa","Type":"ContainerStarted","Data":"468acb224c02959bb3c612a13ae3fb91829d1bca9c300bb9c23dbeea44da77bb"}
Oct 30 22:14:48.528248 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:48.528193    5510 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/test-curl-7403-630546" podStartSLOduration=1.391143657 podStartE2EDuration="4.528173906s" totalImagesPullingTime="3.137030249s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:14:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:14:48.526923712 +0000 UTC m=+1469.087733714" watchObservedRunningTime="2026-10-30 22:14:48.528173906 +0000 UTC m=+1469.088983914"
Oct 30 22:14:50.065682 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:50.065614    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7403-630546"]
Oct 30 22:14:50.071374 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:50.071333    5510 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7403-630546"]
Oct 30 22:14:50.242576 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:50.242541    5510 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7403-96b1e8"]
Oct 30 22:14:50.243322 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:50.243303    5510 cpu_manager.go:414] "containerMap: removing container" podUID="2d6ad901-b630-454a-80ed-bb1531a8f9fa" containerName="test-curl-7403-630546"
Oct 30 22:14:50.243373 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:50.243325    5510 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2d6ad901-b630-454a-80ed-bb1531a8f9fa" containerName="test-curl-7403-630546"
Oct 30 22:14:50.243446 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:50.243437    5510 memory_manager.go:365] "RemoveStaleState removing state" podUID="2d6ad901-b630-454a-80ed-bb1531a8f9fa" containerName="test-curl-7403-630546"
Oct 30 22:14:50.247371 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:50.247354    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7403-96b1e8"
Oct 30 22:14:50.304187 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:50.304098    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7403-96b1e8"]
Oct 30 22:14:50.348698 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:50.348594    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l5kbq\" (UniqueName: \"kubernetes.io/projected/2205f970-83d3-49da-bc25-2f9f18ceab2e-kube-api-access-l5kbq\") pod \"test-curl-7403-96b1e8\" (UID: \"2205f970-83d3-49da-bc25-2f9f18ceab2e\") " pod="opendatahub/test-curl-7403-96b1e8"
Oct 30 22:14:50.423734 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:50.423670    5510 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/test-curl-7403-630546" podUID="2d6ad901-b630-454a-80ed-bb1531a8f9fa" containerName="test-curl-7403-630546" containerID="cri-o://468acb224c02959bb3c612a13ae3fb91829d1bca9c300bb9c23dbeea44da77bb" gracePeriod=2
Oct 30 22:14:50.450135 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:50.450104    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l5kbq\" (UniqueName: \"kubernetes.io/projected/2205f970-83d3-49da-bc25-2f9f18ceab2e-kube-api-access-l5kbq\") pod \"test-curl-7403-96b1e8\" (UID: \"2205f970-83d3-49da-bc25-2f9f18ceab2e\") " pod="opendatahub/test-curl-7403-96b1e8"
Oct 30 22:14:50.475913 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:14:50.475869    5510 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7403-630546\" is forbidden: User \"system:node:ip-10-0-136-210.us-west-2.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-136-210.us-west-2.compute.internal' and this object" podUID="2d6ad901-b630-454a-80ed-bb1531a8f9fa" pod="opendatahub/test-curl-7403-630546"
Oct 30 22:14:50.516783 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:50.516743    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l5kbq\" (UniqueName: \"kubernetes.io/projected/2205f970-83d3-49da-bc25-2f9f18ceab2e-kube-api-access-l5kbq\") pod \"test-curl-7403-96b1e8\" (UID: \"2205f970-83d3-49da-bc25-2f9f18ceab2e\") " pod="opendatahub/test-curl-7403-96b1e8"
Oct 30 22:14:50.559649 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:50.559615    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7403-96b1e8"
Oct 30 22:14:50.938741 ip-10-0-136-210 kubenswrapper[5510]: W1030 22:14:50.938694    5510 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2205f970_83d3_49da_bc25_2f9f18ceab2e.slice/crio-7798af4278d0450784db765320f1ddc8d823e2f6ce09feab3e6d88c5d399c5c8 WatchSource:0}: Error finding container 7798af4278d0450784db765320f1ddc8d823e2f6ce09feab3e6d88c5d399c5c8: Status 404 returned error can't find the container with id 7798af4278d0450784db765320f1ddc8d823e2f6ce09feab3e6d88c5d399c5c8
Oct 30 22:14:50.941406 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:50.941377    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7403-96b1e8"]
Oct 30 22:14:51.433280 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:51.433244    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7403-96b1e8" event={"ID":"2205f970-83d3-49da-bc25-2f9f18ceab2e","Type":"ContainerStarted","Data":"cf976a179590000c71f85ddee6e78d2b6419d746c41717fa20575897fbcb4044"}
Oct 30 22:14:51.433711 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:51.433288    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7403-96b1e8" event={"ID":"2205f970-83d3-49da-bc25-2f9f18ceab2e","Type":"ContainerStarted","Data":"7798af4278d0450784db765320f1ddc8d823e2f6ce09feab3e6d88c5d399c5c8"}
Oct 30 22:14:51.485363 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:14:51.485321    5510 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7403-630546\" is forbidden: User \"system:node:ip-10-0-136-210.us-west-2.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-136-210.us-west-2.compute.internal' and this object" podUID="2d6ad901-b630-454a-80ed-bb1531a8f9fa" pod="opendatahub/test-curl-7403-630546"
Oct 30 22:14:51.606056 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:51.605989    5510 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/test-curl-7403-96b1e8" podStartSLOduration=1.605972415 podStartE2EDuration="1.605972415s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:14:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:14:51.602812393 +0000 UTC m=+1472.163622424" watchObservedRunningTime="2026-10-30 22:14:51.605972415 +0000 UTC m=+1472.166782420"
Oct 30 22:14:52.680878 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:52.680812    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7403-630546"
Oct 30 22:14:52.785741 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:52.785706    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2d6ad901-b630-454a-80ed-bb1531a8f9fa-kube-api-access-2dlsw\" (UniqueName: \"kubernetes.io/projected/2d6ad901-b630-454a-80ed-bb1531a8f9fa-kube-api-access-2dlsw\") pod \"2d6ad901-b630-454a-80ed-bb1531a8f9fa\" (UID: \"2d6ad901-b630-454a-80ed-bb1531a8f9fa\") "
Oct 30 22:14:52.787814 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:52.787784    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2d6ad901-b630-454a-80ed-bb1531a8f9fa-kube-api-access-2dlsw" pod "2d6ad901-b630-454a-80ed-bb1531a8f9fa" (UID: "2d6ad901-b630-454a-80ed-bb1531a8f9fa"). InnerVolumeSpecName "kube-api-access-2dlsw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:14:52.898690 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:52.897780    5510 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2dlsw\" (UniqueName: \"kubernetes.io/projected/2d6ad901-b630-454a-80ed-bb1531a8f9fa-kube-api-access-2dlsw\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:14:52.898690 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:52.898332    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:14:53.266637 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:53.266599    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7403-96b1e8"]
Oct 30 22:14:53.278361 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:53.278316    5510 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7403-96b1e8"]
Oct 30 22:14:53.448382 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:53.448347    5510 generic.go:356] "Generic (PLEG): container finished" podID="2d6ad901-b630-454a-80ed-bb1531a8f9fa" containerID="468acb224c02959bb3c612a13ae3fb91829d1bca9c300bb9c23dbeea44da77bb" exitCode=137
Oct 30 22:14:53.448570 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:53.448405    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7403-630546"
Oct 30 22:14:53.448570 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:53.448444    5510 scope.go:122] "RemoveContainer" containerID="468acb224c02959bb3c612a13ae3fb91829d1bca9c300bb9c23dbeea44da77bb"
Oct 30 22:14:53.448751 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:53.448710    5510 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/test-curl-7403-96b1e8" podUID="2205f970-83d3-49da-bc25-2f9f18ceab2e" containerName="test-curl-7403-96b1e8" containerID="cri-o://cf976a179590000c71f85ddee6e78d2b6419d746c41717fa20575897fbcb4044" gracePeriod=2
Oct 30 22:14:53.460153 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:53.460131    5510 scope.go:122] "RemoveContainer" containerID="468acb224c02959bb3c612a13ae3fb91829d1bca9c300bb9c23dbeea44da77bb"
Oct 30 22:14:53.460443 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:14:53.460422    5510 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"468acb224c02959bb3c612a13ae3fb91829d1bca9c300bb9c23dbeea44da77bb\": container with ID starting with 468acb224c02959bb3c612a13ae3fb91829d1bca9c300bb9c23dbeea44da77bb not found: ID does not exist" containerID="468acb224c02959bb3c612a13ae3fb91829d1bca9c300bb9c23dbeea44da77bb"
Oct 30 22:14:53.460504 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:53.460451    5510 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"468acb224c02959bb3c612a13ae3fb91829d1bca9c300bb9c23dbeea44da77bb"} err="failed to get container status \"468acb224c02959bb3c612a13ae3fb91829d1bca9c300bb9c23dbeea44da77bb\": rpc error: code = NotFound desc = could not find container \"468acb224c02959bb3c612a13ae3fb91829d1bca9c300bb9c23dbeea44da77bb\": container with ID starting with 468acb224c02959bb3c612a13ae3fb91829d1bca9c300bb9c23dbeea44da77bb not found: ID does not exist"
Oct 30 22:14:53.500716 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:14:53.500672    5510 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7403-96b1e8\" is forbidden: User \"system:node:ip-10-0-136-210.us-west-2.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-136-210.us-west-2.compute.internal' and this object" podUID="2205f970-83d3-49da-bc25-2f9f18ceab2e" pod="opendatahub/test-curl-7403-96b1e8"
Oct 30 22:14:53.552592 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:14:53.552556    5510 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7403-630546\" is forbidden: User \"system:node:ip-10-0-136-210.us-west-2.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-136-210.us-west-2.compute.internal' and this object" podUID="2d6ad901-b630-454a-80ed-bb1531a8f9fa" pod="opendatahub/test-curl-7403-630546"
Oct 30 22:14:53.555243 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:53.555218    5510 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2d6ad901-b630-454a-80ed-bb1531a8f9fa" path="/var/lib/kubelet/pods/2d6ad901-b630-454a-80ed-bb1531a8f9fa/volumes"
Oct 30 22:14:53.604550 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:14:53.604503    5510 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7403-96b1e8\" is forbidden: User \"system:node:ip-10-0-136-210.us-west-2.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-136-210.us-west-2.compute.internal' and this object" podUID="2205f970-83d3-49da-bc25-2f9f18ceab2e" pod="opendatahub/test-curl-7403-96b1e8"
Oct 30 22:14:54.254233 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:54.254200    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:14:54.489384 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:54.489345    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:14:55.213948 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:55.213910    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:14:55.718619 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:55.718557    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7403-96b1e8"
Oct 30 22:14:55.740132 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:55.740099    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2205f970-83d3-49da-bc25-2f9f18ceab2e-kube-api-access-l5kbq\" (UniqueName: \"kubernetes.io/projected/2205f970-83d3-49da-bc25-2f9f18ceab2e-kube-api-access-l5kbq\") pod \"2205f970-83d3-49da-bc25-2f9f18ceab2e\" (UID: \"2205f970-83d3-49da-bc25-2f9f18ceab2e\") "
Oct 30 22:14:55.742209 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:55.742176    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2205f970-83d3-49da-bc25-2f9f18ceab2e-kube-api-access-l5kbq" pod "2205f970-83d3-49da-bc25-2f9f18ceab2e" (UID: "2205f970-83d3-49da-bc25-2f9f18ceab2e"). InnerVolumeSpecName "kube-api-access-l5kbq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:14:55.842403 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:55.842366    5510 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l5kbq\" (UniqueName: \"kubernetes.io/projected/2205f970-83d3-49da-bc25-2f9f18ceab2e-kube-api-access-l5kbq\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:14:56.348556 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:56.348516    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:14:56.471503 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:56.471465    5510 generic.go:356] "Generic (PLEG): container finished" podID="2205f970-83d3-49da-bc25-2f9f18ceab2e" containerID="cf976a179590000c71f85ddee6e78d2b6419d746c41717fa20575897fbcb4044" exitCode=137
Oct 30 22:14:56.471705 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:56.471505    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7403-96b1e8"
Oct 30 22:14:56.471705 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:56.471531    5510 scope.go:122] "RemoveContainer" containerID="cf976a179590000c71f85ddee6e78d2b6419d746c41717fa20575897fbcb4044"
Oct 30 22:14:56.481920 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:56.481897    5510 scope.go:122] "RemoveContainer" containerID="cf976a179590000c71f85ddee6e78d2b6419d746c41717fa20575897fbcb4044"
Oct 30 22:14:56.482211 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:14:56.482186    5510 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cf976a179590000c71f85ddee6e78d2b6419d746c41717fa20575897fbcb4044\": container with ID starting with cf976a179590000c71f85ddee6e78d2b6419d746c41717fa20575897fbcb4044 not found: ID does not exist" containerID="cf976a179590000c71f85ddee6e78d2b6419d746c41717fa20575897fbcb4044"
Oct 30 22:14:56.482284 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:56.482222    5510 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cf976a179590000c71f85ddee6e78d2b6419d746c41717fa20575897fbcb4044"} err="failed to get container status \"cf976a179590000c71f85ddee6e78d2b6419d746c41717fa20575897fbcb4044\": rpc error: code = NotFound desc = could not find container \"cf976a179590000c71f85ddee6e78d2b6419d746c41717fa20575897fbcb4044\": container with ID starting with cf976a179590000c71f85ddee6e78d2b6419d746c41717fa20575897fbcb4044 not found: ID does not exist"
Oct 30 22:14:56.531918 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:14:56.531874    5510 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7403-96b1e8\" is forbidden: User \"system:node:ip-10-0-136-210.us-west-2.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-136-210.us-west-2.compute.internal' and this object" podUID="2205f970-83d3-49da-bc25-2f9f18ceab2e" pod="opendatahub/test-curl-7403-96b1e8"
Oct 30 22:14:56.673036 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:56.672951    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:14:56.957965 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:56.957873    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:14:57.167715 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:57.167667    5510 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-67184f2a-gw-openshift-default-5cb54945b9-5c64x"]
Oct 30 22:14:57.168387 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:57.168364    5510 cpu_manager.go:414] "containerMap: removing container" podUID="2205f970-83d3-49da-bc25-2f9f18ceab2e" containerName="test-curl-7403-96b1e8"
Oct 30 22:14:57.168387 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:57.168380    5510 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2205f970-83d3-49da-bc25-2f9f18ceab2e" containerName="test-curl-7403-96b1e8"
Oct 30 22:14:57.168577 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:57.168516    5510 memory_manager.go:365] "RemoveStaleState removing state" podUID="2205f970-83d3-49da-bc25-2f9f18ceab2e" containerName="test-curl-7403-96b1e8"
Oct 30 22:14:57.173430 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:57.173407    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-67184f2a-gw-openshift-default-5cb54945b9-5c64x"
Oct 30 22:14:57.225839 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:14:57.225742    5510 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7403-96b1e8\" is forbidden: User \"system:node:ip-10-0-136-210.us-west-2.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-136-210.us-west-2.compute.internal' and this object" podUID="2205f970-83d3-49da-bc25-2f9f18ceab2e" pod="opendatahub/test-curl-7403-96b1e8"
Oct 30 22:14:57.226885 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:57.226853    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-67184f2a-gw-openshift-default-dockercfg-8556d\""
Oct 30 22:14:57.240094 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:57.240064    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-67184f2a-gw-openshift-default-5cb54945b9-5c64x"]
Oct 30 22:14:57.265393 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:57.265348    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p6g2v\" (UniqueName: \"kubernetes.io/projected/f067f5fc-05db-4807-982f-21e7a5997acc-kube-api-access-p6g2v\") pod \"e2e-migrate-67184f2a-gw-openshift-default-5cb54945b9-5c64x\" (UID: \"f067f5fc-05db-4807-982f-21e7a5997acc\") " pod="openshift-ingress/e2e-migrate-67184f2a-gw-openshift-default-5cb54945b9-5c64x"
Oct 30 22:14:57.265579 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:57.265411    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f067f5fc-05db-4807-982f-21e7a5997acc-workload-certs\") pod \"e2e-migrate-67184f2a-gw-openshift-default-5cb54945b9-5c64x\" (UID: \"f067f5fc-05db-4807-982f-21e7a5997acc\") " pod="openshift-ingress/e2e-migrate-67184f2a-gw-openshift-default-5cb54945b9-5c64x"
Oct 30 22:14:57.265579 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:57.265546    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f067f5fc-05db-4807-982f-21e7a5997acc-istiod-ca-cert\") pod \"e2e-migrate-67184f2a-gw-openshift-default-5cb54945b9-5c64x\" (UID: \"f067f5fc-05db-4807-982f-21e7a5997acc\") " pod="openshift-ingress/e2e-migrate-67184f2a-gw-openshift-default-5cb54945b9-5c64x"
Oct 30 22:14:57.265721 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:57.265610    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f067f5fc-05db-4807-982f-21e7a5997acc-istio-data\") pod \"e2e-migrate-67184f2a-gw-openshift-default-5cb54945b9-5c64x\" (UID: \"f067f5fc-05db-4807-982f-21e7a5997acc\") " pod="openshift-ingress/e2e-migrate-67184f2a-gw-openshift-default-5cb54945b9-5c64x"
Oct 30 22:14:57.265796 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:57.265773    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f067f5fc-05db-4807-982f-21e7a5997acc-credential-socket\") pod \"e2e-migrate-67184f2a-gw-openshift-default-5cb54945b9-5c64x\" (UID: \"f067f5fc-05db-4807-982f-21e7a5997acc\") " pod="openshift-ingress/e2e-migrate-67184f2a-gw-openshift-default-5cb54945b9-5c64x"
Oct 30 22:14:57.265897 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:57.265878    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f067f5fc-05db-4807-982f-21e7a5997acc-istio-envoy\") pod \"e2e-migrate-67184f2a-gw-openshift-default-5cb54945b9-5c64x\" (UID: \"f067f5fc-05db-4807-982f-21e7a5997acc\") " pod="openshift-ingress/e2e-migrate-67184f2a-gw-openshift-default-5cb54945b9-5c64x"
Oct 30 22:14:57.265935 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:57.265924    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f067f5fc-05db-4807-982f-21e7a5997acc-istio-token\") pod \"e2e-migrate-67184f2a-gw-openshift-default-5cb54945b9-5c64x\" (UID: \"f067f5fc-05db-4807-982f-21e7a5997acc\") " pod="openshift-ingress/e2e-migrate-67184f2a-gw-openshift-default-5cb54945b9-5c64x"
Oct 30 22:14:57.266034 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:57.266017    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f067f5fc-05db-4807-982f-21e7a5997acc-workload-socket\") pod \"e2e-migrate-67184f2a-gw-openshift-default-5cb54945b9-5c64x\" (UID: \"f067f5fc-05db-4807-982f-21e7a5997acc\") " pod="openshift-ingress/e2e-migrate-67184f2a-gw-openshift-default-5cb54945b9-5c64x"
Oct 30 22:14:57.266092 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:57.266050    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f067f5fc-05db-4807-982f-21e7a5997acc-istio-podinfo\") pod \"e2e-migrate-67184f2a-gw-openshift-default-5cb54945b9-5c64x\" (UID: \"f067f5fc-05db-4807-982f-21e7a5997acc\") " pod="openshift-ingress/e2e-migrate-67184f2a-gw-openshift-default-5cb54945b9-5c64x"
Oct 30 22:14:57.306425 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:57.306374    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:14:57.371715 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:57.371674    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p6g2v\" (UniqueName: \"kubernetes.io/projected/f067f5fc-05db-4807-982f-21e7a5997acc-kube-api-access-p6g2v\") pod \"e2e-migrate-67184f2a-gw-openshift-default-5cb54945b9-5c64x\" (UID: \"f067f5fc-05db-4807-982f-21e7a5997acc\") " pod="openshift-ingress/e2e-migrate-67184f2a-gw-openshift-default-5cb54945b9-5c64x"
Oct 30 22:14:57.392684 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:57.371967    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f067f5fc-05db-4807-982f-21e7a5997acc-workload-certs\") pod \"e2e-migrate-67184f2a-gw-openshift-default-5cb54945b9-5c64x\" (UID: \"f067f5fc-05db-4807-982f-21e7a5997acc\") " pod="openshift-ingress/e2e-migrate-67184f2a-gw-openshift-default-5cb54945b9-5c64x"
Oct 30 22:14:57.393091 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:57.393056    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f067f5fc-05db-4807-982f-21e7a5997acc-istiod-ca-cert\") pod \"e2e-migrate-67184f2a-gw-openshift-default-5cb54945b9-5c64x\" (UID: \"f067f5fc-05db-4807-982f-21e7a5997acc\") " pod="openshift-ingress/e2e-migrate-67184f2a-gw-openshift-default-5cb54945b9-5c64x"
Oct 30 22:14:57.393282 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:57.393265    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f067f5fc-05db-4807-982f-21e7a5997acc-istio-data\") pod \"e2e-migrate-67184f2a-gw-openshift-default-5cb54945b9-5c64x\" (UID: \"f067f5fc-05db-4807-982f-21e7a5997acc\") " pod="openshift-ingress/e2e-migrate-67184f2a-gw-openshift-default-5cb54945b9-5c64x"
Oct 30 22:14:57.396681 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:57.393459    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f067f5fc-05db-4807-982f-21e7a5997acc-credential-socket\") pod \"e2e-migrate-67184f2a-gw-openshift-default-5cb54945b9-5c64x\" (UID: \"f067f5fc-05db-4807-982f-21e7a5997acc\") " pod="openshift-ingress/e2e-migrate-67184f2a-gw-openshift-default-5cb54945b9-5c64x"
Oct 30 22:14:57.409129 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:57.400792    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f067f5fc-05db-4807-982f-21e7a5997acc-istio-envoy\") pod \"e2e-migrate-67184f2a-gw-openshift-default-5cb54945b9-5c64x\" (UID: \"f067f5fc-05db-4807-982f-21e7a5997acc\") " pod="openshift-ingress/e2e-migrate-67184f2a-gw-openshift-default-5cb54945b9-5c64x"
Oct 30 22:14:57.409129 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:57.401819    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f067f5fc-05db-4807-982f-21e7a5997acc-istio-token\") pod \"e2e-migrate-67184f2a-gw-openshift-default-5cb54945b9-5c64x\" (UID: \"f067f5fc-05db-4807-982f-21e7a5997acc\") " pod="openshift-ingress/e2e-migrate-67184f2a-gw-openshift-default-5cb54945b9-5c64x"
Oct 30 22:14:57.409129 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:57.401946    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f067f5fc-05db-4807-982f-21e7a5997acc-workload-socket\") pod \"e2e-migrate-67184f2a-gw-openshift-default-5cb54945b9-5c64x\" (UID: \"f067f5fc-05db-4807-982f-21e7a5997acc\") " pod="openshift-ingress/e2e-migrate-67184f2a-gw-openshift-default-5cb54945b9-5c64x"
Oct 30 22:14:57.409129 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:57.401973    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f067f5fc-05db-4807-982f-21e7a5997acc-istio-podinfo\") pod \"e2e-migrate-67184f2a-gw-openshift-default-5cb54945b9-5c64x\" (UID: \"f067f5fc-05db-4807-982f-21e7a5997acc\") " pod="openshift-ingress/e2e-migrate-67184f2a-gw-openshift-default-5cb54945b9-5c64x"
Oct 30 22:14:57.409129 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:57.398256    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f067f5fc-05db-4807-982f-21e7a5997acc-workload-certs\") pod \"e2e-migrate-67184f2a-gw-openshift-default-5cb54945b9-5c64x\" (UID: \"f067f5fc-05db-4807-982f-21e7a5997acc\") " pod="openshift-ingress/e2e-migrate-67184f2a-gw-openshift-default-5cb54945b9-5c64x"
Oct 30 22:14:57.409129 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:57.399310    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f067f5fc-05db-4807-982f-21e7a5997acc-istiod-ca-cert\") pod \"e2e-migrate-67184f2a-gw-openshift-default-5cb54945b9-5c64x\" (UID: \"f067f5fc-05db-4807-982f-21e7a5997acc\") " pod="openshift-ingress/e2e-migrate-67184f2a-gw-openshift-default-5cb54945b9-5c64x"
Oct 30 22:14:57.409129 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:57.399539    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f067f5fc-05db-4807-982f-21e7a5997acc-istio-data\") pod \"e2e-migrate-67184f2a-gw-openshift-default-5cb54945b9-5c64x\" (UID: \"f067f5fc-05db-4807-982f-21e7a5997acc\") " pod="openshift-ingress/e2e-migrate-67184f2a-gw-openshift-default-5cb54945b9-5c64x"
Oct 30 22:14:57.409129 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:57.402771    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f067f5fc-05db-4807-982f-21e7a5997acc-workload-socket\") pod \"e2e-migrate-67184f2a-gw-openshift-default-5cb54945b9-5c64x\" (UID: \"f067f5fc-05db-4807-982f-21e7a5997acc\") " pod="openshift-ingress/e2e-migrate-67184f2a-gw-openshift-default-5cb54945b9-5c64x"
Oct 30 22:14:57.409129 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:57.399774    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f067f5fc-05db-4807-982f-21e7a5997acc-credential-socket\") pod \"e2e-migrate-67184f2a-gw-openshift-default-5cb54945b9-5c64x\" (UID: \"f067f5fc-05db-4807-982f-21e7a5997acc\") " pod="openshift-ingress/e2e-migrate-67184f2a-gw-openshift-default-5cb54945b9-5c64x"
Oct 30 22:14:57.443690 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:57.437742    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f067f5fc-05db-4807-982f-21e7a5997acc-istio-podinfo\") pod \"e2e-migrate-67184f2a-gw-openshift-default-5cb54945b9-5c64x\" (UID: \"f067f5fc-05db-4807-982f-21e7a5997acc\") " pod="openshift-ingress/e2e-migrate-67184f2a-gw-openshift-default-5cb54945b9-5c64x"
Oct 30 22:14:57.536360 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:57.515167    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f067f5fc-05db-4807-982f-21e7a5997acc-istio-token\") pod \"e2e-migrate-67184f2a-gw-openshift-default-5cb54945b9-5c64x\" (UID: \"f067f5fc-05db-4807-982f-21e7a5997acc\") " pod="openshift-ingress/e2e-migrate-67184f2a-gw-openshift-default-5cb54945b9-5c64x"
Oct 30 22:14:57.536360 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:57.515875    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p6g2v\" (UniqueName: \"kubernetes.io/projected/f067f5fc-05db-4807-982f-21e7a5997acc-kube-api-access-p6g2v\") pod \"e2e-migrate-67184f2a-gw-openshift-default-5cb54945b9-5c64x\" (UID: \"f067f5fc-05db-4807-982f-21e7a5997acc\") " pod="openshift-ingress/e2e-migrate-67184f2a-gw-openshift-default-5cb54945b9-5c64x"
Oct 30 22:14:57.536360 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:57.534065    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f067f5fc-05db-4807-982f-21e7a5997acc-istio-envoy\") pod \"e2e-migrate-67184f2a-gw-openshift-default-5cb54945b9-5c64x\" (UID: \"f067f5fc-05db-4807-982f-21e7a5997acc\") " pod="openshift-ingress/e2e-migrate-67184f2a-gw-openshift-default-5cb54945b9-5c64x"
Oct 30 22:14:57.536360 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:57.534216    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-67184f2a-gw-openshift-default-5cb54945b9-5c64x"
Oct 30 22:14:57.649601 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:57.649526    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:14:57.696825 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:57.696767    5510 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2205f970-83d3-49da-bc25-2f9f18ceab2e" path="/var/lib/kubelet/pods/2205f970-83d3-49da-bc25-2f9f18ceab2e/volumes"
Oct 30 22:14:57.972681 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:57.971550    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:14:58.206282 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:58.206199    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-67184f2a-gw-openshift-default-5cb54945b9-5c64x"]
Oct 30 22:14:58.234059 ip-10-0-136-210 kubenswrapper[5510]: W1030 22:14:58.234030    5510 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf067f5fc_05db_4807_982f_21e7a5997acc.slice/crio-b5fd0cae1043845ebf8ea572edf1a3890e743aef23f8579cb3589b6a282d56e9 WatchSource:0}: Error finding container b5fd0cae1043845ebf8ea572edf1a3890e743aef23f8579cb3589b6a282d56e9: Status 404 returned error can't find the container with id b5fd0cae1043845ebf8ea572edf1a3890e743aef23f8579cb3589b6a282d56e9
Oct 30 22:14:58.236723 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:58.236684    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:14:58.236836 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:58.236800    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:14:58.236903 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:58.236850    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:14:58.711716 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:58.711673    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-67184f2a-gw-openshift-default-5cb54945b9-5c64x" event={"ID":"f067f5fc-05db-4807-982f-21e7a5997acc","Type":"ContainerStarted","Data":"9ef24aa63fe5461c2c7c015615611beddd21df935aa93d916eb5ba896912143f"}
Oct 30 22:14:58.711946 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:58.711738    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-67184f2a-gw-openshift-default-5cb54945b9-5c64x" event={"ID":"f067f5fc-05db-4807-982f-21e7a5997acc","Type":"ContainerStarted","Data":"b5fd0cae1043845ebf8ea572edf1a3890e743aef23f8579cb3589b6a282d56e9"}
Oct 30 22:14:58.837345 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:58.837291    5510 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-67184f2a-gw-openshift-default-5cb54945b9-5c64x" podStartSLOduration=1.837277431 podStartE2EDuration="1.837277431s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:14:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:14:58.834298675 +0000 UTC m=+1479.395108680" watchObservedRunningTime="2026-10-30 22:14:58.837277431 +0000 UTC m=+1479.398087437"
Oct 30 22:14:59.534990 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:59.534953    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-67184f2a-gw-openshift-default-5cb54945b9-5c64x"
Oct 30 22:14:59.544056 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:59.543982    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-67184f2a-gw-openshift-default-5cb54945b9-5c64x"
Oct 30 22:14:59.724090 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:59.724053    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-67184f2a-gw-openshift-default-5cb54945b9-5c64x"
Oct 30 22:14:59.725015 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:14:59.724989    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-67184f2a-gw-openshift-default-5cb54945b9-5c64x"
Oct 30 22:15:01.321848 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:01.321816    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:15:01.581825 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:01.581733    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:15:02.837750 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:02.837710    5510 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-g64nz"]
Oct 30 22:15:02.842231 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:02.842215    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-g64nz"
Oct 30 22:15:02.924918 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:02.924864    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/71b34137-bc68-4228-a7e0-30fda97183aa-credential-socket\") pod \"e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-g64nz\" (UID: \"71b34137-bc68-4228-a7e0-30fda97183aa\") " pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-g64nz"
Oct 30 22:15:02.924918 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:02.924918    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/71b34137-bc68-4228-a7e0-30fda97183aa-workload-certs\") pod \"e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-g64nz\" (UID: \"71b34137-bc68-4228-a7e0-30fda97183aa\") " pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-g64nz"
Oct 30 22:15:02.925163 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:02.924960    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/71b34137-bc68-4228-a7e0-30fda97183aa-istio-token\") pod \"e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-g64nz\" (UID: \"71b34137-bc68-4228-a7e0-30fda97183aa\") " pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-g64nz"
Oct 30 22:15:02.925163 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:02.924998    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/71b34137-bc68-4228-a7e0-30fda97183aa-workload-socket\") pod \"e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-g64nz\" (UID: \"71b34137-bc68-4228-a7e0-30fda97183aa\") " pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-g64nz"
Oct 30 22:15:02.925163 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:02.925022    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/71b34137-bc68-4228-a7e0-30fda97183aa-istio-podinfo\") pod \"e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-g64nz\" (UID: \"71b34137-bc68-4228-a7e0-30fda97183aa\") " pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-g64nz"
Oct 30 22:15:02.925163 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:02.925073    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/71b34137-bc68-4228-a7e0-30fda97183aa-istio-envoy\") pod \"e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-g64nz\" (UID: \"71b34137-bc68-4228-a7e0-30fda97183aa\") " pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-g64nz"
Oct 30 22:15:02.925163 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:02.925142    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mmbqj\" (UniqueName: \"kubernetes.io/projected/71b34137-bc68-4228-a7e0-30fda97183aa-kube-api-access-mmbqj\") pod \"e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-g64nz\" (UID: \"71b34137-bc68-4228-a7e0-30fda97183aa\") " pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-g64nz"
Oct 30 22:15:02.925469 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:02.925242    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/71b34137-bc68-4228-a7e0-30fda97183aa-istio-data\") pod \"e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-g64nz\" (UID: \"71b34137-bc68-4228-a7e0-30fda97183aa\") " pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-g64nz"
Oct 30 22:15:02.925469 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:02.925445    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/71b34137-bc68-4228-a7e0-30fda97183aa-istiod-ca-cert\") pod \"e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-g64nz\" (UID: \"71b34137-bc68-4228-a7e0-30fda97183aa\") " pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-g64nz"
Oct 30 22:15:02.956583 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:02.956549    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-g64nz"]
Oct 30 22:15:03.027369 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:03.027334    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mmbqj\" (UniqueName: \"kubernetes.io/projected/71b34137-bc68-4228-a7e0-30fda97183aa-kube-api-access-mmbqj\") pod \"e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-g64nz\" (UID: \"71b34137-bc68-4228-a7e0-30fda97183aa\") " pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-g64nz"
Oct 30 22:15:03.027586 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:03.027406    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/71b34137-bc68-4228-a7e0-30fda97183aa-istio-data\") pod \"e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-g64nz\" (UID: \"71b34137-bc68-4228-a7e0-30fda97183aa\") " pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-g64nz"
Oct 30 22:15:03.027586 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:03.027506    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/71b34137-bc68-4228-a7e0-30fda97183aa-istiod-ca-cert\") pod \"e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-g64nz\" (UID: \"71b34137-bc68-4228-a7e0-30fda97183aa\") " pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-g64nz"
Oct 30 22:15:03.027586 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:03.027569    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/71b34137-bc68-4228-a7e0-30fda97183aa-credential-socket\") pod \"e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-g64nz\" (UID: \"71b34137-bc68-4228-a7e0-30fda97183aa\") " pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-g64nz"
Oct 30 22:15:03.027586 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:03.027585    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/71b34137-bc68-4228-a7e0-30fda97183aa-workload-certs\") pod \"e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-g64nz\" (UID: \"71b34137-bc68-4228-a7e0-30fda97183aa\") " pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-g64nz"
Oct 30 22:15:03.027838 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:03.027628    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/71b34137-bc68-4228-a7e0-30fda97183aa-istio-token\") pod \"e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-g64nz\" (UID: \"71b34137-bc68-4228-a7e0-30fda97183aa\") " pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-g64nz"
Oct 30 22:15:03.027838 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:03.027651    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/71b34137-bc68-4228-a7e0-30fda97183aa-workload-socket\") pod \"e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-g64nz\" (UID: \"71b34137-bc68-4228-a7e0-30fda97183aa\") " pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-g64nz"
Oct 30 22:15:03.027838 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:03.027691    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/71b34137-bc68-4228-a7e0-30fda97183aa-istio-podinfo\") pod \"e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-g64nz\" (UID: \"71b34137-bc68-4228-a7e0-30fda97183aa\") " pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-g64nz"
Oct 30 22:15:03.027838 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:03.027753    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/71b34137-bc68-4228-a7e0-30fda97183aa-istio-envoy\") pod \"e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-g64nz\" (UID: \"71b34137-bc68-4228-a7e0-30fda97183aa\") " pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-g64nz"
Oct 30 22:15:03.028063 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:03.027860    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/71b34137-bc68-4228-a7e0-30fda97183aa-istio-data\") pod \"e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-g64nz\" (UID: \"71b34137-bc68-4228-a7e0-30fda97183aa\") " pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-g64nz"
Oct 30 22:15:03.028063 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:03.027880    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/71b34137-bc68-4228-a7e0-30fda97183aa-credential-socket\") pod \"e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-g64nz\" (UID: \"71b34137-bc68-4228-a7e0-30fda97183aa\") " pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-g64nz"
Oct 30 22:15:03.028063 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:03.028007    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/71b34137-bc68-4228-a7e0-30fda97183aa-workload-certs\") pod \"e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-g64nz\" (UID: \"71b34137-bc68-4228-a7e0-30fda97183aa\") " pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-g64nz"
Oct 30 22:15:03.028185 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:03.028082    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/71b34137-bc68-4228-a7e0-30fda97183aa-workload-socket\") pod \"e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-g64nz\" (UID: \"71b34137-bc68-4228-a7e0-30fda97183aa\") " pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-g64nz"
Oct 30 22:15:03.028258 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:03.028239    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/71b34137-bc68-4228-a7e0-30fda97183aa-istiod-ca-cert\") pod \"e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-g64nz\" (UID: \"71b34137-bc68-4228-a7e0-30fda97183aa\") " pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-g64nz"
Oct 30 22:15:03.030026 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:03.030004    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/71b34137-bc68-4228-a7e0-30fda97183aa-istio-envoy\") pod \"e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-g64nz\" (UID: \"71b34137-bc68-4228-a7e0-30fda97183aa\") " pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-g64nz"
Oct 30 22:15:03.030268 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:03.030251    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/71b34137-bc68-4228-a7e0-30fda97183aa-istio-podinfo\") pod \"e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-g64nz\" (UID: \"71b34137-bc68-4228-a7e0-30fda97183aa\") " pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-g64nz"
Oct 30 22:15:03.074290 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:03.074255    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:15:03.107009 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:03.106877    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mmbqj\" (UniqueName: \"kubernetes.io/projected/71b34137-bc68-4228-a7e0-30fda97183aa-kube-api-access-mmbqj\") pod \"e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-g64nz\" (UID: \"71b34137-bc68-4228-a7e0-30fda97183aa\") " pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-g64nz"
Oct 30 22:15:03.116560 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:03.116436    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/71b34137-bc68-4228-a7e0-30fda97183aa-istio-token\") pod \"e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-g64nz\" (UID: \"71b34137-bc68-4228-a7e0-30fda97183aa\") " pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-g64nz"
Oct 30 22:15:03.157304 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:03.156998    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-g64nz"
Oct 30 22:15:03.811168 ip-10-0-136-210 kubenswrapper[5510]: W1030 22:15:03.811133    5510 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71b34137_bc68_4228_a7e0_30fda97183aa.slice/crio-ec05d0c6797ccbf7e66f71ace3ab0e03d94d51d6b94c385e599fec6096663472 WatchSource:0}: Error finding container ec05d0c6797ccbf7e66f71ace3ab0e03d94d51d6b94c385e599fec6096663472: Status 404 returned error can't find the container with id ec05d0c6797ccbf7e66f71ace3ab0e03d94d51d6b94c385e599fec6096663472
Oct 30 22:15:03.814225 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:03.814187    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:15:03.814353 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:03.814269    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:15:03.814353 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:03.814315    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:15:03.814353 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:03.814321    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-g64nz"]
Oct 30 22:15:03.952111 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:03.952068    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:15:04.227883 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:04.223721    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:15:04.406679 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:04.402073    5510 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-67184f2a-6fc5c46555-n47rx"]
Oct 30 22:15:04.460687 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:04.460025    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-67184f2a-6fc5c46555-n47rx"]
Oct 30 22:15:04.460687 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:04.460190    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-67184f2a-6fc5c46555-n47rx"
Oct 30 22:15:04.490973 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:04.490886    5510 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-67184f2a-66ffdf88d7-wd7x9"]
Oct 30 22:15:04.554679 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:04.545504    5510 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-67184f2a\""
Oct 30 22:15:04.610678 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:04.600461    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-67184f2a-dockercfg-pm68g\""
Oct 30 22:15:04.610678 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:04.609464    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d3670624-670c-4d8c-9444-3dcdde37c2b0-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-67184f2a-6fc5c46555-n47rx\" (UID: \"d3670624-670c-4d8c-9444-3dcdde37c2b0\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-67184f2a-6fc5c46555-n47rx"
Oct 30 22:15:04.610678 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:04.609706    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2xmjx\" (UniqueName: \"kubernetes.io/projected/d3670624-670c-4d8c-9444-3dcdde37c2b0-kube-api-access-2xmjx\") pod \"payload-pre-processing-e2e-migrate-67184f2a-6fc5c46555-n47rx\" (UID: \"d3670624-670c-4d8c-9444-3dcdde37c2b0\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-67184f2a-6fc5c46555-n47rx"
Oct 30 22:15:04.632054 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:04.619956    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-67184f2a-66ffdf88d7-wd7x9"
Oct 30 22:15:04.717075 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:04.717040    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/11d70426-1070-491a-8b69-d107582114f0-plugins-config-volume\") pod \"payload-processing-e2e-migrate-67184f2a-66ffdf88d7-wd7x9\" (UID: \"11d70426-1070-491a-8b69-d107582114f0\") " pod="openshift-ingress/payload-processing-e2e-migrate-67184f2a-66ffdf88d7-wd7x9"
Oct 30 22:15:04.717478 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:04.717449    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tvcxd\" (UniqueName: \"kubernetes.io/projected/11d70426-1070-491a-8b69-d107582114f0-kube-api-access-tvcxd\") pod \"payload-processing-e2e-migrate-67184f2a-66ffdf88d7-wd7x9\" (UID: \"11d70426-1070-491a-8b69-d107582114f0\") " pod="openshift-ingress/payload-processing-e2e-migrate-67184f2a-66ffdf88d7-wd7x9"
Oct 30 22:15:04.717630 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:04.717610    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d3670624-670c-4d8c-9444-3dcdde37c2b0-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-67184f2a-6fc5c46555-n47rx\" (UID: \"d3670624-670c-4d8c-9444-3dcdde37c2b0\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-67184f2a-6fc5c46555-n47rx"
Oct 30 22:15:04.717949 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:04.717927    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2xmjx\" (UniqueName: \"kubernetes.io/projected/d3670624-670c-4d8c-9444-3dcdde37c2b0-kube-api-access-2xmjx\") pod \"payload-pre-processing-e2e-migrate-67184f2a-6fc5c46555-n47rx\" (UID: \"d3670624-670c-4d8c-9444-3dcdde37c2b0\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-67184f2a-6fc5c46555-n47rx"
Oct 30 22:15:04.722631 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:04.719472    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d3670624-670c-4d8c-9444-3dcdde37c2b0-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-67184f2a-6fc5c46555-n47rx\" (UID: \"d3670624-670c-4d8c-9444-3dcdde37c2b0\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-67184f2a-6fc5c46555-n47rx"
Oct 30 22:15:04.766945 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:04.760612    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:15:04.771947 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:04.771912    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-g64nz" event={"ID":"71b34137-bc68-4228-a7e0-30fda97183aa","Type":"ContainerStarted","Data":"39d70433e536b73933ad2fb97c7e017a6f0de14f8c2b04088965c5afeaa31ebb"}
Oct 30 22:15:04.773255 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:04.773226    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-g64nz" event={"ID":"71b34137-bc68-4228-a7e0-30fda97183aa","Type":"ContainerStarted","Data":"ec05d0c6797ccbf7e66f71ace3ab0e03d94d51d6b94c385e599fec6096663472"}
Oct 30 22:15:04.793765 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:04.793728    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-67184f2a-66ffdf88d7-wd7x9"]
Oct 30 22:15:04.890291 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:04.862176    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tvcxd\" (UniqueName: \"kubernetes.io/projected/11d70426-1070-491a-8b69-d107582114f0-kube-api-access-tvcxd\") pod \"payload-processing-e2e-migrate-67184f2a-66ffdf88d7-wd7x9\" (UID: \"11d70426-1070-491a-8b69-d107582114f0\") " pod="openshift-ingress/payload-processing-e2e-migrate-67184f2a-66ffdf88d7-wd7x9"
Oct 30 22:15:04.891425 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:04.891027    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/11d70426-1070-491a-8b69-d107582114f0-plugins-config-volume\") pod \"payload-processing-e2e-migrate-67184f2a-66ffdf88d7-wd7x9\" (UID: \"11d70426-1070-491a-8b69-d107582114f0\") " pod="openshift-ingress/payload-processing-e2e-migrate-67184f2a-66ffdf88d7-wd7x9"
Oct 30 22:15:04.914686 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:04.911838    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/11d70426-1070-491a-8b69-d107582114f0-plugins-config-volume\") pod \"payload-processing-e2e-migrate-67184f2a-66ffdf88d7-wd7x9\" (UID: \"11d70426-1070-491a-8b69-d107582114f0\") " pod="openshift-ingress/payload-processing-e2e-migrate-67184f2a-66ffdf88d7-wd7x9"
Oct 30 22:15:04.927363 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:04.926472    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2xmjx\" (UniqueName: \"kubernetes.io/projected/d3670624-670c-4d8c-9444-3dcdde37c2b0-kube-api-access-2xmjx\") pod \"payload-pre-processing-e2e-migrate-67184f2a-6fc5c46555-n47rx\" (UID: \"d3670624-670c-4d8c-9444-3dcdde37c2b0\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-67184f2a-6fc5c46555-n47rx"
Oct 30 22:15:04.934432 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:04.934399    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tvcxd\" (UniqueName: \"kubernetes.io/projected/11d70426-1070-491a-8b69-d107582114f0-kube-api-access-tvcxd\") pod \"payload-processing-e2e-migrate-67184f2a-66ffdf88d7-wd7x9\" (UID: \"11d70426-1070-491a-8b69-d107582114f0\") " pod="openshift-ingress/payload-processing-e2e-migrate-67184f2a-66ffdf88d7-wd7x9"
Oct 30 22:15:04.963621 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:04.963578    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:15:04.977224 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:04.977165    5510 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-g64nz" podStartSLOduration=2.977149581 podStartE2EDuration="2.977149581s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:15:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:15:04.975863714 +0000 UTC m=+1485.536673756" watchObservedRunningTime="2026-10-30 22:15:04.977149581 +0000 UTC m=+1485.537959585"
Oct 30 22:15:05.004542 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:05.004504    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-67184f2a-6fc5c46555-n47rx"
Oct 30 22:15:05.028587 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:05.028560    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-67184f2a-66ffdf88d7-wd7x9"
Oct 30 22:15:05.175724 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:05.167945    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:15:05.175724 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:05.169518    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-g64nz"
Oct 30 22:15:05.175724 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:05.169642    5510 patch_prober.go:29] interesting pod/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-g64nz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused" start-of-body=
Oct 30 22:15:05.175724 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:05.169722    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-g64nz" podUID="71b34137-bc68-4228-a7e0-30fda97183aa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused"
Oct 30 22:15:05.399734 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:05.399648    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-67184f2a-6fc5c46555-n47rx"]
Oct 30 22:15:05.431942 ip-10-0-136-210 kubenswrapper[5510]: W1030 22:15:05.431914    5510 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3670624_670c_4d8c_9444_3dcdde37c2b0.slice/crio-bc61321647803fcf46315cc145469ed2878be8239b0a672967db8cb70e9a43bc WatchSource:0}: Error finding container bc61321647803fcf46315cc145469ed2878be8239b0a672967db8cb70e9a43bc: Status 404 returned error can't find the container with id bc61321647803fcf46315cc145469ed2878be8239b0a672967db8cb70e9a43bc
Oct 30 22:15:05.480755 ip-10-0-136-210 kubenswrapper[5510]: W1030 22:15:05.479054    5510 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11d70426_1070_491a_8b69_d107582114f0.slice/crio-139159db2676a62fc140505ccec09ced04e2a255f23072e4de3c86da852a35c1 WatchSource:0}: Error finding container 139159db2676a62fc140505ccec09ced04e2a255f23072e4de3c86da852a35c1: Status 404 returned error can't find the container with id 139159db2676a62fc140505ccec09ced04e2a255f23072e4de3c86da852a35c1
Oct 30 22:15:05.483009 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:05.482981    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-67184f2a-66ffdf88d7-wd7x9"]
Oct 30 22:15:05.672752 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:05.669957    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:15:05.809315 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:05.809275    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-67184f2a-6fc5c46555-n47rx" event={"ID":"d3670624-670c-4d8c-9444-3dcdde37c2b0","Type":"ContainerStarted","Data":"e1944240dfe35a962f87367ada2ac4c19067f1db5694e4a81fb3bd90cd32aa78"}
Oct 30 22:15:05.809580 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:05.809561    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-67184f2a-6fc5c46555-n47rx" event={"ID":"d3670624-670c-4d8c-9444-3dcdde37c2b0","Type":"ContainerStarted","Data":"bc61321647803fcf46315cc145469ed2878be8239b0a672967db8cb70e9a43bc"}
Oct 30 22:15:05.820679 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:05.809976    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-67184f2a-6fc5c46555-n47rx"
Oct 30 22:15:05.829737 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:05.829699    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-67184f2a-66ffdf88d7-wd7x9" event={"ID":"11d70426-1070-491a-8b69-d107582114f0","Type":"ContainerStarted","Data":"1b46aa79142b706cba5223a89e6fcb4c379342f8fcb65dcb66d820deddc8f3e6"}
Oct 30 22:15:05.829986 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:05.829966    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-67184f2a-66ffdf88d7-wd7x9" event={"ID":"11d70426-1070-491a-8b69-d107582114f0","Type":"ContainerStarted","Data":"139159db2676a62fc140505ccec09ced04e2a255f23072e4de3c86da852a35c1"}
Oct 30 22:15:05.833748 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:05.833718    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-migrate-67184f2a-66ffdf88d7-wd7x9"
Oct 30 22:15:05.939582 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:05.939111    5510 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-migrate-67184f2a-6fc5c46555-n47rx" podStartSLOduration=1.9390943059999999 podStartE2EDuration="1.939094306s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:15:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:15:05.934242623 +0000 UTC m=+1486.495052631" watchObservedRunningTime="2026-10-30 22:15:05.939094306 +0000 UTC m=+1486.499904312"
Oct 30 22:15:06.124124 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:06.124064    5510 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-migrate-67184f2a-66ffdf88d7-wd7x9" podStartSLOduration=2.1240494070000002 podStartE2EDuration="2.124049407s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:15:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:15:06.061941686 +0000 UTC m=+1486.622751692" watchObservedRunningTime="2026-10-30 22:15:06.124049407 +0000 UTC m=+1486.684859473"
Oct 30 22:15:06.127101 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:06.127076    5510 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-72b5472b-openshift-default-6c5c978fc5-xs68s"]
Oct 30 22:15:06.131442 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:06.131425    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-72b5472b-openshift-default-6c5c978fc5-xs68s"
Oct 30 22:15:06.161793 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:06.161755    5510 patch_prober.go:29] interesting pod/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-g64nz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused" start-of-body=
Oct 30 22:15:06.162050 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:06.162017    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-g64nz" podUID="71b34137-bc68-4228-a7e0-30fda97183aa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused"
Oct 30 22:15:06.197365 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:06.197284    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-72b5472b-openshift-default-6c5c978fc5-xs68s"]
Oct 30 22:15:06.240382 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:06.240348    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-72b5472b-openshift-default-dockercfg-tmkf9\""
Oct 30 22:15:06.280057 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:06.280013    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ebf683b5-5605-4790-813e-3f06b8f8c38f-istio-token\") pod \"e2e-ait-72b5472b-openshift-default-6c5c978fc5-xs68s\" (UID: \"ebf683b5-5605-4790-813e-3f06b8f8c38f\") " pod="openshift-ingress/e2e-ait-72b5472b-openshift-default-6c5c978fc5-xs68s"
Oct 30 22:15:06.280483 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:06.280458    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ebf683b5-5605-4790-813e-3f06b8f8c38f-istio-data\") pod \"e2e-ait-72b5472b-openshift-default-6c5c978fc5-xs68s\" (UID: \"ebf683b5-5605-4790-813e-3f06b8f8c38f\") " pod="openshift-ingress/e2e-ait-72b5472b-openshift-default-6c5c978fc5-xs68s"
Oct 30 22:15:06.280728 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:06.280709    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ebf683b5-5605-4790-813e-3f06b8f8c38f-workload-socket\") pod \"e2e-ait-72b5472b-openshift-default-6c5c978fc5-xs68s\" (UID: \"ebf683b5-5605-4790-813e-3f06b8f8c38f\") " pod="openshift-ingress/e2e-ait-72b5472b-openshift-default-6c5c978fc5-xs68s"
Oct 30 22:15:06.280997 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:06.280976    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ebf683b5-5605-4790-813e-3f06b8f8c38f-istio-envoy\") pod \"e2e-ait-72b5472b-openshift-default-6c5c978fc5-xs68s\" (UID: \"ebf683b5-5605-4790-813e-3f06b8f8c38f\") " pod="openshift-ingress/e2e-ait-72b5472b-openshift-default-6c5c978fc5-xs68s"
Oct 30 22:15:06.281177 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:06.281160    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ebf683b5-5605-4790-813e-3f06b8f8c38f-credential-socket\") pod \"e2e-ait-72b5472b-openshift-default-6c5c978fc5-xs68s\" (UID: \"ebf683b5-5605-4790-813e-3f06b8f8c38f\") " pod="openshift-ingress/e2e-ait-72b5472b-openshift-default-6c5c978fc5-xs68s"
Oct 30 22:15:06.281368 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:06.281352    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ebf683b5-5605-4790-813e-3f06b8f8c38f-workload-certs\") pod \"e2e-ait-72b5472b-openshift-default-6c5c978fc5-xs68s\" (UID: \"ebf683b5-5605-4790-813e-3f06b8f8c38f\") " pod="openshift-ingress/e2e-ait-72b5472b-openshift-default-6c5c978fc5-xs68s"
Oct 30 22:15:06.281550 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:06.281535    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jml4m\" (UniqueName: \"kubernetes.io/projected/ebf683b5-5605-4790-813e-3f06b8f8c38f-kube-api-access-jml4m\") pod \"e2e-ait-72b5472b-openshift-default-6c5c978fc5-xs68s\" (UID: \"ebf683b5-5605-4790-813e-3f06b8f8c38f\") " pod="openshift-ingress/e2e-ait-72b5472b-openshift-default-6c5c978fc5-xs68s"
Oct 30 22:15:06.281769 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:06.281752    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ebf683b5-5605-4790-813e-3f06b8f8c38f-istiod-ca-cert\") pod \"e2e-ait-72b5472b-openshift-default-6c5c978fc5-xs68s\" (UID: \"ebf683b5-5605-4790-813e-3f06b8f8c38f\") " pod="openshift-ingress/e2e-ait-72b5472b-openshift-default-6c5c978fc5-xs68s"
Oct 30 22:15:06.281955 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:06.281940    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ebf683b5-5605-4790-813e-3f06b8f8c38f-istio-podinfo\") pod \"e2e-ait-72b5472b-openshift-default-6c5c978fc5-xs68s\" (UID: \"ebf683b5-5605-4790-813e-3f06b8f8c38f\") " pod="openshift-ingress/e2e-ait-72b5472b-openshift-default-6c5c978fc5-xs68s"
Oct 30 22:15:06.284007 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:06.283977    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:15:06.506736 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:06.480404    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jml4m\" (UniqueName: \"kubernetes.io/projected/ebf683b5-5605-4790-813e-3f06b8f8c38f-kube-api-access-jml4m\") pod \"e2e-ait-72b5472b-openshift-default-6c5c978fc5-xs68s\" (UID: \"ebf683b5-5605-4790-813e-3f06b8f8c38f\") " pod="openshift-ingress/e2e-ait-72b5472b-openshift-default-6c5c978fc5-xs68s"
Oct 30 22:15:06.506736 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:06.480526    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ebf683b5-5605-4790-813e-3f06b8f8c38f-istiod-ca-cert\") pod \"e2e-ait-72b5472b-openshift-default-6c5c978fc5-xs68s\" (UID: \"ebf683b5-5605-4790-813e-3f06b8f8c38f\") " pod="openshift-ingress/e2e-ait-72b5472b-openshift-default-6c5c978fc5-xs68s"
Oct 30 22:15:06.506736 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:06.480620    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ebf683b5-5605-4790-813e-3f06b8f8c38f-istio-podinfo\") pod \"e2e-ait-72b5472b-openshift-default-6c5c978fc5-xs68s\" (UID: \"ebf683b5-5605-4790-813e-3f06b8f8c38f\") " pod="openshift-ingress/e2e-ait-72b5472b-openshift-default-6c5c978fc5-xs68s"
Oct 30 22:15:06.506736 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:06.480731    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ebf683b5-5605-4790-813e-3f06b8f8c38f-istio-token\") pod \"e2e-ait-72b5472b-openshift-default-6c5c978fc5-xs68s\" (UID: \"ebf683b5-5605-4790-813e-3f06b8f8c38f\") " pod="openshift-ingress/e2e-ait-72b5472b-openshift-default-6c5c978fc5-xs68s"
Oct 30 22:15:06.506736 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:06.480881    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ebf683b5-5605-4790-813e-3f06b8f8c38f-istio-data\") pod \"e2e-ait-72b5472b-openshift-default-6c5c978fc5-xs68s\" (UID: \"ebf683b5-5605-4790-813e-3f06b8f8c38f\") " pod="openshift-ingress/e2e-ait-72b5472b-openshift-default-6c5c978fc5-xs68s"
Oct 30 22:15:06.506736 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:06.480954    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ebf683b5-5605-4790-813e-3f06b8f8c38f-workload-socket\") pod \"e2e-ait-72b5472b-openshift-default-6c5c978fc5-xs68s\" (UID: \"ebf683b5-5605-4790-813e-3f06b8f8c38f\") " pod="openshift-ingress/e2e-ait-72b5472b-openshift-default-6c5c978fc5-xs68s"
Oct 30 22:15:06.506736 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:06.481098    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ebf683b5-5605-4790-813e-3f06b8f8c38f-istio-envoy\") pod \"e2e-ait-72b5472b-openshift-default-6c5c978fc5-xs68s\" (UID: \"ebf683b5-5605-4790-813e-3f06b8f8c38f\") " pod="openshift-ingress/e2e-ait-72b5472b-openshift-default-6c5c978fc5-xs68s"
Oct 30 22:15:06.506736 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:06.481137    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ebf683b5-5605-4790-813e-3f06b8f8c38f-credential-socket\") pod \"e2e-ait-72b5472b-openshift-default-6c5c978fc5-xs68s\" (UID: \"ebf683b5-5605-4790-813e-3f06b8f8c38f\") " pod="openshift-ingress/e2e-ait-72b5472b-openshift-default-6c5c978fc5-xs68s"
Oct 30 22:15:06.506736 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:06.481215    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ebf683b5-5605-4790-813e-3f06b8f8c38f-workload-certs\") pod \"e2e-ait-72b5472b-openshift-default-6c5c978fc5-xs68s\" (UID: \"ebf683b5-5605-4790-813e-3f06b8f8c38f\") " pod="openshift-ingress/e2e-ait-72b5472b-openshift-default-6c5c978fc5-xs68s"
Oct 30 22:15:06.506736 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:06.482414    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ebf683b5-5605-4790-813e-3f06b8f8c38f-workload-certs\") pod \"e2e-ait-72b5472b-openshift-default-6c5c978fc5-xs68s\" (UID: \"ebf683b5-5605-4790-813e-3f06b8f8c38f\") " pod="openshift-ingress/e2e-ait-72b5472b-openshift-default-6c5c978fc5-xs68s"
Oct 30 22:15:06.506736 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:06.495012    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ebf683b5-5605-4790-813e-3f06b8f8c38f-istio-data\") pod \"e2e-ait-72b5472b-openshift-default-6c5c978fc5-xs68s\" (UID: \"ebf683b5-5605-4790-813e-3f06b8f8c38f\") " pod="openshift-ingress/e2e-ait-72b5472b-openshift-default-6c5c978fc5-xs68s"
Oct 30 22:15:06.506736 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:06.505360    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ebf683b5-5605-4790-813e-3f06b8f8c38f-istiod-ca-cert\") pod \"e2e-ait-72b5472b-openshift-default-6c5c978fc5-xs68s\" (UID: \"ebf683b5-5605-4790-813e-3f06b8f8c38f\") " pod="openshift-ingress/e2e-ait-72b5472b-openshift-default-6c5c978fc5-xs68s"
Oct 30 22:15:06.521680 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:06.509523    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ebf683b5-5605-4790-813e-3f06b8f8c38f-workload-socket\") pod \"e2e-ait-72b5472b-openshift-default-6c5c978fc5-xs68s\" (UID: \"ebf683b5-5605-4790-813e-3f06b8f8c38f\") " pod="openshift-ingress/e2e-ait-72b5472b-openshift-default-6c5c978fc5-xs68s"
Oct 30 22:15:06.543837 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:06.524798    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ebf683b5-5605-4790-813e-3f06b8f8c38f-credential-socket\") pod \"e2e-ait-72b5472b-openshift-default-6c5c978fc5-xs68s\" (UID: \"ebf683b5-5605-4790-813e-3f06b8f8c38f\") " pod="openshift-ingress/e2e-ait-72b5472b-openshift-default-6c5c978fc5-xs68s"
Oct 30 22:15:06.777727 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:06.770252    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:15:06.782645 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:06.782600    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ebf683b5-5605-4790-813e-3f06b8f8c38f-istio-token\") pod \"e2e-ait-72b5472b-openshift-default-6c5c978fc5-xs68s\" (UID: \"ebf683b5-5605-4790-813e-3f06b8f8c38f\") " pod="openshift-ingress/e2e-ait-72b5472b-openshift-default-6c5c978fc5-xs68s"
Oct 30 22:15:06.788699 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:06.788636    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ebf683b5-5605-4790-813e-3f06b8f8c38f-istio-envoy\") pod \"e2e-ait-72b5472b-openshift-default-6c5c978fc5-xs68s\" (UID: \"ebf683b5-5605-4790-813e-3f06b8f8c38f\") " pod="openshift-ingress/e2e-ait-72b5472b-openshift-default-6c5c978fc5-xs68s"
Oct 30 22:15:06.789748 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:06.789630    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ebf683b5-5605-4790-813e-3f06b8f8c38f-istio-podinfo\") pod \"e2e-ait-72b5472b-openshift-default-6c5c978fc5-xs68s\" (UID: \"ebf683b5-5605-4790-813e-3f06b8f8c38f\") " pod="openshift-ingress/e2e-ait-72b5472b-openshift-default-6c5c978fc5-xs68s"
Oct 30 22:15:06.792708 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:06.792620    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jml4m\" (UniqueName: \"kubernetes.io/projected/ebf683b5-5605-4790-813e-3f06b8f8c38f-kube-api-access-jml4m\") pod \"e2e-ait-72b5472b-openshift-default-6c5c978fc5-xs68s\" (UID: \"ebf683b5-5605-4790-813e-3f06b8f8c38f\") " pod="openshift-ingress/e2e-ait-72b5472b-openshift-default-6c5c978fc5-xs68s"
Oct 30 22:15:06.978286 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:06.978097    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:15:07.072050 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:07.071966    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-72b5472b-openshift-default-6c5c978fc5-xs68s"
Oct 30 22:15:07.172641 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:07.166057    5510 patch_prober.go:29] interesting pod/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-g64nz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused" start-of-body=
Oct 30 22:15:07.172641 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:07.166119    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-g64nz" podUID="71b34137-bc68-4228-a7e0-30fda97183aa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused"
Oct 30 22:15:07.395257 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:07.394865    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:15:07.714677 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:07.706923    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-72b5472b-openshift-default-6c5c978fc5-xs68s"]
Oct 30 22:15:07.716718 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:07.716683    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:15:07.728810 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:07.728032    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:15:07.728810 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:07.728125    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:15:07.728810 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:07.728166    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:15:08.105686 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:08.104637    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-72b5472b-openshift-default-6c5c978fc5-xs68s" event={"ID":"ebf683b5-5605-4790-813e-3f06b8f8c38f","Type":"ContainerStarted","Data":"0e581049efefcb2f4e1c0c85388ccfce3199fe27e9e1336b26e6ba65e0e0c000"}
Oct 30 22:15:08.105686 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:08.104736    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-72b5472b-openshift-default-6c5c978fc5-xs68s" event={"ID":"ebf683b5-5605-4790-813e-3f06b8f8c38f","Type":"ContainerStarted","Data":"0da2a89ce6767841f3a34e17f23c0fa62dc660410deaf724d5dc82e0c8c8c4c9"}
Oct 30 22:15:08.193665 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:08.193607    5510 patch_prober.go:29] interesting pod/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-g64nz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused" start-of-body=
Oct 30 22:15:08.194211 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:08.193706    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-g64nz" podUID="71b34137-bc68-4228-a7e0-30fda97183aa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused"
Oct 30 22:15:08.276597 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:08.276499    5510 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-72b5472b-openshift-default-6c5c978fc5-xs68s" podStartSLOduration=2.276479759 podStartE2EDuration="2.276479759s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:15:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:15:08.252829667 +0000 UTC m=+1488.813639669" watchObservedRunningTime="2026-10-30 22:15:08.276479759 +0000 UTC m=+1488.837289765"
Oct 30 22:15:08.612798 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:15:08.612754    5510 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6gjbf" containerName="extractor"
Oct 30 22:15:09.084486 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:09.084453    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-72b5472b-openshift-default-6c5c978fc5-xs68s"
Oct 30 22:15:09.090598 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:09.090572    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-72b5472b-openshift-default-6c5c978fc5-xs68s"
Oct 30 22:15:09.163678 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:09.162161    5510 patch_prober.go:29] interesting pod/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-g64nz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused" start-of-body=
Oct 30 22:15:09.163678 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:09.162223    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-g64nz" podUID="71b34137-bc68-4228-a7e0-30fda97183aa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused"
Oct 30 22:15:09.177686 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:09.171169    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-72b5472b-openshift-default-6c5c978fc5-xs68s"
Oct 30 22:15:09.235684 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:09.226144    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-72b5472b-openshift-default-6c5c978fc5-xs68s"
Oct 30 22:15:09.275970 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:09.275879    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:15:10.072823 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:10.071274    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:15:10.176135 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:10.176003    5510 patch_prober.go:29] interesting pod/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-g64nz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused" start-of-body=
Oct 30 22:15:10.176541 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:10.176506    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-g64nz" podUID="71b34137-bc68-4228-a7e0-30fda97183aa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused"
Oct 30 22:15:10.535394 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:10.535339    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-fb0b93/distinct-w6-fb0b93-kserve-58db6b6757-6mww8" podUID="8ee6579b-acea-4492-862b-9883f70d00c9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:15:10.555508 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:10.548262    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-bc714d/distinct-w2-bc714d-kserve-66cfb968dd-jvgqj" podUID="9e9b11b8-1cc9-4fc8-8097-1afcffa8c422" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": context deadline exceeded"
Oct 30 22:15:10.741690 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:10.727808    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:15:10.824992 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:10.824893    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-779f32/distinct-w0-779f32-kserve-6dcbc948c7-2vzdf" podUID="e17cedd4-7d48-42b3-b127-b6192c1b3795" containerName="main" probeResult="failure" output="Get \"https://10.133.0.65:8000/ready\": context deadline exceeded"
Oct 30 22:15:11.165732 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:11.164917    5510 patch_prober.go:29] interesting pod/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-g64nz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused" start-of-body=
Oct 30 22:15:11.165732 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:11.164980    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-g64nz" podUID="71b34137-bc68-4228-a7e0-30fda97183aa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused"
Oct 30 22:15:11.190687 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:11.182937    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:15:11.514009 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:11.513899    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-7bb1c2/distinct-2-w3-7bb1c2-kserve-c549df844-zt595" podUID="1916e87c-a1a4-4967-8640-8bbcd9d12903" containerName="main" probeResult="failure" output="Get \"https://10.133.0.73:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:15:11.544789 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:11.544754    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:15:11.553460 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:11.553417    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-bc714d/distinct-w2-bc714d-kserve-66cfb968dd-jvgqj" podUID="9e9b11b8-1cc9-4fc8-8097-1afcffa8c422" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:15:11.605905 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:11.605849    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-fb0b93/premium-w6-fb0b93-kserve-5cb9948c54-w7pnk" podUID="290ae7a2-de37-4af3-82ae-a8e855b83bb9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:15:11.607152 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:11.607069    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-fb0b93/distinct-w6-fb0b93-kserve-58db6b6757-6mww8" podUID="8ee6579b-acea-4492-862b-9883f70d00c9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.59:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:15:11.660177 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:11.660137    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-bc714d/distinct-w2-bc714d-kserve-66cfb968dd-jvgqj"
Oct 30 22:15:11.667160 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:11.667132    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-fb0b93/distinct-w6-fb0b93-kserve-58db6b6757-6mww8"
Oct 30 22:15:11.851678 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:11.846833    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-779f32/distinct-w0-779f32-kserve-6dcbc948c7-2vzdf" podUID="e17cedd4-7d48-42b3-b127-b6192c1b3795" containerName="main" probeResult="failure" output="Get \"https://10.133.0.65:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:15:11.852464 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:11.852436    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-779f32/distinct-w0-779f32-kserve-6dcbc948c7-2vzdf"
Oct 30 22:15:11.932724 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:11.899220    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-779f32/distinct-2-w0-779f32-kserve-857ffdf778-gk2h9" podUID="ef500cdd-cc52-45b0-8dd7-1896897acc46" containerName="main" probeResult="failure" output="Get \"https://10.133.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:15:11.995682 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:11.970133    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:15:12.213737 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:12.173876    5510 patch_prober.go:29] interesting pod/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-g64nz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused" start-of-body=
Oct 30 22:15:12.213737 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:12.173937    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-g64nz" podUID="71b34137-bc68-4228-a7e0-30fda97183aa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused"
Oct 30 22:15:12.281591 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:12.281554    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:15:12.525741 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:12.516522    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-7bb1c2/distinct-2-w3-7bb1c2-kserve-c549df844-zt595" podUID="1916e87c-a1a4-4967-8640-8bbcd9d12903" containerName="main" probeResult="failure" output="Get \"https://10.133.0.73:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:15:12.526796 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:12.526766    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-7bb1c2/distinct-2-w3-7bb1c2-kserve-c549df844-zt595"
Oct 30 22:15:12.581945 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:12.581900    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-bc714d/distinct-w2-bc714d-kserve-66cfb968dd-jvgqj" podUID="9e9b11b8-1cc9-4fc8-8097-1afcffa8c422" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": context deadline exceeded"
Oct 30 22:15:12.608806 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:12.606937    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-fb0b93/premium-w6-fb0b93-kserve-5cb9948c54-w7pnk" podUID="290ae7a2-de37-4af3-82ae-a8e855b83bb9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": context deadline exceeded"
Oct 30 22:15:12.609386 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:12.608296    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-fb0b93/distinct-w6-fb0b93-kserve-58db6b6757-6mww8" podUID="8ee6579b-acea-4492-862b-9883f70d00c9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:15:12.609925 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:12.609906    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-fb0b93/premium-w6-fb0b93-kserve-5cb9948c54-w7pnk"
Oct 30 22:15:12.819850 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:12.819740    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-c9dbe6/distinct-w5-c9dbe6-kserve-7599fd86d9-bdhxd" podUID="e903db90-f2cd-4244-b00b-f99b21873f6e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:15:12.851080 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:12.851037    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-779f32/distinct-w0-779f32-kserve-6dcbc948c7-2vzdf" podUID="e17cedd4-7d48-42b3-b127-b6192c1b3795" containerName="main" probeResult="failure" output="Get \"https://10.133.0.65:8000/ready\": context deadline exceeded"
Oct 30 22:15:12.900800 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:12.900753    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-779f32/distinct-2-w0-779f32-kserve-857ffdf778-gk2h9" podUID="ef500cdd-cc52-45b0-8dd7-1896897acc46" containerName="main" probeResult="failure" output="Get \"https://10.133.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:15:12.901464 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:12.901437    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-779f32/distinct-2-w0-779f32-kserve-857ffdf778-gk2h9"
Oct 30 22:15:13.349681 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:13.348970    5510 patch_prober.go:29] interesting pod/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-g64nz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused" start-of-body=
Oct 30 22:15:13.349681 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:13.349047    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-g64nz" podUID="71b34137-bc68-4228-a7e0-30fda97183aa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused"
Oct 30 22:15:13.537140 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:13.537094    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-7bb1c2/distinct-2-w3-7bb1c2-kserve-c549df844-zt595" podUID="1916e87c-a1a4-4967-8640-8bbcd9d12903" containerName="main" probeResult="failure" output="Get \"https://10.133.0.73:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:15:13.650767 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:13.647272    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-fb0b93/premium-w6-fb0b93-kserve-5cb9948c54-w7pnk" podUID="290ae7a2-de37-4af3-82ae-a8e855b83bb9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:15:13.650767 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:13.648023    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-fb0b93/distinct-w6-fb0b93-kserve-58db6b6757-6mww8" podUID="8ee6579b-acea-4492-862b-9883f70d00c9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:15:13.650767 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:13.648331    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-bc714d/distinct-w2-bc714d-kserve-66cfb968dd-jvgqj" podUID="9e9b11b8-1cc9-4fc8-8097-1afcffa8c422" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:15:13.860682 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:13.859693    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-c9dbe6/distinct-w5-c9dbe6-kserve-7599fd86d9-bdhxd" podUID="e903db90-f2cd-4244-b00b-f99b21873f6e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:15:13.953744 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:13.945777    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-779f32/distinct-2-w0-779f32-kserve-857ffdf778-gk2h9" podUID="ef500cdd-cc52-45b0-8dd7-1896897acc46" containerName="main" probeResult="failure" output="Get \"https://10.133.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:15:13.965681 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:13.960457    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-779f32/distinct-w0-779f32-kserve-6dcbc948c7-2vzdf" podUID="e17cedd4-7d48-42b3-b127-b6192c1b3795" containerName="main" probeResult="failure" output="Get \"https://10.133.0.65:8000/ready\": context deadline exceeded"
Oct 30 22:15:14.022696 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:14.003119    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-c9dbe6/distinct-w5-c9dbe6-kserve-7599fd86d9-bdhxd"
Oct 30 22:15:14.196456 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:14.196409    5510 patch_prober.go:29] interesting pod/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-g64nz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused" start-of-body=
Oct 30 22:15:14.196726 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:14.196698    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-g64nz" podUID="71b34137-bc68-4228-a7e0-30fda97183aa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused"
Oct 30 22:15:14.619680 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:14.565546    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-7bb1c2/distinct-2-w3-7bb1c2-kserve-c549df844-zt595" podUID="1916e87c-a1a4-4967-8640-8bbcd9d12903" containerName="main" probeResult="failure" output="Get \"https://10.133.0.73:8000/ready\": context deadline exceeded"
Oct 30 22:15:15.043750 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:14.663062    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-bc714d/distinct-w2-bc714d-kserve-66cfb968dd-jvgqj" podUID="9e9b11b8-1cc9-4fc8-8097-1afcffa8c422" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:15:15.043750 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:14.675744    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-fb0b93/premium-w6-fb0b93-kserve-5cb9948c54-w7pnk" podUID="290ae7a2-de37-4af3-82ae-a8e855b83bb9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": context deadline exceeded"
Oct 30 22:15:15.043750 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:14.687728    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-fb0b93/distinct-w6-fb0b93-kserve-58db6b6757-6mww8" podUID="8ee6579b-acea-4492-862b-9883f70d00c9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.59:8000/ready\": context deadline exceeded"
Oct 30 22:15:15.043750 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:14.883178    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-c9dbe6/distinct-w5-c9dbe6-kserve-7599fd86d9-bdhxd" podUID="e903db90-f2cd-4244-b00b-f99b21873f6e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.67:8000/ready\": context deadline exceeded"
Oct 30 22:15:15.043750 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:14.967788    5510 patch_prober.go:29] interesting pod/e2e-worker-w2-bc714d-openshift-default-67558b95cf-v4cxn container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.81:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 22:15:15.043750 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:15.011230    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-v4cxn" podUID="5542b62f-bc73-47d7-8615-180f43324911" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.81:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:15:15.043750 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:15.006890    5510 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-7bb1c2/distinct-2-w3-7bb1c2-kserve-c549df844-zt595" podUID="1916e87c-a1a4-4967-8640-8bbcd9d12903" containerName="main" probeResult="failure" output="Get \"https://10.133.0.73:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:15:15.043750 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:15.011818    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-779f32/distinct-w0-779f32-kserve-6dcbc948c7-2vzdf" podUID="e17cedd4-7d48-42b3-b127-b6192c1b3795" containerName="main" probeResult="failure" output="Get \"https://10.133.0.65:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:15:15.043750 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:15.013985    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-779f32/distinct-2-w0-779f32-kserve-857ffdf778-gk2h9" podUID="ef500cdd-cc52-45b0-8dd7-1896897acc46" containerName="main" probeResult="failure" output="Get \"https://10.133.0.74:8000/ready\": context deadline exceeded"
Oct 30 22:15:15.138688 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:15.113748    5510 patch_prober.go:29] interesting pod/e2e-worker-w2-bc714d-openshift-default-67558b95cf-f969x container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.80:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 22:15:15.138688 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:15.113828    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-f969x" podUID="b0d37b4a-b986-438a-a811-9037505f0e6a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.80:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:15:15.194131 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:15.161530    5510 patch_prober.go:29] interesting pod/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-g64nz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused" start-of-body=
Oct 30 22:15:15.194131 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:15.161598    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-g64nz" podUID="71b34137-bc68-4228-a7e0-30fda97183aa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused"
Oct 30 22:15:15.912682 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:15.911994    5510 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-bc714d/distinct-w2-bc714d-kserve-66cfb968dd-jvgqj" podUID="9e9b11b8-1cc9-4fc8-8097-1afcffa8c422" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:15:16.005688 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:15.989210    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:15:16.019026 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:16.018969    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-779f32/distinct-w0-779f32-kserve-6dcbc948c7-2vzdf" podUID="e17cedd4-7d48-42b3-b127-b6192c1b3795" containerName="main" probeResult="failure" output="Get \"https://10.133.0.65:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:15:16.019448 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:16.018893    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-779f32/distinct-2-w0-779f32-kserve-857ffdf778-gk2h9" podUID="ef500cdd-cc52-45b0-8dd7-1896897acc46" containerName="main" probeResult="failure" output="Get \"https://10.133.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:15:16.025995 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:16.020379    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-c9dbe6/distinct-w5-c9dbe6-kserve-7599fd86d9-bdhxd" podUID="e903db90-f2cd-4244-b00b-f99b21873f6e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:15:16.025995 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:16.020693    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-7bb1c2/distinct-2-w3-7bb1c2-kserve-c549df844-zt595" podUID="1916e87c-a1a4-4967-8640-8bbcd9d12903" containerName="main" probeResult="failure" output="Get \"https://10.133.0.73:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:15:16.046902 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:16.046165    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-bc714d/distinct-w2-bc714d-kserve-66cfb968dd-jvgqj" podUID="9e9b11b8-1cc9-4fc8-8097-1afcffa8c422" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:15:16.077708 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:16.047590    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-fb0b93/premium-w6-fb0b93-kserve-5cb9948c54-w7pnk" podUID="290ae7a2-de37-4af3-82ae-a8e855b83bb9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": context deadline exceeded"
Oct 30 22:15:16.077708 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:16.048149    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-fb0b93/distinct-w6-fb0b93-kserve-58db6b6757-6mww8" podUID="8ee6579b-acea-4492-862b-9883f70d00c9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:15:16.292797 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:16.292431    5510 patch_prober.go:29] interesting pod/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-g64nz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused" start-of-body=
Oct 30 22:15:16.292797 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:16.292495    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-g64nz" podUID="71b34137-bc68-4228-a7e0-30fda97183aa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused"
Oct 30 22:15:16.294444 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:16.294409    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:15:16.516912 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:16.516872    5510 patch_prober.go:29] interesting pod/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6fvq6 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 22:15:16.517189 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:16.517140    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6fvq6" podUID="5ff054ea-b27c-4c64-be2d-c645c0ca3d24" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:15:16.521833 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:16.521801    5510 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-ff716ef6-f89c8bd9c-mhgqb"]
Oct 30 22:15:16.669778 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:16.665597    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-ff716ef6-f89c8bd9c-mhgqb"
Oct 30 22:15:16.767182 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:16.767149    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-ff716ef6-dockercfg-pp657\""
Oct 30 22:15:16.812686 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:16.802145    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l4jkk\" (UniqueName: \"kubernetes.io/projected/46927be5-d550-469c-baa4-f9f124952a84-kube-api-access-l4jkk\") pod \"payload-processing-e2e-ait-ff716ef6-f89c8bd9c-mhgqb\" (UID: \"46927be5-d550-469c-baa4-f9f124952a84\") " pod="openshift-ingress/payload-processing-e2e-ait-ff716ef6-f89c8bd9c-mhgqb"
Oct 30 22:15:16.812686 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:16.802295    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/46927be5-d550-469c-baa4-f9f124952a84-plugins-config-volume\") pod \"payload-processing-e2e-ait-ff716ef6-f89c8bd9c-mhgqb\" (UID: \"46927be5-d550-469c-baa4-f9f124952a84\") " pod="openshift-ingress/payload-processing-e2e-ait-ff716ef6-f89c8bd9c-mhgqb"
Oct 30 22:15:16.857997 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:16.823891    5510 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-ff716ef6\""
Oct 30 22:15:16.904775 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:16.882606    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-ff716ef6-f89c8bd9c-mhgqb"]
Oct 30 22:15:17.058137 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:17.046363    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-bc714d/distinct-w2-bc714d-kserve-66cfb968dd-jvgqj" podUID="9e9b11b8-1cc9-4fc8-8097-1afcffa8c422" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:15:17.058137 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:17.046704    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-779f32/distinct-w0-779f32-kserve-6dcbc948c7-2vzdf" podUID="e17cedd4-7d48-42b3-b127-b6192c1b3795" containerName="main" probeResult="failure" output="Get \"https://10.133.0.65:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:15:17.058137 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:17.046959    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-779f32/distinct-2-w0-779f32-kserve-857ffdf778-gk2h9" podUID="ef500cdd-cc52-45b0-8dd7-1896897acc46" containerName="main" probeResult="failure" output="Get \"https://10.133.0.74:8000/ready\": context deadline exceeded"
Oct 30 22:15:17.058137 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:17.047195    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-c9dbe6/distinct-w5-c9dbe6-kserve-7599fd86d9-bdhxd" podUID="e903db90-f2cd-4244-b00b-f99b21873f6e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.67:8000/ready\": context deadline exceeded"
Oct 30 22:15:17.058137 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:17.047425    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-7bb1c2/distinct-2-w3-7bb1c2-kserve-c549df844-zt595" podUID="1916e87c-a1a4-4967-8640-8bbcd9d12903" containerName="main" probeResult="failure" output="Get \"https://10.133.0.73:8000/ready\": context deadline exceeded"
Oct 30 22:15:17.083743 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:17.061019    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l4jkk\" (UniqueName: \"kubernetes.io/projected/46927be5-d550-469c-baa4-f9f124952a84-kube-api-access-l4jkk\") pod \"payload-processing-e2e-ait-ff716ef6-f89c8bd9c-mhgqb\" (UID: \"46927be5-d550-469c-baa4-f9f124952a84\") " pod="openshift-ingress/payload-processing-e2e-ait-ff716ef6-f89c8bd9c-mhgqb"
Oct 30 22:15:17.083743 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:17.061156    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/46927be5-d550-469c-baa4-f9f124952a84-plugins-config-volume\") pod \"payload-processing-e2e-ait-ff716ef6-f89c8bd9c-mhgqb\" (UID: \"46927be5-d550-469c-baa4-f9f124952a84\") " pod="openshift-ingress/payload-processing-e2e-ait-ff716ef6-f89c8bd9c-mhgqb"
Oct 30 22:15:17.083743 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:17.063735    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-fb0b93/distinct-w6-fb0b93-kserve-58db6b6757-6mww8" podUID="8ee6579b-acea-4492-862b-9883f70d00c9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:15:17.083743 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:17.064115    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-fb0b93/premium-w6-fb0b93-kserve-5cb9948c54-w7pnk" podUID="290ae7a2-de37-4af3-82ae-a8e855b83bb9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": context deadline exceeded"
Oct 30 22:15:17.083743 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:17.074440    5510 patch_prober.go:29] interesting pod/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-ccsc2 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 22:15:17.083743 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:17.074500    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-ccsc2" podUID="caef4e8d-7dcc-455a-94a2-4bb341662a88" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:15:17.083743 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:17.076515    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/46927be5-d550-469c-baa4-f9f124952a84-plugins-config-volume\") pod \"payload-processing-e2e-ait-ff716ef6-f89c8bd9c-mhgqb\" (UID: \"46927be5-d550-469c-baa4-f9f124952a84\") " pod="openshift-ingress/payload-processing-e2e-ait-ff716ef6-f89c8bd9c-mhgqb"
Oct 30 22:15:17.186617 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:17.109534    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-67184f2a-6fc5c46555-n47rx"
Oct 30 22:15:17.493262 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:17.489428    5510 patch_prober.go:29] interesting pod/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-g64nz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused" start-of-body=
Oct 30 22:15:17.493262 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:17.489490    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-g64nz" podUID="71b34137-bc68-4228-a7e0-30fda97183aa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused"
Oct 30 22:15:17.903472 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:17.903420    5510 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w6-fb0b93/distinct-w6-fb0b93-kserve-58db6b6757-6mww8" podUID="8ee6579b-acea-4492-862b-9883f70d00c9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.59:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:15:18.084684 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:18.055762    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-7bb1c2/distinct-2-w3-7bb1c2-kserve-c549df844-zt595" podUID="1916e87c-a1a4-4967-8640-8bbcd9d12903" containerName="main" probeResult="failure" output="Get \"https://10.133.0.73:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:15:18.225138 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:18.056767    5510 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w6-fb0b93/premium-w6-fb0b93-kserve-5cb9948c54-w7pnk" podUID="290ae7a2-de37-4af3-82ae-a8e855b83bb9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:15:18.225572 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:18.056807    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-779f32/distinct-2-w0-779f32-kserve-857ffdf778-gk2h9" podUID="ef500cdd-cc52-45b0-8dd7-1896897acc46" containerName="main" probeResult="failure" output="Get \"https://10.133.0.74:8000/ready\": context deadline exceeded"
Oct 30 22:15:18.225913 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:18.056833    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-bc714d/distinct-w2-bc714d-kserve-66cfb968dd-jvgqj" podUID="9e9b11b8-1cc9-4fc8-8097-1afcffa8c422" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:15:18.226259 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:18.056852    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-779f32/distinct-w0-779f32-kserve-6dcbc948c7-2vzdf" podUID="e17cedd4-7d48-42b3-b127-b6192c1b3795" containerName="main" probeResult="failure" output="Get \"https://10.133.0.65:8000/ready\": context deadline exceeded"
Oct 30 22:15:18.226599 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:18.056880    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-c9dbe6/distinct-w5-c9dbe6-kserve-7599fd86d9-bdhxd" podUID="e903db90-f2cd-4244-b00b-f99b21873f6e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:15:18.226884 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:18.066884    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-fb0b93/premium-w6-fb0b93-kserve-5cb9948c54-w7pnk" podUID="290ae7a2-de37-4af3-82ae-a8e855b83bb9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:15:18.231720 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:18.066966    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-fb0b93/distinct-w6-fb0b93-kserve-58db6b6757-6mww8" podUID="8ee6579b-acea-4492-862b-9883f70d00c9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.59:8000/ready\": context deadline exceeded"
Oct 30 22:15:18.238730 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:18.085397    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-migrate-67184f2a-66ffdf88d7-wd7x9" podUID="11d70426-1070-491a-8b69-d107582114f0" containerName="payload-processing" probeResult="failure" output="timeout: failed to connect service \"10.133.0.88:50052\" within 1s: context deadline exceeded"
Oct 30 22:15:18.238981 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:18.100419    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l4jkk\" (UniqueName: \"kubernetes.io/projected/46927be5-d550-469c-baa4-f9f124952a84-kube-api-access-l4jkk\") pod \"payload-processing-e2e-ait-ff716ef6-f89c8bd9c-mhgqb\" (UID: \"46927be5-d550-469c-baa4-f9f124952a84\") " pod="openshift-ingress/payload-processing-e2e-ait-ff716ef6-f89c8bd9c-mhgqb"
Oct 30 22:15:18.243973 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:18.131397    5510 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-ff716ef6-5574d46599-svwhz"]
Oct 30 22:15:18.276886 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:18.275086    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-g64nz"
Oct 30 22:15:18.276886 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:18.200872    5510 patch_prober.go:29] interesting pod/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-g64nz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused" start-of-body=
Oct 30 22:15:18.276886 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:18.275737    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-g64nz" podUID="71b34137-bc68-4228-a7e0-30fda97183aa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused"
Oct 30 22:15:18.276886 ip-10-0-136-210 kubenswrapper[5510]: W1030 22:15:18.203785    5510 logging.go:55] [core] [Channel #1079 SubChannel #1080]grpc: addrConn.createTransport failed to connect to {Addr: "10.133.0.88:50052", ServerName: "10.133.0.88:50052", BalancerAttributes: {"<%!p(pickfirstleaf.managedByPickfirstKeyType={})>": "<%!p(bool=true)>" }}. Err: connection error: desc = "transport: failed to write client preface: write tcp 10.133.0.2:55820->10.133.0.88:50052: use of closed network connection"
Oct 30 22:15:18.276886 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:18.203134    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:15:18.296773 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:18.296734    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-ff716ef6-5574d46599-svwhz"
Oct 30 22:15:18.341690 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:18.332495    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-ff716ef6-f89c8bd9c-mhgqb"
Oct 30 22:15:18.699678 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:18.652258    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dmwc6\" (UniqueName: \"kubernetes.io/projected/1a1e6300-d0d1-41c7-b089-b7f27eafe2bb-kube-api-access-dmwc6\") pod \"payload-pre-processing-e2e-ait-ff716ef6-5574d46599-svwhz\" (UID: \"1a1e6300-d0d1-41c7-b089-b7f27eafe2bb\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-ff716ef6-5574d46599-svwhz"
Oct 30 22:15:18.699678 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:18.652324    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1a1e6300-d0d1-41c7-b089-b7f27eafe2bb-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-ff716ef6-5574d46599-svwhz\" (UID: \"1a1e6300-d0d1-41c7-b089-b7f27eafe2bb\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-ff716ef6-5574d46599-svwhz"
Oct 30 22:15:18.702408 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:18.702365    5510 prober.go:122] "Probe failed" probeType="Liveness" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-zz2gk" podUID="cb6133b6-d8dc-4687-9974-68874f9a34d4" containerName="main" probeResult="failure" output="Get \"https://10.133.0.34:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:15:18.742724 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:18.735745    5510 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-779f32/distinct-2-w0-779f32-kserve-857ffdf778-gk2h9" podUID="ef500cdd-cc52-45b0-8dd7-1896897acc46" containerName="main" probeResult="failure" output="Get \"https://10.133.0.74:8000/health\": context deadline exceeded"
Oct 30 22:15:18.782708 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:18.761752    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:15:18.812740 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:18.800092    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dmwc6\" (UniqueName: \"kubernetes.io/projected/1a1e6300-d0d1-41c7-b089-b7f27eafe2bb-kube-api-access-dmwc6\") pod \"payload-pre-processing-e2e-ait-ff716ef6-5574d46599-svwhz\" (UID: \"1a1e6300-d0d1-41c7-b089-b7f27eafe2bb\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-ff716ef6-5574d46599-svwhz"
Oct 30 22:15:18.812740 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:18.800189    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1a1e6300-d0d1-41c7-b089-b7f27eafe2bb-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-ff716ef6-5574d46599-svwhz\" (UID: \"1a1e6300-d0d1-41c7-b089-b7f27eafe2bb\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-ff716ef6-5574d46599-svwhz"
Oct 30 22:15:18.907679 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:18.838722    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1a1e6300-d0d1-41c7-b089-b7f27eafe2bb-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-ff716ef6-5574d46599-svwhz\" (UID: \"1a1e6300-d0d1-41c7-b089-b7f27eafe2bb\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-ff716ef6-5574d46599-svwhz"
Oct 30 22:15:18.974721 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:18.970481    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dmwc6\" (UniqueName: \"kubernetes.io/projected/1a1e6300-d0d1-41c7-b089-b7f27eafe2bb-kube-api-access-dmwc6\") pod \"payload-pre-processing-e2e-ait-ff716ef6-5574d46599-svwhz\" (UID: \"1a1e6300-d0d1-41c7-b089-b7f27eafe2bb\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-ff716ef6-5574d46599-svwhz"
Oct 30 22:15:19.072737 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:19.055886    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-ff716ef6-5574d46599-svwhz"
Oct 30 22:15:19.189960 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:19.177360    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-ff716ef6-5574d46599-svwhz"]
Oct 30 22:15:19.230719 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:19.198623    5510 patch_prober.go:29] interesting pod/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-g64nz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused" start-of-body=
Oct 30 22:15:19.231360 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:19.231315    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-g64nz" podUID="71b34137-bc68-4228-a7e0-30fda97183aa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused"
Oct 30 22:15:19.231870 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:19.231840    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-fb0b93/premium-w6-fb0b93-kserve-5cb9948c54-w7pnk" podUID="290ae7a2-de37-4af3-82ae-a8e855b83bb9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:15:19.233504 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:19.233476    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-fb0b93/distinct-w6-fb0b93-kserve-58db6b6757-6mww8" podUID="8ee6579b-acea-4492-862b-9883f70d00c9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:15:19.234670 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:19.232018    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-c9dbe6/distinct-w5-c9dbe6-kserve-7599fd86d9-bdhxd" podUID="e903db90-f2cd-4244-b00b-f99b21873f6e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.67:8000/ready\": context deadline exceeded"
Oct 30 22:15:19.276087 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:19.252566    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-bc714d/distinct-w2-bc714d-kserve-66cfb968dd-jvgqj" podUID="9e9b11b8-1cc9-4fc8-8097-1afcffa8c422" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:15:19.664958 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:19.276824    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-779f32/distinct-w0-779f32-kserve-6dcbc948c7-2vzdf" podUID="e17cedd4-7d48-42b3-b127-b6192c1b3795" containerName="main" probeResult="failure" output="Get \"https://10.133.0.65:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:15:19.664958 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:19.276890    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-779f32/distinct-2-w0-779f32-kserve-857ffdf778-gk2h9" podUID="ef500cdd-cc52-45b0-8dd7-1896897acc46" containerName="main" probeResult="failure" output="Get \"https://10.133.0.74:8000/ready\": context deadline exceeded"
Oct 30 22:15:19.664958 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:19.276910    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-7bb1c2/distinct-2-w3-7bb1c2-kserve-c549df844-zt595" podUID="1916e87c-a1a4-4967-8640-8bbcd9d12903" containerName="main" probeResult="failure" output="Get \"https://10.133.0.73:8000/ready\": context deadline exceeded"
Oct 30 22:15:19.873629 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:19.873597    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-fb0b93/distinct-w6-fb0b93-kserve-58db6b6757-6mww8"
Oct 30 22:15:19.873928 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:19.873908    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-fb0b93/premium-w6-fb0b93-kserve-5cb9948c54-w7pnk"
Oct 30 22:15:19.874088 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:19.874075    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-c9dbe6/distinct-w5-c9dbe6-kserve-7599fd86d9-bdhxd"
Oct 30 22:15:19.874195 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:19.874182    5510 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-72b5472b-5dfd8db78c-zwg2j"]
Oct 30 22:15:19.881428 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:19.881404    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-72b5472b-5dfd8db78c-zwg2j"]
Oct 30 22:15:19.881753 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:19.881735    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-7bb1c2/distinct-2-w3-7bb1c2-kserve-c549df844-zt595"
Oct 30 22:15:19.881893 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:19.881880    5510 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-72b5472b-544d6795d5-vtsjp"]
Oct 30 22:15:19.903091 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:19.887812    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-72b5472b-5dfd8db78c-zwg2j"
Oct 30 22:15:19.903091 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:19.902263    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-779f32/distinct-2-w0-779f32-kserve-857ffdf778-gk2h9"
Oct 30 22:15:19.903091 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:19.902950    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-72b5472b-544d6795d5-vtsjp"
Oct 30 22:15:19.907365 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:19.907344    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-779f32/distinct-w0-779f32-kserve-6dcbc948c7-2vzdf"
Oct 30 22:15:19.930858 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:19.930765    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-72b5472b-544d6795d5-vtsjp"]
Oct 30 22:15:19.945563 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:19.945527    5510 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-72b5472b\""
Oct 30 22:15:19.945920 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:19.945890    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-72b5472b-dockercfg-l5694\""
Oct 30 22:15:20.020683 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:20.017944    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2679ae9b-9620-4340-8701-aab7b4cc1e13-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-72b5472b-5dfd8db78c-zwg2j\" (UID: \"2679ae9b-9620-4340-8701-aab7b4cc1e13\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-72b5472b-5dfd8db78c-zwg2j"
Oct 30 22:15:20.020683 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:20.018086    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8dc39be6-e92f-4d15-afca-1035e750f9a6-plugins-config-volume\") pod \"payload-processing-e2e-ait-72b5472b-544d6795d5-vtsjp\" (UID: \"8dc39be6-e92f-4d15-afca-1035e750f9a6\") " pod="openshift-ingress/payload-processing-e2e-ait-72b5472b-544d6795d5-vtsjp"
Oct 30 22:15:20.020683 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:20.018413    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5vp5w\" (UniqueName: \"kubernetes.io/projected/8dc39be6-e92f-4d15-afca-1035e750f9a6-kube-api-access-5vp5w\") pod \"payload-processing-e2e-ait-72b5472b-544d6795d5-vtsjp\" (UID: \"8dc39be6-e92f-4d15-afca-1035e750f9a6\") " pod="openshift-ingress/payload-processing-e2e-ait-72b5472b-544d6795d5-vtsjp"
Oct 30 22:15:20.020683 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:20.018493    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8rqdr\" (UniqueName: \"kubernetes.io/projected/2679ae9b-9620-4340-8701-aab7b4cc1e13-kube-api-access-8rqdr\") pod \"payload-pre-processing-e2e-ait-72b5472b-5dfd8db78c-zwg2j\" (UID: \"2679ae9b-9620-4340-8701-aab7b4cc1e13\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-72b5472b-5dfd8db78c-zwg2j"
Oct 30 22:15:20.025543 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:20.025398    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-bc714d/distinct-w2-bc714d-kserve-66cfb968dd-jvgqj"
Oct 30 22:15:20.134521 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:20.134485    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2679ae9b-9620-4340-8701-aab7b4cc1e13-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-72b5472b-5dfd8db78c-zwg2j\" (UID: \"2679ae9b-9620-4340-8701-aab7b4cc1e13\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-72b5472b-5dfd8db78c-zwg2j"
Oct 30 22:15:20.134856 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:20.134827    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8dc39be6-e92f-4d15-afca-1035e750f9a6-plugins-config-volume\") pod \"payload-processing-e2e-ait-72b5472b-544d6795d5-vtsjp\" (UID: \"8dc39be6-e92f-4d15-afca-1035e750f9a6\") " pod="openshift-ingress/payload-processing-e2e-ait-72b5472b-544d6795d5-vtsjp"
Oct 30 22:15:20.135347 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:20.135182    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5vp5w\" (UniqueName: \"kubernetes.io/projected/8dc39be6-e92f-4d15-afca-1035e750f9a6-kube-api-access-5vp5w\") pod \"payload-processing-e2e-ait-72b5472b-544d6795d5-vtsjp\" (UID: \"8dc39be6-e92f-4d15-afca-1035e750f9a6\") " pod="openshift-ingress/payload-processing-e2e-ait-72b5472b-544d6795d5-vtsjp"
Oct 30 22:15:20.135561 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:20.135543    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8rqdr\" (UniqueName: \"kubernetes.io/projected/2679ae9b-9620-4340-8701-aab7b4cc1e13-kube-api-access-8rqdr\") pod \"payload-pre-processing-e2e-ait-72b5472b-5dfd8db78c-zwg2j\" (UID: \"2679ae9b-9620-4340-8701-aab7b4cc1e13\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-72b5472b-5dfd8db78c-zwg2j"
Oct 30 22:15:20.137751 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:20.137724    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2679ae9b-9620-4340-8701-aab7b4cc1e13-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-72b5472b-5dfd8db78c-zwg2j\" (UID: \"2679ae9b-9620-4340-8701-aab7b4cc1e13\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-72b5472b-5dfd8db78c-zwg2j"
Oct 30 22:15:20.138836 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:20.138618    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8dc39be6-e92f-4d15-afca-1035e750f9a6-plugins-config-volume\") pod \"payload-processing-e2e-ait-72b5472b-544d6795d5-vtsjp\" (UID: \"8dc39be6-e92f-4d15-afca-1035e750f9a6\") " pod="openshift-ingress/payload-processing-e2e-ait-72b5472b-544d6795d5-vtsjp"
Oct 30 22:15:20.163122 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:20.160940    5510 patch_prober.go:29] interesting pod/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-g64nz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused" start-of-body=
Oct 30 22:15:20.163122 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:20.161019    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-g64nz" podUID="71b34137-bc68-4228-a7e0-30fda97183aa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused"
Oct 30 22:15:20.209706 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:20.204175    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5vp5w\" (UniqueName: \"kubernetes.io/projected/8dc39be6-e92f-4d15-afca-1035e750f9a6-kube-api-access-5vp5w\") pod \"payload-processing-e2e-ait-72b5472b-544d6795d5-vtsjp\" (UID: \"8dc39be6-e92f-4d15-afca-1035e750f9a6\") " pod="openshift-ingress/payload-processing-e2e-ait-72b5472b-544d6795d5-vtsjp"
Oct 30 22:15:20.211378 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:20.211344    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8rqdr\" (UniqueName: \"kubernetes.io/projected/2679ae9b-9620-4340-8701-aab7b4cc1e13-kube-api-access-8rqdr\") pod \"payload-pre-processing-e2e-ait-72b5472b-5dfd8db78c-zwg2j\" (UID: \"2679ae9b-9620-4340-8701-aab7b4cc1e13\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-72b5472b-5dfd8db78c-zwg2j"
Oct 30 22:15:20.236194 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:20.236155    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-72b5472b-5dfd8db78c-zwg2j"
Oct 30 22:15:20.251221 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:20.251192    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-ff716ef6-5574d46599-svwhz"]
Oct 30 22:15:20.253539 ip-10-0-136-210 kubenswrapper[5510]: W1030 22:15:20.253472    5510 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a1e6300_d0d1_41c7_b089_b7f27eafe2bb.slice/crio-56f042d14ba82abbfefec7b2d91692ca6f8a5b77afa82194e0931b1462516757 WatchSource:0}: Error finding container 56f042d14ba82abbfefec7b2d91692ca6f8a5b77afa82194e0931b1462516757: Status 404 returned error can't find the container with id 56f042d14ba82abbfefec7b2d91692ca6f8a5b77afa82194e0931b1462516757
Oct 30 22:15:20.292515 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:20.292355    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:15:20.317970 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:20.317934    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-ff716ef6-f89c8bd9c-mhgqb"]
Oct 30 22:15:20.341931 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:20.341904    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-72b5472b-544d6795d5-vtsjp"
Oct 30 22:15:20.579124 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:20.570549    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-ff716ef6-f89c8bd9c-mhgqb" event={"ID":"46927be5-d550-469c-baa4-f9f124952a84","Type":"ContainerStarted","Data":"1b00206ea6bd65594d463e7f22118eb1fe289f986da6b6fc3db7852aec58896f"}
Oct 30 22:15:20.655064 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:20.651501    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-ff716ef6-5574d46599-svwhz" event={"ID":"1a1e6300-d0d1-41c7-b089-b7f27eafe2bb","Type":"ContainerStarted","Data":"68643faca0840d83d21ca5655dc89b59b7bd51cc57041ef16cbe04be87eccc22"}
Oct 30 22:15:20.655064 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:20.651567    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-ff716ef6-5574d46599-svwhz" event={"ID":"1a1e6300-d0d1-41c7-b089-b7f27eafe2bb","Type":"ContainerStarted","Data":"56f042d14ba82abbfefec7b2d91692ca6f8a5b77afa82194e0931b1462516757"}
Oct 30 22:15:20.655064 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:20.652485    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-ff716ef6-5574d46599-svwhz"
Oct 30 22:15:20.686014 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:20.685808    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-72b5472b-5dfd8db78c-zwg2j"]
Oct 30 22:15:20.688006 ip-10-0-136-210 kubenswrapper[5510]: W1030 22:15:20.687981    5510 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2679ae9b_9620_4340_8701_aab7b4cc1e13.slice/crio-656d1d75990d86a3f9b37633f6bac4937f97780e40e2f649d3768b64ed6c2ccd WatchSource:0}: Error finding container 656d1d75990d86a3f9b37633f6bac4937f97780e40e2f649d3768b64ed6c2ccd: Status 404 returned error can't find the container with id 656d1d75990d86a3f9b37633f6bac4937f97780e40e2f649d3768b64ed6c2ccd
Oct 30 22:15:20.964825 ip-10-0-136-210 kubenswrapper[5510]: W1030 22:15:20.964789    5510 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8dc39be6_e92f_4d15_afca_1035e750f9a6.slice/crio-01b8aa077a50c8ec8ddeebeccc812bb9b10b8f94349a343882da62bb4ae5bac0 WatchSource:0}: Error finding container 01b8aa077a50c8ec8ddeebeccc812bb9b10b8f94349a343882da62bb4ae5bac0: Status 404 returned error can't find the container with id 01b8aa077a50c8ec8ddeebeccc812bb9b10b8f94349a343882da62bb4ae5bac0
Oct 30 22:15:20.966017 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:20.965989    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-72b5472b-544d6795d5-vtsjp"]
Oct 30 22:15:21.144118 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:21.144060    5510 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-ff716ef6-5574d46599-svwhz" podStartSLOduration=5.14404635 podStartE2EDuration="5.14404635s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:15:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:15:21.14291916 +0000 UTC m=+1501.703729221" watchObservedRunningTime="2026-10-30 22:15:21.14404635 +0000 UTC m=+1501.704856354"
Oct 30 22:15:21.661828 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:21.661790    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-72b5472b-544d6795d5-vtsjp" event={"ID":"8dc39be6-e92f-4d15-afca-1035e750f9a6","Type":"ContainerStarted","Data":"b6fc1c053eac4b69f90904dade68a5afd4acfbc17bad6fa881b5e301926f8d85"}
Oct 30 22:15:21.662351 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:21.661846    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-72b5472b-544d6795d5-vtsjp" event={"ID":"8dc39be6-e92f-4d15-afca-1035e750f9a6","Type":"ContainerStarted","Data":"01b8aa077a50c8ec8ddeebeccc812bb9b10b8f94349a343882da62bb4ae5bac0"}
Oct 30 22:15:21.662351 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:21.661876    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-72b5472b-544d6795d5-vtsjp"
Oct 30 22:15:21.664068 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:21.664028    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-ff716ef6-f89c8bd9c-mhgqb" event={"ID":"46927be5-d550-469c-baa4-f9f124952a84","Type":"ContainerStarted","Data":"ea3dd327695220c042deb86ce05d2a8eb1619962e730615341b71329fb7ed09a"}
Oct 30 22:15:21.664209 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:21.664120    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-ff716ef6-f89c8bd9c-mhgqb"
Oct 30 22:15:21.666527 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:21.666502    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-72b5472b-5dfd8db78c-zwg2j" event={"ID":"2679ae9b-9620-4340-8701-aab7b4cc1e13","Type":"ContainerStarted","Data":"0f09b6a5919282acb849777ab5f6a5ffe90c65b391e19c455b58a6d33528b32c"}
Oct 30 22:15:21.666702 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:21.666543    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-72b5472b-5dfd8db78c-zwg2j" event={"ID":"2679ae9b-9620-4340-8701-aab7b4cc1e13","Type":"ContainerStarted","Data":"656d1d75990d86a3f9b37633f6bac4937f97780e40e2f649d3768b64ed6c2ccd"}
Oct 30 22:15:21.666702 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:21.666562    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-72b5472b-5dfd8db78c-zwg2j"
Oct 30 22:15:21.786173 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:21.786120    5510 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-72b5472b-544d6795d5-vtsjp" podStartSLOduration=3.786106309 podStartE2EDuration="3.786106309s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:15:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:15:21.782085735 +0000 UTC m=+1502.342895741" watchObservedRunningTime="2026-10-30 22:15:21.786106309 +0000 UTC m=+1502.346916615"
Oct 30 22:15:21.914401 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:21.914203    5510 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-ff716ef6-f89c8bd9c-mhgqb" podStartSLOduration=6.914181687 podStartE2EDuration="6.914181687s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:15:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:15:21.905165677 +0000 UTC m=+1502.465975705" watchObservedRunningTime="2026-10-30 22:15:21.914181687 +0000 UTC m=+1502.474991692"
Oct 30 22:15:22.127680 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:22.126917    5510 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-72b5472b-5dfd8db78c-zwg2j" podStartSLOduration=4.12689399 podStartE2EDuration="4.12689399s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:15:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:15:22.065616127 +0000 UTC m=+1502.626426155" watchObservedRunningTime="2026-10-30 22:15:22.12689399 +0000 UTC m=+1502.687704010"
Oct 30 22:15:22.216734 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:22.210832    5510 patch_prober.go:29] interesting pod/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-g64nz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.86:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 22:15:22.216734 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:22.210920    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-g64nz" podUID="71b34137-bc68-4228-a7e0-30fda97183aa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.86:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:15:22.963918 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:22.963884    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-g64nz"
Oct 30 22:15:22.965692 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:22.965642    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-g64nz"
Oct 30 22:15:23.900471 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:23.900423    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:15:25.482877 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:25.482766    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:15:26.476426 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:26.476385    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:15:26.983927 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:26.983897    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-migrate-67184f2a-66ffdf88d7-wd7x9"
Oct 30 22:15:29.480628 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:29.480587    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:15:29.721066 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:29.721027    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:15:30.034987 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:30.034950    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:15:30.243504 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:30.243464    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:15:32.680035 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:32.679999    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-ff716ef6-5574d46599-svwhz"
Oct 30 22:15:32.680493 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:32.680090    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-72b5472b-544d6795d5-vtsjp"
Oct 30 22:15:32.680493 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:32.680125    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-72b5472b-5dfd8db78c-zwg2j"
Oct 30 22:15:32.680493 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:32.680367    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-ff716ef6-f89c8bd9c-mhgqb"
Oct 30 22:15:34.824552 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:34.824500    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:15:38.422828 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:15:38.422788    5510 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6gjbf" containerName="extractor"
Oct 30 22:15:39.513681 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:39.509044    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:15:44.130350 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:44.130309    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:15:46.777354 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:46.777312    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:15:46.990199 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:46.990155    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:15:47.673221 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:47.673181    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:15:47.867720 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:47.867647    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:15:48.700557 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:48.700517    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:15:49.045016 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:49.044977    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:15:52.030816 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:52.030760    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-c9dbe6/distinct-w5-c9dbe6-kserve-7599fd86d9-bdhxd" podUID="e903db90-f2cd-4244-b00b-f99b21873f6e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:15:54.115834 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:54.115785    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:15:58.424562 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:58.424528    5510 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-ff716ef6-6c87f5cf68-vrk8m"]
Oct 30 22:15:58.430835 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:58.430813    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ff716ef6-6c87f5cf68-vrk8m"
Oct 30 22:15:58.535628 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:58.535596    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-ff716ef6-serving-cert\""
Oct 30 22:15:58.539866 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:58.539836    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-ff716ef6-6c87f5cf68-vrk8m"]
Oct 30 22:15:58.584266 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:58.584223    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qwdz9\" (UniqueName: \"kubernetes.io/projected/fa1f9fc4-483e-4423-9242-8cd97885bb41-kube-api-access-qwdz9\") pod \"maas-api-e2e-ait-ff716ef6-6c87f5cf68-vrk8m\" (UID: \"fa1f9fc4-483e-4423-9242-8cd97885bb41\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ff716ef6-6c87f5cf68-vrk8m"
Oct 30 22:15:58.584431 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:58.584381    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fa1f9fc4-483e-4423-9242-8cd97885bb41-openshift-service-ca\") pod \"maas-api-e2e-ait-ff716ef6-6c87f5cf68-vrk8m\" (UID: \"fa1f9fc4-483e-4423-9242-8cd97885bb41\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ff716ef6-6c87f5cf68-vrk8m"
Oct 30 22:15:58.584431 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:58.584416    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fa1f9fc4-483e-4423-9242-8cd97885bb41-metrics-certs\") pod \"maas-api-e2e-ait-ff716ef6-6c87f5cf68-vrk8m\" (UID: \"fa1f9fc4-483e-4423-9242-8cd97885bb41\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ff716ef6-6c87f5cf68-vrk8m"
Oct 30 22:15:58.584732 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:58.584710    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fa1f9fc4-483e-4423-9242-8cd97885bb41-maas-api-tls\") pod \"maas-api-e2e-ait-ff716ef6-6c87f5cf68-vrk8m\" (UID: \"fa1f9fc4-483e-4423-9242-8cd97885bb41\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ff716ef6-6c87f5cf68-vrk8m"
Oct 30 22:15:58.713742 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:58.687955    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fa1f9fc4-483e-4423-9242-8cd97885bb41-openshift-service-ca\") pod \"maas-api-e2e-ait-ff716ef6-6c87f5cf68-vrk8m\" (UID: \"fa1f9fc4-483e-4423-9242-8cd97885bb41\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ff716ef6-6c87f5cf68-vrk8m"
Oct 30 22:15:58.721684 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:58.714754    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fa1f9fc4-483e-4423-9242-8cd97885bb41-metrics-certs\") pod \"maas-api-e2e-ait-ff716ef6-6c87f5cf68-vrk8m\" (UID: \"fa1f9fc4-483e-4423-9242-8cd97885bb41\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ff716ef6-6c87f5cf68-vrk8m"
Oct 30 22:15:58.731681 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:58.731626    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fa1f9fc4-483e-4423-9242-8cd97885bb41-maas-api-tls\") pod \"maas-api-e2e-ait-ff716ef6-6c87f5cf68-vrk8m\" (UID: \"fa1f9fc4-483e-4423-9242-8cd97885bb41\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ff716ef6-6c87f5cf68-vrk8m"
Oct 30 22:15:58.732615 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:58.689626    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fa1f9fc4-483e-4423-9242-8cd97885bb41-openshift-service-ca\") pod \"maas-api-e2e-ait-ff716ef6-6c87f5cf68-vrk8m\" (UID: \"fa1f9fc4-483e-4423-9242-8cd97885bb41\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ff716ef6-6c87f5cf68-vrk8m"
Oct 30 22:15:58.744959 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:58.743961    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qwdz9\" (UniqueName: \"kubernetes.io/projected/fa1f9fc4-483e-4423-9242-8cd97885bb41-kube-api-access-qwdz9\") pod \"maas-api-e2e-ait-ff716ef6-6c87f5cf68-vrk8m\" (UID: \"fa1f9fc4-483e-4423-9242-8cd97885bb41\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ff716ef6-6c87f5cf68-vrk8m"
Oct 30 22:15:58.751846 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:58.710348    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:15:58.802740 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:58.771941    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fa1f9fc4-483e-4423-9242-8cd97885bb41-metrics-certs\") pod \"maas-api-e2e-ait-ff716ef6-6c87f5cf68-vrk8m\" (UID: \"fa1f9fc4-483e-4423-9242-8cd97885bb41\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ff716ef6-6c87f5cf68-vrk8m"
Oct 30 22:15:58.806686 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:58.805990    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fa1f9fc4-483e-4423-9242-8cd97885bb41-maas-api-tls\") pod \"maas-api-e2e-ait-ff716ef6-6c87f5cf68-vrk8m\" (UID: \"fa1f9fc4-483e-4423-9242-8cd97885bb41\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ff716ef6-6c87f5cf68-vrk8m"
Oct 30 22:15:58.810867 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:58.810361    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qwdz9\" (UniqueName: \"kubernetes.io/projected/fa1f9fc4-483e-4423-9242-8cd97885bb41-kube-api-access-qwdz9\") pod \"maas-api-e2e-ait-ff716ef6-6c87f5cf68-vrk8m\" (UID: \"fa1f9fc4-483e-4423-9242-8cd97885bb41\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ff716ef6-6c87f5cf68-vrk8m"
Oct 30 22:15:58.951500 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:58.951459    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:15:59.046269 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:59.046235    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ff716ef6-6c87f5cf68-vrk8m"
Oct 30 22:15:59.427324 ip-10-0-136-210 kubenswrapper[5510]: W1030 22:15:59.427286    5510 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa1f9fc4_483e_4423_9242_8cd97885bb41.slice/crio-371125ca7fd6c35b70a6780db3344d38d084f3b0fd203975ca464febcab25fbf WatchSource:0}: Error finding container 371125ca7fd6c35b70a6780db3344d38d084f3b0fd203975ca464febcab25fbf: Status 404 returned error can't find the container with id 371125ca7fd6c35b70a6780db3344d38d084f3b0fd203975ca464febcab25fbf
Oct 30 22:15:59.429863 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:15:59.429840    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-ff716ef6-6c87f5cf68-vrk8m"]
Oct 30 22:16:00.067517 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:00.067437    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ff716ef6-6c87f5cf68-vrk8m" event={"ID":"fa1f9fc4-483e-4423-9242-8cd97885bb41","Type":"ContainerStarted","Data":"371125ca7fd6c35b70a6780db3344d38d084f3b0fd203975ca464febcab25fbf"}
Oct 30 22:16:01.081817 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:01.081773    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ff716ef6-6c87f5cf68-vrk8m" event={"ID":"fa1f9fc4-483e-4423-9242-8cd97885bb41","Type":"ContainerStarted","Data":"74df5e6dbd1fbb7998d0ff7e9a5fd00fac8d13c1c245e3fcfd470afd3405f280"}
Oct 30 22:16:01.082254 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:01.081983    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ff716ef6-6c87f5cf68-vrk8m"
Oct 30 22:16:01.251483 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:01.251425    5510 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ff716ef6-6c87f5cf68-vrk8m" podStartSLOduration=2.34164268 podStartE2EDuration="3.251408543s" totalImagesPullingTime="909.765863ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:15:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:16:01.201523833 +0000 UTC m=+1541.762333850" watchObservedRunningTime="2026-10-30 22:16:01.251408543 +0000 UTC m=+1541.812218547"
Oct 30 22:16:03.437115 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:03.437054    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:16:03.645681 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:03.640286    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:16:05.195697 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:05.195638    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:16:05.486131 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:05.483800    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:16:06.759670 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:06.759613    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:16:07.023081 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:07.023047    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:16:07.094746 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:07.094718    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ff716ef6-6c87f5cf68-vrk8m"
Oct 30 22:16:08.161537 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:08.161497    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:16:08.382165 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:08.382129    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:16:08.421495 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:16:08.421411    5510 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6gjbf" containerName="extractor"
Oct 30 22:16:12.805692 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:12.800224    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:16:14.235014 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:14.234971    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:16:17.499460 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:17.499425    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:16:19.919068 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:19.919027    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:16:21.057981 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:21.057929    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:16:22.166189 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:22.163830    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:16:26.700861 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:26.700799    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-bc714d/distinct-w2-bc714d-kserve-66cfb968dd-jvgqj" podUID="9e9b11b8-1cc9-4fc8-8097-1afcffa8c422" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:16:26.906684 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:26.903754    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-779f32/distinct-2-w0-779f32-kserve-857ffdf778-gk2h9" podUID="ef500cdd-cc52-45b0-8dd7-1896897acc46" containerName="main" probeResult="failure" output="Get \"https://10.133.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:16:26.965762 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:26.963774    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-779f32/distinct-w0-779f32-kserve-6dcbc948c7-2vzdf" podUID="e17cedd4-7d48-42b3-b127-b6192c1b3795" containerName="main" probeResult="failure" output="Get \"https://10.133.0.65:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:16:27.014936 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:27.014889    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-c9dbe6/distinct-w5-c9dbe6-kserve-7599fd86d9-bdhxd" podUID="e903db90-f2cd-4244-b00b-f99b21873f6e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:16:27.053863 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:27.053816    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-7bb1c2/distinct-2-w3-7bb1c2-kserve-c549df844-zt595" podUID="1916e87c-a1a4-4967-8640-8bbcd9d12903" containerName="main" probeResult="failure" output="Get \"https://10.133.0.73:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:16:27.138493 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:27.138449    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:16:27.679633 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:27.679583    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-fb0b93/premium-w6-fb0b93-kserve-5cb9948c54-w7pnk" podUID="290ae7a2-de37-4af3-82ae-a8e855b83bb9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:16:27.798268 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:27.702968    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-bc714d/distinct-w2-bc714d-kserve-66cfb968dd-jvgqj" podUID="9e9b11b8-1cc9-4fc8-8097-1afcffa8c422" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": context deadline exceeded"
Oct 30 22:16:27.834428 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:27.802841    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-fb0b93/distinct-w6-fb0b93-kserve-58db6b6757-6mww8" podUID="8ee6579b-acea-4492-862b-9883f70d00c9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.59:8000/ready\": context deadline exceeded"
Oct 30 22:16:27.953081 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:27.952972    5510 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w6-fb0b93/distinct-w6-fb0b93-kserve-58db6b6757-6mww8" podUID="8ee6579b-acea-4492-862b-9883f70d00c9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.59:8000/health\": context deadline exceeded"
Oct 30 22:16:27.955739 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:27.955709    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-bc714d/distinct-w2-bc714d-kserve-66cfb968dd-jvgqj"
Oct 30 22:16:27.975492 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:27.955965    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-779f32/distinct-2-w0-779f32-kserve-857ffdf778-gk2h9" podUID="ef500cdd-cc52-45b0-8dd7-1896897acc46" containerName="main" probeResult="failure" output="Get \"https://10.133.0.74:8000/ready\": context deadline exceeded"
Oct 30 22:16:27.976236 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:27.956003    5510 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w6-fb0b93/premium-w6-fb0b93-kserve-5cb9948c54-w7pnk" podUID="290ae7a2-de37-4af3-82ae-a8e855b83bb9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:16:27.976431 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:27.975788    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-779f32/distinct-w0-779f32-kserve-6dcbc948c7-2vzdf" podUID="e17cedd4-7d48-42b3-b127-b6192c1b3795" containerName="main" probeResult="failure" output="Get \"https://10.133.0.65:8000/ready\": context deadline exceeded"
Oct 30 22:16:27.976807 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:27.976780    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-779f32/distinct-2-w0-779f32-kserve-857ffdf778-gk2h9"
Oct 30 22:16:27.976918 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:27.976844    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-779f32/distinct-w0-779f32-kserve-6dcbc948c7-2vzdf"
Oct 30 22:16:28.163796 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:28.163750    5510 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-779f32/distinct-2-w0-779f32-kserve-857ffdf778-gk2h9" podUID="ef500cdd-cc52-45b0-8dd7-1896897acc46" containerName="main" probeResult="failure" output="Get \"https://10.133.0.74:8000/health\": context deadline exceeded"
Oct 30 22:16:28.164337 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:28.164301    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-7bb1c2/distinct-2-w3-7bb1c2-kserve-c549df844-zt595" podUID="1916e87c-a1a4-4967-8640-8bbcd9d12903" containerName="main" probeResult="failure" output="Get \"https://10.133.0.73:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:16:28.243806 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:28.195853    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-c9dbe6/distinct-w5-c9dbe6-kserve-7599fd86d9-bdhxd" podUID="e903db90-f2cd-4244-b00b-f99b21873f6e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.67:8000/ready\": context deadline exceeded"
Oct 30 22:16:28.298054 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:28.251787    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-7bb1c2/distinct-2-w3-7bb1c2-kserve-c549df844-zt595"
Oct 30 22:16:28.298367 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:28.298349    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-c9dbe6/distinct-w5-c9dbe6-kserve-7599fd86d9-bdhxd"
Oct 30 22:16:28.306688 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:28.303059    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:16:28.786739 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:28.730524    5510 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-779f32/distinct-w0-779f32-kserve-6dcbc948c7-2vzdf" podUID="e17cedd4-7d48-42b3-b127-b6192c1b3795" containerName="main" probeResult="failure" output="Get \"https://10.133.0.65:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:16:29.004682 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:28.870353    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-fb0b93/distinct-w6-fb0b93-kserve-58db6b6757-6mww8" podUID="8ee6579b-acea-4492-862b-9883f70d00c9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.59:8000/ready\": context deadline exceeded"
Oct 30 22:16:29.054793 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:29.031784    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-fb0b93/distinct-w6-fb0b93-kserve-58db6b6757-6mww8"
Oct 30 22:16:29.054793 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:28.879169    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-bc714d/distinct-w2-bc714d-kserve-66cfb968dd-jvgqj" podUID="9e9b11b8-1cc9-4fc8-8097-1afcffa8c422" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:16:29.054793 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:28.923914    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-fb0b93/premium-w6-fb0b93-kserve-5cb9948c54-w7pnk" podUID="290ae7a2-de37-4af3-82ae-a8e855b83bb9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:16:29.083792 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:28.981944    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-779f32/distinct-w0-779f32-kserve-6dcbc948c7-2vzdf" podUID="e17cedd4-7d48-42b3-b127-b6192c1b3795" containerName="main" probeResult="failure" output="Get \"https://10.133.0.65:8000/ready\": context deadline exceeded"
Oct 30 22:16:29.083792 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:28.897931    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:16:29.083792 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:28.983480    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-779f32/distinct-2-w0-779f32-kserve-857ffdf778-gk2h9" podUID="ef500cdd-cc52-45b0-8dd7-1896897acc46" containerName="main" probeResult="failure" output="Get \"https://10.133.0.74:8000/ready\": context deadline exceeded"
Oct 30 22:16:29.109738 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:29.096873    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-fb0b93/premium-w6-fb0b93-kserve-5cb9948c54-w7pnk"
Oct 30 22:16:29.292696 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:29.292633    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-7bb1c2/distinct-2-w3-7bb1c2-kserve-c549df844-zt595" podUID="1916e87c-a1a4-4967-8640-8bbcd9d12903" containerName="main" probeResult="failure" output="Get \"https://10.133.0.73:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:16:29.299819 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:29.299772    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-c9dbe6/distinct-w5-c9dbe6-kserve-7599fd86d9-bdhxd" podUID="e903db90-f2cd-4244-b00b-f99b21873f6e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:16:30.033774 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:30.033732    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-fb0b93/distinct-w6-fb0b93-kserve-58db6b6757-6mww8" podUID="8ee6579b-acea-4492-862b-9883f70d00c9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.59:8000/ready\": context deadline exceeded"
Oct 30 22:16:30.035215 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:30.035183    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-bc714d/distinct-w2-bc714d-kserve-66cfb968dd-jvgqj" podUID="9e9b11b8-1cc9-4fc8-8097-1afcffa8c422" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": context deadline exceeded"
Oct 30 22:16:30.074362 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:30.074317    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-fb0b93/premium-w6-fb0b93-kserve-5cb9948c54-w7pnk" podUID="290ae7a2-de37-4af3-82ae-a8e855b83bb9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": context deadline exceeded"
Oct 30 22:16:30.108086 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:30.105584    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-779f32/distinct-w0-779f32-kserve-6dcbc948c7-2vzdf" podUID="e17cedd4-7d48-42b3-b127-b6192c1b3795" containerName="main" probeResult="failure" output="Get \"https://10.133.0.65:8000/ready\": context deadline exceeded"
Oct 30 22:16:30.111580 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:30.111539    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-779f32/distinct-2-w0-779f32-kserve-857ffdf778-gk2h9" podUID="ef500cdd-cc52-45b0-8dd7-1896897acc46" containerName="main" probeResult="failure" output="Get \"https://10.133.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:16:30.214464 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:30.214423    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-bc714d/distinct-w2-bc714d-kserve-66cfb968dd-jvgqj"
Oct 30 22:16:30.268685 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:30.239475    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-779f32/distinct-w0-779f32-kserve-6dcbc948c7-2vzdf"
Oct 30 22:16:30.293890 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:30.293785    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-7bb1c2/distinct-2-w3-7bb1c2-kserve-c549df844-zt595" podUID="1916e87c-a1a4-4967-8640-8bbcd9d12903" containerName="main" probeResult="failure" output="Get \"https://10.133.0.73:8000/ready\": context deadline exceeded"
Oct 30 22:16:30.318543 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:30.318500    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-c9dbe6/distinct-w5-c9dbe6-kserve-7599fd86d9-bdhxd" podUID="e903db90-f2cd-4244-b00b-f99b21873f6e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:16:30.337110 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:30.332811    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-779f32/distinct-2-w0-779f32-kserve-857ffdf778-gk2h9"
Oct 30 22:16:30.337110 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:30.333136    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-fb0b93/distinct-w6-fb0b93-kserve-58db6b6757-6mww8"
Oct 30 22:16:30.337110 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:30.334019    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-7bb1c2/distinct-2-w3-7bb1c2-kserve-c549df844-zt595"
Oct 30 22:16:30.337110 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:30.336143    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-fb0b93/premium-w6-fb0b93-kserve-5cb9948c54-w7pnk"
Oct 30 22:16:30.337110 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:30.336468    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-c9dbe6/distinct-w5-c9dbe6-kserve-7599fd86d9-bdhxd"
Oct 30 22:16:34.364682 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:34.339518    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:16:38.138425 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:38.138396    5510 generic.go:356] "Generic (PLEG): container finished" podID="ebf683b5-5605-4790-813e-3f06b8f8c38f" containerID="0e581049efefcb2f4e1c0c85388ccfce3199fe27e9e1336b26e6ba65e0e0c000" exitCode=0
Oct 30 22:16:38.138868 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:38.138463    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-72b5472b-openshift-default-6c5c978fc5-xs68s" event={"ID":"ebf683b5-5605-4790-813e-3f06b8f8c38f","Type":"ContainerDied","Data":"0e581049efefcb2f4e1c0c85388ccfce3199fe27e9e1336b26e6ba65e0e0c000"}
Oct 30 22:16:38.139014 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:38.138996    5510 scope.go:122] "RemoveContainer" containerID="0e581049efefcb2f4e1c0c85388ccfce3199fe27e9e1336b26e6ba65e0e0c000"
Oct 30 22:16:38.140771 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:38.140739    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:16:38.140885 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:38.140829    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:16:38.140885 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:38.140874    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:16:38.140965 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:38.140882    5510 generic.go:356] "Generic (PLEG): container finished" podID="f067f5fc-05db-4807-982f-21e7a5997acc" containerID="9ef24aa63fe5461c2c7c015615611beddd21df935aa93d916eb5ba896912143f" exitCode=0
Oct 30 22:16:38.140965 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:38.140946    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-67184f2a-gw-openshift-default-5cb54945b9-5c64x" event={"ID":"f067f5fc-05db-4807-982f-21e7a5997acc","Type":"ContainerDied","Data":"9ef24aa63fe5461c2c7c015615611beddd21df935aa93d916eb5ba896912143f"}
Oct 30 22:16:38.141431 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:38.141329    5510 scope.go:122] "RemoveContainer" containerID="9ef24aa63fe5461c2c7c015615611beddd21df935aa93d916eb5ba896912143f"
Oct 30 22:16:38.142843 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:38.142808    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:16:38.142930 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:38.142878    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:16:38.142930 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:38.142916    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:16:38.469377 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:16:38.469326    5510 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6gjbf" containerName="extractor"
Oct 30 22:16:39.176245 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:39.175733    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-72b5472b-openshift-default-6c5c978fc5-xs68s"
Oct 30 22:16:39.176978 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:39.176944    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:16:39.177912 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:39.177889    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-67184f2a-gw-openshift-default-5cb54945b9-5c64x" event={"ID":"f067f5fc-05db-4807-982f-21e7a5997acc","Type":"ContainerStarted","Data":"ce5d937cbad5658849effa727bf0135def5e85e56e4cd5e7d24d4e4a8318f8cb"}
Oct 30 22:16:39.211786 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:39.211734    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-72b5472b-openshift-default-6c5c978fc5-xs68s" event={"ID":"ebf683b5-5605-4790-813e-3f06b8f8c38f","Type":"ContainerStarted","Data":"820b206c67fa43a19deb4fd4d8194dd69ebc402da24c0acdfdd026efa12a50e8"}
Oct 30 22:16:39.536020 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:39.535985    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-67184f2a-gw-openshift-default-5cb54945b9-5c64x"
Oct 30 22:16:39.536201 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:39.536156    5510 patch_prober.go:29] interesting pod/e2e-migrate-67184f2a-gw-openshift-default-5cb54945b9-5c64x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused" start-of-body=
Oct 30 22:16:39.536246 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:39.536202    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-67184f2a-gw-openshift-default-5cb54945b9-5c64x" podUID="f067f5fc-05db-4807-982f-21e7a5997acc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused"
Oct 30 22:16:40.085680 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:40.084887    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-72b5472b-openshift-default-6c5c978fc5-xs68s"
Oct 30 22:16:40.090692 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:40.088820    5510 patch_prober.go:29] interesting pod/e2e-ait-72b5472b-openshift-default-6c5c978fc5-xs68s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.89:15021/healthz/ready\": dial tcp 10.133.0.89:15021: connect: connection refused" start-of-body=
Oct 30 22:16:40.090692 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:40.088898    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-72b5472b-openshift-default-6c5c978fc5-xs68s" podUID="ebf683b5-5605-4790-813e-3f06b8f8c38f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.89:15021/healthz/ready\": dial tcp 10.133.0.89:15021: connect: connection refused"
Oct 30 22:16:40.535099 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:40.535069    5510 patch_prober.go:29] interesting pod/e2e-migrate-67184f2a-gw-openshift-default-5cb54945b9-5c64x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused" start-of-body=
Oct 30 22:16:40.535493 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:40.535125    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-67184f2a-gw-openshift-default-5cb54945b9-5c64x" podUID="f067f5fc-05db-4807-982f-21e7a5997acc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused"
Oct 30 22:16:41.095115 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:41.093765    5510 patch_prober.go:29] interesting pod/e2e-ait-72b5472b-openshift-default-6c5c978fc5-xs68s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.89:15021/healthz/ready\": dial tcp 10.133.0.89:15021: connect: connection refused" start-of-body=
Oct 30 22:16:41.095115 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:41.093830    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-72b5472b-openshift-default-6c5c978fc5-xs68s" podUID="ebf683b5-5605-4790-813e-3f06b8f8c38f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.89:15021/healthz/ready\": dial tcp 10.133.0.89:15021: connect: connection refused"
Oct 30 22:16:41.535182 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:41.535142    5510 patch_prober.go:29] interesting pod/e2e-migrate-67184f2a-gw-openshift-default-5cb54945b9-5c64x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused" start-of-body=
Oct 30 22:16:41.535577 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:41.535199    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-67184f2a-gw-openshift-default-5cb54945b9-5c64x" podUID="f067f5fc-05db-4807-982f-21e7a5997acc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused"
Oct 30 22:16:42.085144 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:42.085105    5510 patch_prober.go:29] interesting pod/e2e-ait-72b5472b-openshift-default-6c5c978fc5-xs68s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.89:15021/healthz/ready\": dial tcp 10.133.0.89:15021: connect: connection refused" start-of-body=
Oct 30 22:16:42.085450 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:42.085422    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-72b5472b-openshift-default-6c5c978fc5-xs68s" podUID="ebf683b5-5605-4790-813e-3f06b8f8c38f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.89:15021/healthz/ready\": dial tcp 10.133.0.89:15021: connect: connection refused"
Oct 30 22:16:42.535420 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:42.535390    5510 patch_prober.go:29] interesting pod/e2e-migrate-67184f2a-gw-openshift-default-5cb54945b9-5c64x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused" start-of-body=
Oct 30 22:16:42.535813 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:42.535443    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-67184f2a-gw-openshift-default-5cb54945b9-5c64x" podUID="f067f5fc-05db-4807-982f-21e7a5997acc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused"
Oct 30 22:16:43.087681 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:43.087111    5510 patch_prober.go:29] interesting pod/e2e-ait-72b5472b-openshift-default-6c5c978fc5-xs68s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.89:15021/healthz/ready\": dial tcp 10.133.0.89:15021: connect: connection refused" start-of-body=
Oct 30 22:16:43.087681 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:43.087179    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-72b5472b-openshift-default-6c5c978fc5-xs68s" podUID="ebf683b5-5605-4790-813e-3f06b8f8c38f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.89:15021/healthz/ready\": dial tcp 10.133.0.89:15021: connect: connection refused"
Oct 30 22:16:43.535817 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:43.535774    5510 patch_prober.go:29] interesting pod/e2e-migrate-67184f2a-gw-openshift-default-5cb54945b9-5c64x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused" start-of-body=
Oct 30 22:16:43.536233 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:43.535840    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-67184f2a-gw-openshift-default-5cb54945b9-5c64x" podUID="f067f5fc-05db-4807-982f-21e7a5997acc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused"
Oct 30 22:16:44.097681 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:44.092842    5510 patch_prober.go:29] interesting pod/e2e-ait-72b5472b-openshift-default-6c5c978fc5-xs68s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.89:15021/healthz/ready\": dial tcp 10.133.0.89:15021: connect: connection refused" start-of-body=
Oct 30 22:16:44.097681 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:44.092905    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-72b5472b-openshift-default-6c5c978fc5-xs68s" podUID="ebf683b5-5605-4790-813e-3f06b8f8c38f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.89:15021/healthz/ready\": dial tcp 10.133.0.89:15021: connect: connection refused"
Oct 30 22:16:44.171680 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:44.168279    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:16:44.535472 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:44.535440    5510 patch_prober.go:29] interesting pod/e2e-migrate-67184f2a-gw-openshift-default-5cb54945b9-5c64x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused" start-of-body=
Oct 30 22:16:44.535730 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:44.535704    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-67184f2a-gw-openshift-default-5cb54945b9-5c64x" podUID="f067f5fc-05db-4807-982f-21e7a5997acc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused"
Oct 30 22:16:44.593387 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:44.593344    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:16:44.724172 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:44.724133    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-67184f2a-gw-openshift-default-5cb54945b9-5c64x"
Oct 30 22:16:44.936498 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:44.936407    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:16:45.089612 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:45.089582    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-72b5472b-openshift-default-6c5c978fc5-xs68s"
Oct 30 22:16:45.089877 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:45.089848    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-72b5472b-openshift-default-6c5c978fc5-xs68s"
Oct 30 22:16:45.090530 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:45.090508    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-72b5472b-openshift-default-6c5c978fc5-xs68s"
Oct 30 22:16:45.192261 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:45.192177    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:16:45.539810 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:45.539775    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-67184f2a-gw-openshift-default-5cb54945b9-5c64x"
Oct 30 22:16:45.540887 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:45.540866    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-67184f2a-gw-openshift-default-5cb54945b9-5c64x"
Oct 30 22:16:47.996450 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:47.996413    5510 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-j4m88"]
Oct 30 22:16:48.000223 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:48.000198    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-j4m88"
Oct 30 22:16:48.036036 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:48.036005    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a089cfee-9de2-4d44-b7d9-dbbc2a66ebd4-istio-data\") pod \"e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-j4m88\" (UID: \"a089cfee-9de2-4d44-b7d9-dbbc2a66ebd4\") " pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-j4m88"
Oct 30 22:16:48.036165 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:48.036046    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a089cfee-9de2-4d44-b7d9-dbbc2a66ebd4-istio-token\") pod \"e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-j4m88\" (UID: \"a089cfee-9de2-4d44-b7d9-dbbc2a66ebd4\") " pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-j4m88"
Oct 30 22:16:48.036248 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:48.036166    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a089cfee-9de2-4d44-b7d9-dbbc2a66ebd4-istiod-ca-cert\") pod \"e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-j4m88\" (UID: \"a089cfee-9de2-4d44-b7d9-dbbc2a66ebd4\") " pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-j4m88"
Oct 30 22:16:48.036248 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:48.036199    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a089cfee-9de2-4d44-b7d9-dbbc2a66ebd4-istio-envoy\") pod \"e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-j4m88\" (UID: \"a089cfee-9de2-4d44-b7d9-dbbc2a66ebd4\") " pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-j4m88"
Oct 30 22:16:48.036248 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:48.036216    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t29zw\" (UniqueName: \"kubernetes.io/projected/a089cfee-9de2-4d44-b7d9-dbbc2a66ebd4-kube-api-access-t29zw\") pod \"e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-j4m88\" (UID: \"a089cfee-9de2-4d44-b7d9-dbbc2a66ebd4\") " pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-j4m88"
Oct 30 22:16:48.036424 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:48.036405    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a089cfee-9de2-4d44-b7d9-dbbc2a66ebd4-istio-podinfo\") pod \"e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-j4m88\" (UID: \"a089cfee-9de2-4d44-b7d9-dbbc2a66ebd4\") " pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-j4m88"
Oct 30 22:16:48.036511 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:48.036497    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a089cfee-9de2-4d44-b7d9-dbbc2a66ebd4-workload-certs\") pod \"e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-j4m88\" (UID: \"a089cfee-9de2-4d44-b7d9-dbbc2a66ebd4\") " pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-j4m88"
Oct 30 22:16:48.036554 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:48.036523    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a089cfee-9de2-4d44-b7d9-dbbc2a66ebd4-credential-socket\") pod \"e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-j4m88\" (UID: \"a089cfee-9de2-4d44-b7d9-dbbc2a66ebd4\") " pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-j4m88"
Oct 30 22:16:48.036694 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:48.036646    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a089cfee-9de2-4d44-b7d9-dbbc2a66ebd4-workload-socket\") pod \"e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-j4m88\" (UID: \"a089cfee-9de2-4d44-b7d9-dbbc2a66ebd4\") " pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-j4m88"
Oct 30 22:16:48.113042 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:48.113002    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-j4m88"]
Oct 30 22:16:48.137776 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:48.137747    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a089cfee-9de2-4d44-b7d9-dbbc2a66ebd4-credential-socket\") pod \"e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-j4m88\" (UID: \"a089cfee-9de2-4d44-b7d9-dbbc2a66ebd4\") " pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-j4m88"
Oct 30 22:16:48.137913 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:48.137840    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a089cfee-9de2-4d44-b7d9-dbbc2a66ebd4-workload-socket\") pod \"e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-j4m88\" (UID: \"a089cfee-9de2-4d44-b7d9-dbbc2a66ebd4\") " pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-j4m88"
Oct 30 22:16:48.137979 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:48.137943    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a089cfee-9de2-4d44-b7d9-dbbc2a66ebd4-istio-data\") pod \"e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-j4m88\" (UID: \"a089cfee-9de2-4d44-b7d9-dbbc2a66ebd4\") " pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-j4m88"
Oct 30 22:16:48.137979 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:48.137962    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a089cfee-9de2-4d44-b7d9-dbbc2a66ebd4-istio-token\") pod \"e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-j4m88\" (UID: \"a089cfee-9de2-4d44-b7d9-dbbc2a66ebd4\") " pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-j4m88"
Oct 30 22:16:48.138081 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:48.138005    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a089cfee-9de2-4d44-b7d9-dbbc2a66ebd4-istiod-ca-cert\") pod \"e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-j4m88\" (UID: \"a089cfee-9de2-4d44-b7d9-dbbc2a66ebd4\") " pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-j4m88"
Oct 30 22:16:48.138081 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:48.138035    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a089cfee-9de2-4d44-b7d9-dbbc2a66ebd4-istio-envoy\") pod \"e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-j4m88\" (UID: \"a089cfee-9de2-4d44-b7d9-dbbc2a66ebd4\") " pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-j4m88"
Oct 30 22:16:48.138081 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:48.138048    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a089cfee-9de2-4d44-b7d9-dbbc2a66ebd4-credential-socket\") pod \"e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-j4m88\" (UID: \"a089cfee-9de2-4d44-b7d9-dbbc2a66ebd4\") " pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-j4m88"
Oct 30 22:16:48.138081 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:48.138059    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t29zw\" (UniqueName: \"kubernetes.io/projected/a089cfee-9de2-4d44-b7d9-dbbc2a66ebd4-kube-api-access-t29zw\") pod \"e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-j4m88\" (UID: \"a089cfee-9de2-4d44-b7d9-dbbc2a66ebd4\") " pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-j4m88"
Oct 30 22:16:48.138300 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:48.138189    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a089cfee-9de2-4d44-b7d9-dbbc2a66ebd4-istio-podinfo\") pod \"e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-j4m88\" (UID: \"a089cfee-9de2-4d44-b7d9-dbbc2a66ebd4\") " pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-j4m88"
Oct 30 22:16:48.138300 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:48.138261    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a089cfee-9de2-4d44-b7d9-dbbc2a66ebd4-workload-certs\") pod \"e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-j4m88\" (UID: \"a089cfee-9de2-4d44-b7d9-dbbc2a66ebd4\") " pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-j4m88"
Oct 30 22:16:48.138300 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:48.138188    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a089cfee-9de2-4d44-b7d9-dbbc2a66ebd4-workload-socket\") pod \"e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-j4m88\" (UID: \"a089cfee-9de2-4d44-b7d9-dbbc2a66ebd4\") " pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-j4m88"
Oct 30 22:16:48.138427 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:48.138314    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a089cfee-9de2-4d44-b7d9-dbbc2a66ebd4-istio-data\") pod \"e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-j4m88\" (UID: \"a089cfee-9de2-4d44-b7d9-dbbc2a66ebd4\") " pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-j4m88"
Oct 30 22:16:48.138596 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:48.138573    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a089cfee-9de2-4d44-b7d9-dbbc2a66ebd4-workload-certs\") pod \"e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-j4m88\" (UID: \"a089cfee-9de2-4d44-b7d9-dbbc2a66ebd4\") " pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-j4m88"
Oct 30 22:16:48.138721 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:48.138697    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a089cfee-9de2-4d44-b7d9-dbbc2a66ebd4-istiod-ca-cert\") pod \"e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-j4m88\" (UID: \"a089cfee-9de2-4d44-b7d9-dbbc2a66ebd4\") " pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-j4m88"
Oct 30 22:16:48.140459 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:48.140431    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a089cfee-9de2-4d44-b7d9-dbbc2a66ebd4-istio-envoy\") pod \"e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-j4m88\" (UID: \"a089cfee-9de2-4d44-b7d9-dbbc2a66ebd4\") " pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-j4m88"
Oct 30 22:16:48.140671 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:48.140639    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a089cfee-9de2-4d44-b7d9-dbbc2a66ebd4-istio-podinfo\") pod \"e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-j4m88\" (UID: \"a089cfee-9de2-4d44-b7d9-dbbc2a66ebd4\") " pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-j4m88"
Oct 30 22:16:48.202001 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:48.201961    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a089cfee-9de2-4d44-b7d9-dbbc2a66ebd4-istio-token\") pod \"e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-j4m88\" (UID: \"a089cfee-9de2-4d44-b7d9-dbbc2a66ebd4\") " pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-j4m88"
Oct 30 22:16:48.202518 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:48.202496    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t29zw\" (UniqueName: \"kubernetes.io/projected/a089cfee-9de2-4d44-b7d9-dbbc2a66ebd4-kube-api-access-t29zw\") pod \"e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-j4m88\" (UID: \"a089cfee-9de2-4d44-b7d9-dbbc2a66ebd4\") " pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-j4m88"
Oct 30 22:16:48.314795 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:48.314763    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-j4m88"
Oct 30 22:16:48.729052 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:48.728833    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:16:48.771905 ip-10-0-136-210 kubenswrapper[5510]: W1030 22:16:48.771785    5510 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda089cfee_9de2_4d44_b7d9_dbbc2a66ebd4.slice/crio-cd6dea1499f666a7cc7afc0a66e18a52aca824a9aa9167d0fe6df971e9176e13 WatchSource:0}: Error finding container cd6dea1499f666a7cc7afc0a66e18a52aca824a9aa9167d0fe6df971e9176e13: Status 404 returned error can't find the container with id cd6dea1499f666a7cc7afc0a66e18a52aca824a9aa9167d0fe6df971e9176e13
Oct 30 22:16:48.788681 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:48.776107    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-j4m88"]
Oct 30 22:16:48.798689 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:48.790559    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:16:48.798689 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:48.790689    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:16:48.798689 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:48.790742    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:16:49.381181 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:49.381146    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-j4m88" event={"ID":"a089cfee-9de2-4d44-b7d9-dbbc2a66ebd4","Type":"ContainerStarted","Data":"431729942c9ae9dc292e10d5cdf7329d815661b49d6bdcd85877e365b8488573"}
Oct 30 22:16:49.381625 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:49.381238    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-j4m88" event={"ID":"a089cfee-9de2-4d44-b7d9-dbbc2a66ebd4","Type":"ContainerStarted","Data":"cd6dea1499f666a7cc7afc0a66e18a52aca824a9aa9167d0fe6df971e9176e13"}
Oct 30 22:16:49.571909 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:49.571826    5510 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-j4m88" podStartSLOduration=2.5718030819999997 podStartE2EDuration="2.571803082s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:16:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:16:49.559039205 +0000 UTC m=+1590.119849239" watchObservedRunningTime="2026-10-30 22:16:49.571803082 +0000 UTC m=+1590.132613088"
Oct 30 22:16:50.314920 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:50.314885    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-j4m88"
Oct 30 22:16:50.316522 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:50.316496    5510 patch_prober.go:29] interesting pod/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-j4m88 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.95:15021/healthz/ready\": dial tcp 10.133.0.95:15021: connect: connection refused" start-of-body=
Oct 30 22:16:50.316669 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:50.316541    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-j4m88" podUID="a089cfee-9de2-4d44-b7d9-dbbc2a66ebd4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.95:15021/healthz/ready\": dial tcp 10.133.0.95:15021: connect: connection refused"
Oct 30 22:16:51.316043 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:51.316010    5510 patch_prober.go:29] interesting pod/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-j4m88 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.95:15021/healthz/ready\": dial tcp 10.133.0.95:15021: connect: connection refused" start-of-body=
Oct 30 22:16:51.316472 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:51.316065    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-j4m88" podUID="a089cfee-9de2-4d44-b7d9-dbbc2a66ebd4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.95:15021/healthz/ready\": dial tcp 10.133.0.95:15021: connect: connection refused"
Oct 30 22:16:52.315593 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:52.315556    5510 patch_prober.go:29] interesting pod/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-j4m88 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.95:15021/healthz/ready\": dial tcp 10.133.0.95:15021: connect: connection refused" start-of-body=
Oct 30 22:16:52.315797 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:52.315611    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-j4m88" podUID="a089cfee-9de2-4d44-b7d9-dbbc2a66ebd4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.95:15021/healthz/ready\": dial tcp 10.133.0.95:15021: connect: connection refused"
Oct 30 22:16:52.679100 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:52.679011    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:16:52.940895 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:52.940820    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:16:53.315168 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:53.315124    5510 patch_prober.go:29] interesting pod/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-j4m88 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.95:15021/healthz/ready\": dial tcp 10.133.0.95:15021: connect: connection refused" start-of-body=
Oct 30 22:16:53.315345 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:53.315202    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-j4m88" podUID="a089cfee-9de2-4d44-b7d9-dbbc2a66ebd4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.95:15021/healthz/ready\": dial tcp 10.133.0.95:15021: connect: connection refused"
Oct 30 22:16:53.317591 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:53.317555    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:16:54.028451 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:54.028414    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:16:54.270554 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:54.270517    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:16:54.315224 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:54.315128    5510 patch_prober.go:29] interesting pod/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-j4m88 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.95:15021/healthz/ready\": dial tcp 10.133.0.95:15021: connect: connection refused" start-of-body=
Oct 30 22:16:54.315224 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:54.315182    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-j4m88" podUID="a089cfee-9de2-4d44-b7d9-dbbc2a66ebd4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.95:15021/healthz/ready\": dial tcp 10.133.0.95:15021: connect: connection refused"
Oct 30 22:16:55.315873 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:55.315839    5510 patch_prober.go:29] interesting pod/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-j4m88 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.95:15021/healthz/ready\": dial tcp 10.133.0.95:15021: connect: connection refused" start-of-body=
Oct 30 22:16:55.316279 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:55.315892    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-j4m88" podUID="a089cfee-9de2-4d44-b7d9-dbbc2a66ebd4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.95:15021/healthz/ready\": dial tcp 10.133.0.95:15021: connect: connection refused"
Oct 30 22:16:56.320098 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:56.320070    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-j4m88"
Oct 30 22:16:56.320537 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:56.320448    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-j4m88"
Oct 30 22:16:56.321280 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:56.321262    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-j4m88"
Oct 30 22:16:58.720292 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:58.720237    5510 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-89h6s"]
Oct 30 22:16:58.724433 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:58.724409    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-89h6s"
Oct 30 22:16:58.770936 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:58.770902    5510 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-6dft4"]
Oct 30 22:16:58.774991 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:58.774965    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-6dft4"
Oct 30 22:16:58.816375 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:58.816332    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/85f900f2-2722-4143-8974-d1c7ab81770b-istio-envoy\") pod \"maas-default-gateway-openshift-default-85544d4cd7-89h6s\" (UID: \"85f900f2-2722-4143-8974-d1c7ab81770b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-89h6s"
Oct 30 22:16:58.816617 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:58.816477    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cc069342-244c-467a-9d89-2b7fb41338e9-istio-data\") pod \"maas-default-gateway-openshift-default-85544d4cd7-6dft4\" (UID: \"cc069342-244c-467a-9d89-2b7fb41338e9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-6dft4"
Oct 30 22:16:58.816617 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:58.816569    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/85f900f2-2722-4143-8974-d1c7ab81770b-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-85544d4cd7-89h6s\" (UID: \"85f900f2-2722-4143-8974-d1c7ab81770b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-89h6s"
Oct 30 22:16:58.816617 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:58.816599    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cc069342-244c-467a-9d89-2b7fb41338e9-workload-socket\") pod \"maas-default-gateway-openshift-default-85544d4cd7-6dft4\" (UID: \"cc069342-244c-467a-9d89-2b7fb41338e9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-6dft4"
Oct 30 22:16:58.816783 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:58.816728    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/85f900f2-2722-4143-8974-d1c7ab81770b-credential-socket\") pod \"maas-default-gateway-openshift-default-85544d4cd7-89h6s\" (UID: \"85f900f2-2722-4143-8974-d1c7ab81770b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-89h6s"
Oct 30 22:16:58.816823 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:58.816786    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wwmj4\" (UniqueName: \"kubernetes.io/projected/85f900f2-2722-4143-8974-d1c7ab81770b-kube-api-access-wwmj4\") pod \"maas-default-gateway-openshift-default-85544d4cd7-89h6s\" (UID: \"85f900f2-2722-4143-8974-d1c7ab81770b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-89h6s"
Oct 30 22:16:58.816889 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:58.816843    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cc069342-244c-467a-9d89-2b7fb41338e9-istio-token\") pod \"maas-default-gateway-openshift-default-85544d4cd7-6dft4\" (UID: \"cc069342-244c-467a-9d89-2b7fb41338e9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-6dft4"
Oct 30 22:16:58.816924 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:58.816906    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-twgnb\" (UniqueName: \"kubernetes.io/projected/cc069342-244c-467a-9d89-2b7fb41338e9-kube-api-access-twgnb\") pod \"maas-default-gateway-openshift-default-85544d4cd7-6dft4\" (UID: \"cc069342-244c-467a-9d89-2b7fb41338e9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-6dft4"
Oct 30 22:16:58.816997 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:58.816983    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cc069342-244c-467a-9d89-2b7fb41338e9-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-85544d4cd7-6dft4\" (UID: \"cc069342-244c-467a-9d89-2b7fb41338e9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-6dft4"
Oct 30 22:16:58.817031 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:58.816987    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-v8cwr\""
Oct 30 22:16:58.817123 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:58.817109    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/85f900f2-2722-4143-8974-d1c7ab81770b-workload-socket\") pod \"maas-default-gateway-openshift-default-85544d4cd7-89h6s\" (UID: \"85f900f2-2722-4143-8974-d1c7ab81770b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-89h6s"
Oct 30 22:16:58.817168 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:58.817155    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cc069342-244c-467a-9d89-2b7fb41338e9-istio-podinfo\") pod \"maas-default-gateway-openshift-default-85544d4cd7-6dft4\" (UID: \"cc069342-244c-467a-9d89-2b7fb41338e9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-6dft4"
Oct 30 22:16:58.817309 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:58.817296    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/85f900f2-2722-4143-8974-d1c7ab81770b-workload-certs\") pod \"maas-default-gateway-openshift-default-85544d4cd7-89h6s\" (UID: \"85f900f2-2722-4143-8974-d1c7ab81770b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-89h6s"
Oct 30 22:16:58.817342 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:58.817320    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/85f900f2-2722-4143-8974-d1c7ab81770b-istio-data\") pod \"maas-default-gateway-openshift-default-85544d4cd7-89h6s\" (UID: \"85f900f2-2722-4143-8974-d1c7ab81770b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-89h6s"
Oct 30 22:16:58.817377 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:58.817338    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cc069342-244c-467a-9d89-2b7fb41338e9-istio-envoy\") pod \"maas-default-gateway-openshift-default-85544d4cd7-6dft4\" (UID: \"cc069342-244c-467a-9d89-2b7fb41338e9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-6dft4"
Oct 30 22:16:58.817410 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:58.817392    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cc069342-244c-467a-9d89-2b7fb41338e9-credential-socket\") pod \"maas-default-gateway-openshift-default-85544d4cd7-6dft4\" (UID: \"cc069342-244c-467a-9d89-2b7fb41338e9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-6dft4"
Oct 30 22:16:58.817447 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:58.817425    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cc069342-244c-467a-9d89-2b7fb41338e9-workload-certs\") pod \"maas-default-gateway-openshift-default-85544d4cd7-6dft4\" (UID: \"cc069342-244c-467a-9d89-2b7fb41338e9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-6dft4"
Oct 30 22:16:58.817481 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:58.817458    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/85f900f2-2722-4143-8974-d1c7ab81770b-istio-podinfo\") pod \"maas-default-gateway-openshift-default-85544d4cd7-89h6s\" (UID: \"85f900f2-2722-4143-8974-d1c7ab81770b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-89h6s"
Oct 30 22:16:58.817481 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:58.817472    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/85f900f2-2722-4143-8974-d1c7ab81770b-istio-token\") pod \"maas-default-gateway-openshift-default-85544d4cd7-89h6s\" (UID: \"85f900f2-2722-4143-8974-d1c7ab81770b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-89h6s"
Oct 30 22:16:58.832900 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:58.832863    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-89h6s"]
Oct 30 22:16:58.884972 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:58.884925    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-6dft4"]
Oct 30 22:16:58.919374 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:58.919339    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cc069342-244c-467a-9d89-2b7fb41338e9-credential-socket\") pod \"maas-default-gateway-openshift-default-85544d4cd7-6dft4\" (UID: \"cc069342-244c-467a-9d89-2b7fb41338e9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-6dft4"
Oct 30 22:16:58.919540 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:58.919424    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cc069342-244c-467a-9d89-2b7fb41338e9-workload-certs\") pod \"maas-default-gateway-openshift-default-85544d4cd7-6dft4\" (UID: \"cc069342-244c-467a-9d89-2b7fb41338e9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-6dft4"
Oct 30 22:16:58.919540 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:58.919483    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/85f900f2-2722-4143-8974-d1c7ab81770b-istio-podinfo\") pod \"maas-default-gateway-openshift-default-85544d4cd7-89h6s\" (UID: \"85f900f2-2722-4143-8974-d1c7ab81770b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-89h6s"
Oct 30 22:16:58.919540 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:58.919517    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/85f900f2-2722-4143-8974-d1c7ab81770b-istio-token\") pod \"maas-default-gateway-openshift-default-85544d4cd7-89h6s\" (UID: \"85f900f2-2722-4143-8974-d1c7ab81770b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-89h6s"
Oct 30 22:16:58.919788 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:58.919599    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/85f900f2-2722-4143-8974-d1c7ab81770b-istio-envoy\") pod \"maas-default-gateway-openshift-default-85544d4cd7-89h6s\" (UID: \"85f900f2-2722-4143-8974-d1c7ab81770b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-89h6s"
Oct 30 22:16:58.919788 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:58.919682    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cc069342-244c-467a-9d89-2b7fb41338e9-istio-data\") pod \"maas-default-gateway-openshift-default-85544d4cd7-6dft4\" (UID: \"cc069342-244c-467a-9d89-2b7fb41338e9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-6dft4"
Oct 30 22:16:58.919788 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:58.919731    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/85f900f2-2722-4143-8974-d1c7ab81770b-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-85544d4cd7-89h6s\" (UID: \"85f900f2-2722-4143-8974-d1c7ab81770b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-89h6s"
Oct 30 22:16:58.919788 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:58.919760    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cc069342-244c-467a-9d89-2b7fb41338e9-workload-socket\") pod \"maas-default-gateway-openshift-default-85544d4cd7-6dft4\" (UID: \"cc069342-244c-467a-9d89-2b7fb41338e9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-6dft4"
Oct 30 22:16:58.920015 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:58.919851    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/85f900f2-2722-4143-8974-d1c7ab81770b-credential-socket\") pod \"maas-default-gateway-openshift-default-85544d4cd7-89h6s\" (UID: \"85f900f2-2722-4143-8974-d1c7ab81770b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-89h6s"
Oct 30 22:16:58.920015 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:58.919880    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cc069342-244c-467a-9d89-2b7fb41338e9-credential-socket\") pod \"maas-default-gateway-openshift-default-85544d4cd7-6dft4\" (UID: \"cc069342-244c-467a-9d89-2b7fb41338e9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-6dft4"
Oct 30 22:16:58.920015 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:58.919899    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cc069342-244c-467a-9d89-2b7fb41338e9-workload-certs\") pod \"maas-default-gateway-openshift-default-85544d4cd7-6dft4\" (UID: \"cc069342-244c-467a-9d89-2b7fb41338e9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-6dft4"
Oct 30 22:16:58.920015 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:58.919912    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wwmj4\" (UniqueName: \"kubernetes.io/projected/85f900f2-2722-4143-8974-d1c7ab81770b-kube-api-access-wwmj4\") pod \"maas-default-gateway-openshift-default-85544d4cd7-89h6s\" (UID: \"85f900f2-2722-4143-8974-d1c7ab81770b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-89h6s"
Oct 30 22:16:58.920232 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:58.920012    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cc069342-244c-467a-9d89-2b7fb41338e9-istio-token\") pod \"maas-default-gateway-openshift-default-85544d4cd7-6dft4\" (UID: \"cc069342-244c-467a-9d89-2b7fb41338e9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-6dft4"
Oct 30 22:16:58.920232 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:58.920051    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-twgnb\" (UniqueName: \"kubernetes.io/projected/cc069342-244c-467a-9d89-2b7fb41338e9-kube-api-access-twgnb\") pod \"maas-default-gateway-openshift-default-85544d4cd7-6dft4\" (UID: \"cc069342-244c-467a-9d89-2b7fb41338e9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-6dft4"
Oct 30 22:16:58.920232 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:58.920142    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cc069342-244c-467a-9d89-2b7fb41338e9-istio-data\") pod \"maas-default-gateway-openshift-default-85544d4cd7-6dft4\" (UID: \"cc069342-244c-467a-9d89-2b7fb41338e9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-6dft4"
Oct 30 22:16:58.920232 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:58.920151    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cc069342-244c-467a-9d89-2b7fb41338e9-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-85544d4cd7-6dft4\" (UID: \"cc069342-244c-467a-9d89-2b7fb41338e9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-6dft4"
Oct 30 22:16:58.920232 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:58.920211    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/85f900f2-2722-4143-8974-d1c7ab81770b-workload-socket\") pod \"maas-default-gateway-openshift-default-85544d4cd7-89h6s\" (UID: \"85f900f2-2722-4143-8974-d1c7ab81770b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-89h6s"
Oct 30 22:16:58.920487 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:58.920242    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cc069342-244c-467a-9d89-2b7fb41338e9-istio-podinfo\") pod \"maas-default-gateway-openshift-default-85544d4cd7-6dft4\" (UID: \"cc069342-244c-467a-9d89-2b7fb41338e9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-6dft4"
Oct 30 22:16:58.920487 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:58.920339    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/85f900f2-2722-4143-8974-d1c7ab81770b-workload-certs\") pod \"maas-default-gateway-openshift-default-85544d4cd7-89h6s\" (UID: \"85f900f2-2722-4143-8974-d1c7ab81770b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-89h6s"
Oct 30 22:16:58.920487 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:58.920370    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/85f900f2-2722-4143-8974-d1c7ab81770b-istio-data\") pod \"maas-default-gateway-openshift-default-85544d4cd7-89h6s\" (UID: \"85f900f2-2722-4143-8974-d1c7ab81770b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-89h6s"
Oct 30 22:16:58.920487 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:58.920407    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cc069342-244c-467a-9d89-2b7fb41338e9-istio-envoy\") pod \"maas-default-gateway-openshift-default-85544d4cd7-6dft4\" (UID: \"cc069342-244c-467a-9d89-2b7fb41338e9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-6dft4"
Oct 30 22:16:58.920487 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:58.920472    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/85f900f2-2722-4143-8974-d1c7ab81770b-credential-socket\") pod \"maas-default-gateway-openshift-default-85544d4cd7-89h6s\" (UID: \"85f900f2-2722-4143-8974-d1c7ab81770b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-89h6s"
Oct 30 22:16:58.920763 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:58.920539    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/85f900f2-2722-4143-8974-d1c7ab81770b-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-85544d4cd7-89h6s\" (UID: \"85f900f2-2722-4143-8974-d1c7ab81770b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-89h6s"
Oct 30 22:16:58.920865 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:58.920842    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/85f900f2-2722-4143-8974-d1c7ab81770b-workload-certs\") pod \"maas-default-gateway-openshift-default-85544d4cd7-89h6s\" (UID: \"85f900f2-2722-4143-8974-d1c7ab81770b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-89h6s"
Oct 30 22:16:58.920931 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:58.920850    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cc069342-244c-467a-9d89-2b7fb41338e9-workload-socket\") pod \"maas-default-gateway-openshift-default-85544d4cd7-6dft4\" (UID: \"cc069342-244c-467a-9d89-2b7fb41338e9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-6dft4"
Oct 30 22:16:58.921102 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:58.921082    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/85f900f2-2722-4143-8974-d1c7ab81770b-workload-socket\") pod \"maas-default-gateway-openshift-default-85544d4cd7-89h6s\" (UID: \"85f900f2-2722-4143-8974-d1c7ab81770b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-89h6s"
Oct 30 22:16:58.921262 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:58.921237    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/85f900f2-2722-4143-8974-d1c7ab81770b-istio-data\") pod \"maas-default-gateway-openshift-default-85544d4cd7-89h6s\" (UID: \"85f900f2-2722-4143-8974-d1c7ab81770b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-89h6s"
Oct 30 22:16:58.921415 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:58.921249    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cc069342-244c-467a-9d89-2b7fb41338e9-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-85544d4cd7-6dft4\" (UID: \"cc069342-244c-467a-9d89-2b7fb41338e9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-6dft4"
Oct 30 22:16:58.923438 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:58.923412    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/85f900f2-2722-4143-8974-d1c7ab81770b-istio-envoy\") pod \"maas-default-gateway-openshift-default-85544d4cd7-89h6s\" (UID: \"85f900f2-2722-4143-8974-d1c7ab81770b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-89h6s"
Oct 30 22:16:58.923785 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:58.923761    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cc069342-244c-467a-9d89-2b7fb41338e9-istio-envoy\") pod \"maas-default-gateway-openshift-default-85544d4cd7-6dft4\" (UID: \"cc069342-244c-467a-9d89-2b7fb41338e9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-6dft4"
Oct 30 22:16:58.924063 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:58.924031    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/85f900f2-2722-4143-8974-d1c7ab81770b-istio-podinfo\") pod \"maas-default-gateway-openshift-default-85544d4cd7-89h6s\" (UID: \"85f900f2-2722-4143-8974-d1c7ab81770b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-89h6s"
Oct 30 22:16:58.924063 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:58.924053    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cc069342-244c-467a-9d89-2b7fb41338e9-istio-podinfo\") pod \"maas-default-gateway-openshift-default-85544d4cd7-6dft4\" (UID: \"cc069342-244c-467a-9d89-2b7fb41338e9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-6dft4"
Oct 30 22:16:58.984468 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:58.984391    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/85f900f2-2722-4143-8974-d1c7ab81770b-istio-token\") pod \"maas-default-gateway-openshift-default-85544d4cd7-89h6s\" (UID: \"85f900f2-2722-4143-8974-d1c7ab81770b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-89h6s"
Oct 30 22:16:58.984866 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:58.984843    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wwmj4\" (UniqueName: \"kubernetes.io/projected/85f900f2-2722-4143-8974-d1c7ab81770b-kube-api-access-wwmj4\") pod \"maas-default-gateway-openshift-default-85544d4cd7-89h6s\" (UID: \"85f900f2-2722-4143-8974-d1c7ab81770b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-89h6s"
Oct 30 22:16:59.006811 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:59.006770    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-twgnb\" (UniqueName: \"kubernetes.io/projected/cc069342-244c-467a-9d89-2b7fb41338e9-kube-api-access-twgnb\") pod \"maas-default-gateway-openshift-default-85544d4cd7-6dft4\" (UID: \"cc069342-244c-467a-9d89-2b7fb41338e9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-6dft4"
Oct 30 22:16:59.007872 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:59.007845    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cc069342-244c-467a-9d89-2b7fb41338e9-istio-token\") pod \"maas-default-gateway-openshift-default-85544d4cd7-6dft4\" (UID: \"cc069342-244c-467a-9d89-2b7fb41338e9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-6dft4"
Oct 30 22:16:59.042211 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:59.042178    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-89h6s"
Oct 30 22:16:59.088637 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:59.088606    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-6dft4"
Oct 30 22:16:59.343983 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:59.342578    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:16:59.478436 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:59.478404    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-89h6s"]
Oct 30 22:16:59.481521 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:59.481482    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:16:59.481615 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:59.481553    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:16:59.481615 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:59.481594    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:16:59.494952 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:59.494917    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-89h6s" event={"ID":"85f900f2-2722-4143-8974-d1c7ab81770b","Type":"ContainerStarted","Data":"62dd34e3fde3204018c7f692174b98fcfdc439a4d658339544915025ae14de8f"}
Oct 30 22:16:59.535172 ip-10-0-136-210 kubenswrapper[5510]: W1030 22:16:59.535135    5510 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc069342_244c_467a_9d89_2b7fb41338e9.slice/crio-f7aab8caef859c88ad85688eca7c4b008b7040cc4705e2d80067e64973be5e31 WatchSource:0}: Error finding container f7aab8caef859c88ad85688eca7c4b008b7040cc4705e2d80067e64973be5e31: Status 404 returned error can't find the container with id f7aab8caef859c88ad85688eca7c4b008b7040cc4705e2d80067e64973be5e31
Oct 30 22:16:59.537446 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:59.537424    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-6dft4"]
Oct 30 22:16:59.538607 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:59.538568    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:16:59.538753 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:59.538635    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:16:59.538753 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:16:59.538705    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:17:00.076689 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:00.074220    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:17:00.298986 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:00.298295    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:17:00.508711 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:00.508614    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-6dft4" event={"ID":"cc069342-244c-467a-9d89-2b7fb41338e9","Type":"ContainerStarted","Data":"eda1bf976520f473dae4c2d09c2e060f7ddc72c5b16ba293ca1ffbe62b420107"}
Oct 30 22:17:00.508711 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:00.508685    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-6dft4" event={"ID":"cc069342-244c-467a-9d89-2b7fb41338e9","Type":"ContainerStarted","Data":"f7aab8caef859c88ad85688eca7c4b008b7040cc4705e2d80067e64973be5e31"}
Oct 30 22:17:00.511277 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:00.511245    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-89h6s" event={"ID":"85f900f2-2722-4143-8974-d1c7ab81770b","Type":"ContainerStarted","Data":"734bd0c319d99e4c352fe98062c5d98140fcaf5ac298096be7c343827d2ece60"}
Oct 30 22:17:00.643097 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:00.643042    5510 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-6dft4" podStartSLOduration=2.643023408 podStartE2EDuration="2.643023408s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:16:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:17:00.640845126 +0000 UTC m=+1601.201655131" watchObservedRunningTime="2026-10-30 22:17:00.643023408 +0000 UTC m=+1601.203833413"
Oct 30 22:17:00.801763 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:00.801693    5510 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-89h6s" podStartSLOduration=2.801676548 podStartE2EDuration="2.801676548s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:16:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:17:00.798804728 +0000 UTC m=+1601.359614713" watchObservedRunningTime="2026-10-30 22:17:00.801676548 +0000 UTC m=+1601.362486553"
Oct 30 22:17:01.043320 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:01.043286    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-89h6s"
Oct 30 22:17:01.069742 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:01.063722    5510 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-85544d4cd7-89h6s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.96:15021/healthz/ready\": dial tcp 10.133.0.96:15021: connect: connection refused" start-of-body=
Oct 30 22:17:01.069742 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:01.063804    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-89h6s" podUID="85f900f2-2722-4143-8974-d1c7ab81770b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.96:15021/healthz/ready\": dial tcp 10.133.0.96:15021: connect: connection refused"
Oct 30 22:17:01.098039 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:01.096115    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-6dft4"
Oct 30 22:17:01.098523 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:01.098163    5510 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-85544d4cd7-6dft4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.97:15021/healthz/ready\": dial tcp 10.133.0.97:15021: connect: connection refused" start-of-body=
Oct 30 22:17:01.098523 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:01.098241    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-6dft4" podUID="cc069342-244c-467a-9d89-2b7fb41338e9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.97:15021/healthz/ready\": dial tcp 10.133.0.97:15021: connect: connection refused"
Oct 30 22:17:02.043162 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:02.043124    5510 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-85544d4cd7-89h6s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.96:15021/healthz/ready\": dial tcp 10.133.0.96:15021: connect: connection refused" start-of-body=
Oct 30 22:17:02.043348 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:02.043177    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-89h6s" podUID="85f900f2-2722-4143-8974-d1c7ab81770b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.96:15021/healthz/ready\": dial tcp 10.133.0.96:15021: connect: connection refused"
Oct 30 22:17:02.089683 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:02.089021    5510 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-85544d4cd7-6dft4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.97:15021/healthz/ready\": dial tcp 10.133.0.97:15021: connect: connection refused" start-of-body=
Oct 30 22:17:02.089683 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:02.089081    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-6dft4" podUID="cc069342-244c-467a-9d89-2b7fb41338e9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.97:15021/healthz/ready\": dial tcp 10.133.0.97:15021: connect: connection refused"
Oct 30 22:17:03.042914 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:03.042876    5510 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-85544d4cd7-89h6s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.96:15021/healthz/ready\": dial tcp 10.133.0.96:15021: connect: connection refused" start-of-body=
Oct 30 22:17:03.043394 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:03.042935    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-89h6s" podUID="85f900f2-2722-4143-8974-d1c7ab81770b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.96:15021/healthz/ready\": dial tcp 10.133.0.96:15021: connect: connection refused"
Oct 30 22:17:03.093681 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:03.093199    5510 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-85544d4cd7-6dft4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.97:15021/healthz/ready\": dial tcp 10.133.0.97:15021: connect: connection refused" start-of-body=
Oct 30 22:17:03.093681 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:03.093265    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-6dft4" podUID="cc069342-244c-467a-9d89-2b7fb41338e9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.97:15021/healthz/ready\": dial tcp 10.133.0.97:15021: connect: connection refused"
Oct 30 22:17:04.056802 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:04.054632    5510 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-85544d4cd7-89h6s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.96:15021/healthz/ready\": dial tcp 10.133.0.96:15021: connect: connection refused" start-of-body=
Oct 30 22:17:04.056802 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:04.054721    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-89h6s" podUID="85f900f2-2722-4143-8974-d1c7ab81770b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.96:15021/healthz/ready\": dial tcp 10.133.0.96:15021: connect: connection refused"
Oct 30 22:17:04.099683 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:04.093470    5510 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-85544d4cd7-6dft4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.97:15021/healthz/ready\": dial tcp 10.133.0.97:15021: connect: connection refused" start-of-body=
Oct 30 22:17:04.099683 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:04.093531    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-6dft4" podUID="cc069342-244c-467a-9d89-2b7fb41338e9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.97:15021/healthz/ready\": dial tcp 10.133.0.97:15021: connect: connection refused"
Oct 30 22:17:04.821680 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:04.816629    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:17:05.040680 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:05.027801    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:17:05.063678 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:05.043606    5510 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-85544d4cd7-89h6s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.96:15021/healthz/ready\": dial tcp 10.133.0.96:15021: connect: connection refused" start-of-body=
Oct 30 22:17:05.063678 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:05.043721    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-89h6s" podUID="85f900f2-2722-4143-8974-d1c7ab81770b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.96:15021/healthz/ready\": dial tcp 10.133.0.96:15021: connect: connection refused"
Oct 30 22:17:05.089741 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:05.085432    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-67184f2a-66ffdf88d7-wd7x9"]
Oct 30 22:17:05.106735 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:05.099757    5510 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-85544d4cd7-6dft4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.97:15021/healthz/ready\": dial tcp 10.133.0.97:15021: connect: connection refused" start-of-body=
Oct 30 22:17:05.106735 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:05.099826    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-6dft4" podUID="cc069342-244c-467a-9d89-2b7fb41338e9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.97:15021/healthz/ready\": dial tcp 10.133.0.97:15021: connect: connection refused"
Oct 30 22:17:05.136148 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:05.136115    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-67184f2a-6fc5c46555-n47rx"]
Oct 30 22:17:05.603707 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:05.571045    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:17:05.926323 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:05.926236    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:17:06.043043 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:06.043008    5510 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-85544d4cd7-89h6s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.96:15021/healthz/ready\": dial tcp 10.133.0.96:15021: connect: connection refused" start-of-body=
Oct 30 22:17:06.043232 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:06.043120    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-89h6s" podUID="85f900f2-2722-4143-8974-d1c7ab81770b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.96:15021/healthz/ready\": dial tcp 10.133.0.96:15021: connect: connection refused"
Oct 30 22:17:06.089923 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:06.089884    5510 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-85544d4cd7-6dft4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.97:15021/healthz/ready\": dial tcp 10.133.0.97:15021: connect: connection refused" start-of-body=
Oct 30 22:17:06.090410 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:06.089951    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-6dft4" podUID="cc069342-244c-467a-9d89-2b7fb41338e9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.97:15021/healthz/ready\": dial tcp 10.133.0.97:15021: connect: connection refused"
Oct 30 22:17:07.049715 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:07.045384    5510 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-85544d4cd7-89h6s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.96:15021/healthz/ready\": dial tcp 10.133.0.96:15021: connect: connection refused" start-of-body=
Oct 30 22:17:07.049715 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:07.045444    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-89h6s" podUID="85f900f2-2722-4143-8974-d1c7ab81770b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.96:15021/healthz/ready\": dial tcp 10.133.0.96:15021: connect: connection refused"
Oct 30 22:17:07.097682 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:07.092043    5510 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-85544d4cd7-6dft4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.97:15021/healthz/ready\": dial tcp 10.133.0.97:15021: connect: connection refused" start-of-body=
Oct 30 22:17:07.097682 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:07.092105    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-6dft4" podUID="cc069342-244c-467a-9d89-2b7fb41338e9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.97:15021/healthz/ready\": dial tcp 10.133.0.97:15021: connect: connection refused"
Oct 30 22:17:08.043535 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:08.043501    5510 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-85544d4cd7-89h6s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.96:15021/healthz/ready\": dial tcp 10.133.0.96:15021: connect: connection refused" start-of-body=
Oct 30 22:17:08.043769 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:08.043559    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-89h6s" podUID="85f900f2-2722-4143-8974-d1c7ab81770b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.96:15021/healthz/ready\": dial tcp 10.133.0.96:15021: connect: connection refused"
Oct 30 22:17:08.090211 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:08.090170    5510 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-85544d4cd7-6dft4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.97:15021/healthz/ready\": dial tcp 10.133.0.97:15021: connect: connection refused" start-of-body=
Oct 30 22:17:08.090395 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:08.090242    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-6dft4" podUID="cc069342-244c-467a-9d89-2b7fb41338e9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.97:15021/healthz/ready\": dial tcp 10.133.0.97:15021: connect: connection refused"
Oct 30 22:17:08.441423 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:17:08.441294    5510 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6gjbf" containerName="extractor"
Oct 30 22:17:09.095409 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:09.095377    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-6dft4"
Oct 30 22:17:09.096409 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:09.096382    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-6dft4"
Oct 30 22:17:09.097024 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:09.097006    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-6dft4"
Oct 30 22:17:09.100808 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:09.100772    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-89h6s"
Oct 30 22:17:09.102303 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:09.102285    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-89h6s"
Oct 30 22:17:09.103476 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:09.103458    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-89h6s"
Oct 30 22:17:10.086684 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:10.086623    5510 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-migrate-67184f2a-66ffdf88d7-wd7x9" podUID="11d70426-1070-491a-8b69-d107582114f0" containerName="payload-processing" containerID="cri-o://1b46aa79142b706cba5223a89e6fcb4c379342f8fcb65dcb66d820deddc8f3e6" gracePeriod=25
Oct 30 22:17:10.136524 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:10.136481    5510 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-migrate-67184f2a-6fc5c46555-n47rx" podUID="d3670624-670c-4d8c-9444-3dcdde37c2b0" containerName="payload-pre-processing" containerID="cri-o://e1944240dfe35a962f87367ada2ac4c19067f1db5694e4a81fb3bd90cd32aa78" gracePeriod=25
Oct 30 22:17:10.432049 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:10.404702    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:17:10.570352 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:10.570325    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-67184f2a-66ffdf88d7-wd7x9"
Oct 30 22:17:10.600084 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:10.600048    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/11d70426-1070-491a-8b69-d107582114f0-kube-api-access-tvcxd\" (UniqueName: \"kubernetes.io/projected/11d70426-1070-491a-8b69-d107582114f0-kube-api-access-tvcxd\") pod \"11d70426-1070-491a-8b69-d107582114f0\" (UID: \"11d70426-1070-491a-8b69-d107582114f0\") "
Oct 30 22:17:10.600283 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:10.600102    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/11d70426-1070-491a-8b69-d107582114f0-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/11d70426-1070-491a-8b69-d107582114f0-plugins-config-volume\") pod \"11d70426-1070-491a-8b69-d107582114f0\" (UID: \"11d70426-1070-491a-8b69-d107582114f0\") "
Oct 30 22:17:10.600539 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:10.600512    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/11d70426-1070-491a-8b69-d107582114f0-plugins-config-volume" pod "11d70426-1070-491a-8b69-d107582114f0" (UID: "11d70426-1070-491a-8b69-d107582114f0"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:17:10.600912 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:10.600890    5510 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/11d70426-1070-491a-8b69-d107582114f0-plugins-config-volume\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:17:10.603152 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:10.603129    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/11d70426-1070-491a-8b69-d107582114f0-kube-api-access-tvcxd" pod "11d70426-1070-491a-8b69-d107582114f0" (UID: "11d70426-1070-491a-8b69-d107582114f0"). InnerVolumeSpecName "kube-api-access-tvcxd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:17:10.628569 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:10.628549    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-67184f2a-6fc5c46555-n47rx"
Oct 30 22:17:10.701595 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:10.701505    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d3670624-670c-4d8c-9444-3dcdde37c2b0-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d3670624-670c-4d8c-9444-3dcdde37c2b0-plugins-config-volume\") pod \"d3670624-670c-4d8c-9444-3dcdde37c2b0\" (UID: \"d3670624-670c-4d8c-9444-3dcdde37c2b0\") "
Oct 30 22:17:10.701804 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:10.701651    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d3670624-670c-4d8c-9444-3dcdde37c2b0-kube-api-access-2xmjx\" (UniqueName: \"kubernetes.io/projected/d3670624-670c-4d8c-9444-3dcdde37c2b0-kube-api-access-2xmjx\") pod \"d3670624-670c-4d8c-9444-3dcdde37c2b0\" (UID: \"d3670624-670c-4d8c-9444-3dcdde37c2b0\") "
Oct 30 22:17:10.701949 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:10.701919    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d3670624-670c-4d8c-9444-3dcdde37c2b0-plugins-config-volume" pod "d3670624-670c-4d8c-9444-3dcdde37c2b0" (UID: "d3670624-670c-4d8c-9444-3dcdde37c2b0"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:17:10.702506 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:10.702480    5510 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d3670624-670c-4d8c-9444-3dcdde37c2b0-plugins-config-volume\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:17:10.702506 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:10.702503    5510 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tvcxd\" (UniqueName: \"kubernetes.io/projected/11d70426-1070-491a-8b69-d107582114f0-kube-api-access-tvcxd\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:17:10.703946 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:10.703918    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d3670624-670c-4d8c-9444-3dcdde37c2b0-kube-api-access-2xmjx" pod "d3670624-670c-4d8c-9444-3dcdde37c2b0" (UID: "d3670624-670c-4d8c-9444-3dcdde37c2b0"). InnerVolumeSpecName "kube-api-access-2xmjx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:17:10.804778 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:10.804741    5510 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2xmjx\" (UniqueName: \"kubernetes.io/projected/d3670624-670c-4d8c-9444-3dcdde37c2b0-kube-api-access-2xmjx\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:17:11.000831 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:11.000740    5510 generic.go:356] "Generic (PLEG): container finished" podID="d3670624-670c-4d8c-9444-3dcdde37c2b0" containerID="e1944240dfe35a962f87367ada2ac4c19067f1db5694e4a81fb3bd90cd32aa78" exitCode=0
Oct 30 22:17:11.000831 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:11.000788    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-67184f2a-6fc5c46555-n47rx"
Oct 30 22:17:11.000831 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:11.000817    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-67184f2a-6fc5c46555-n47rx" event={"ID":"d3670624-670c-4d8c-9444-3dcdde37c2b0","Type":"ContainerDied","Data":"e1944240dfe35a962f87367ada2ac4c19067f1db5694e4a81fb3bd90cd32aa78"}
Oct 30 22:17:11.001083 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:11.000869    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-67184f2a-6fc5c46555-n47rx" event={"ID":"d3670624-670c-4d8c-9444-3dcdde37c2b0","Type":"ContainerDied","Data":"bc61321647803fcf46315cc145469ed2878be8239b0a672967db8cb70e9a43bc"}
Oct 30 22:17:11.001083 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:11.000890    5510 scope.go:122] "RemoveContainer" containerID="e1944240dfe35a962f87367ada2ac4c19067f1db5694e4a81fb3bd90cd32aa78"
Oct 30 22:17:11.002447 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:11.002424    5510 generic.go:356] "Generic (PLEG): container finished" podID="11d70426-1070-491a-8b69-d107582114f0" containerID="1b46aa79142b706cba5223a89e6fcb4c379342f8fcb65dcb66d820deddc8f3e6" exitCode=0
Oct 30 22:17:11.002588 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:11.002450    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-67184f2a-66ffdf88d7-wd7x9" event={"ID":"11d70426-1070-491a-8b69-d107582114f0","Type":"ContainerDied","Data":"1b46aa79142b706cba5223a89e6fcb4c379342f8fcb65dcb66d820deddc8f3e6"}
Oct 30 22:17:11.002588 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:11.002473    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-67184f2a-66ffdf88d7-wd7x9"
Oct 30 22:17:11.002588 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:11.002497    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-67184f2a-66ffdf88d7-wd7x9" event={"ID":"11d70426-1070-491a-8b69-d107582114f0","Type":"ContainerDied","Data":"139159db2676a62fc140505ccec09ced04e2a255f23072e4de3c86da852a35c1"}
Oct 30 22:17:11.012625 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:11.012608    5510 scope.go:122] "RemoveContainer" containerID="e1944240dfe35a962f87367ada2ac4c19067f1db5694e4a81fb3bd90cd32aa78"
Oct 30 22:17:11.012927 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:17:11.012908    5510 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e1944240dfe35a962f87367ada2ac4c19067f1db5694e4a81fb3bd90cd32aa78\": container with ID starting with e1944240dfe35a962f87367ada2ac4c19067f1db5694e4a81fb3bd90cd32aa78 not found: ID does not exist" containerID="e1944240dfe35a962f87367ada2ac4c19067f1db5694e4a81fb3bd90cd32aa78"
Oct 30 22:17:11.012977 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:11.012933    5510 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e1944240dfe35a962f87367ada2ac4c19067f1db5694e4a81fb3bd90cd32aa78"} err="failed to get container status \"e1944240dfe35a962f87367ada2ac4c19067f1db5694e4a81fb3bd90cd32aa78\": rpc error: code = NotFound desc = could not find container \"e1944240dfe35a962f87367ada2ac4c19067f1db5694e4a81fb3bd90cd32aa78\": container with ID starting with e1944240dfe35a962f87367ada2ac4c19067f1db5694e4a81fb3bd90cd32aa78 not found: ID does not exist"
Oct 30 22:17:11.012977 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:11.012948    5510 scope.go:122] "RemoveContainer" containerID="1b46aa79142b706cba5223a89e6fcb4c379342f8fcb65dcb66d820deddc8f3e6"
Oct 30 22:17:11.022577 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:11.022556    5510 scope.go:122] "RemoveContainer" containerID="1b46aa79142b706cba5223a89e6fcb4c379342f8fcb65dcb66d820deddc8f3e6"
Oct 30 22:17:11.022914 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:17:11.022890    5510 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1b46aa79142b706cba5223a89e6fcb4c379342f8fcb65dcb66d820deddc8f3e6\": container with ID starting with 1b46aa79142b706cba5223a89e6fcb4c379342f8fcb65dcb66d820deddc8f3e6 not found: ID does not exist" containerID="1b46aa79142b706cba5223a89e6fcb4c379342f8fcb65dcb66d820deddc8f3e6"
Oct 30 22:17:11.023011 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:11.022920    5510 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1b46aa79142b706cba5223a89e6fcb4c379342f8fcb65dcb66d820deddc8f3e6"} err="failed to get container status \"1b46aa79142b706cba5223a89e6fcb4c379342f8fcb65dcb66d820deddc8f3e6\": rpc error: code = NotFound desc = could not find container \"1b46aa79142b706cba5223a89e6fcb4c379342f8fcb65dcb66d820deddc8f3e6\": container with ID starting with 1b46aa79142b706cba5223a89e6fcb4c379342f8fcb65dcb66d820deddc8f3e6 not found: ID does not exist"
Oct 30 22:17:11.170981 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:11.170949    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-67184f2a-66ffdf88d7-wd7x9"]
Oct 30 22:17:11.175424 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:11.175393    5510 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-67184f2a-66ffdf88d7-wd7x9"]
Oct 30 22:17:11.288669 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:11.288617    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-67184f2a-6fc5c46555-n47rx"]
Oct 30 22:17:11.294032 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:11.294003    5510 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-67184f2a-6fc5c46555-n47rx"]
Oct 30 22:17:11.558682 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:11.558569    5510 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="11d70426-1070-491a-8b69-d107582114f0" path="/var/lib/kubelet/pods/11d70426-1070-491a-8b69-d107582114f0/volumes"
Oct 30 22:17:11.559252 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:11.559231    5510 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d3670624-670c-4d8c-9444-3dcdde37c2b0" path="/var/lib/kubelet/pods/d3670624-670c-4d8c-9444-3dcdde37c2b0/volumes"
Oct 30 22:17:13.853946 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:13.853914    5510 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-xv7c7"]
Oct 30 22:17:13.854718 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:13.854701    5510 cpu_manager.go:414] "containerMap: removing container" podUID="11d70426-1070-491a-8b69-d107582114f0" containerName="payload-processing"
Oct 30 22:17:13.854718 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:13.854718    5510 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="11d70426-1070-491a-8b69-d107582114f0" containerName="payload-processing"
Oct 30 22:17:13.854815 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:13.854733    5510 cpu_manager.go:414] "containerMap: removing container" podUID="d3670624-670c-4d8c-9444-3dcdde37c2b0" containerName="payload-pre-processing"
Oct 30 22:17:13.854815 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:13.854739    5510 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d3670624-670c-4d8c-9444-3dcdde37c2b0" containerName="payload-pre-processing"
Oct 30 22:17:13.854876 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:13.854839    5510 memory_manager.go:365] "RemoveStaleState removing state" podUID="11d70426-1070-491a-8b69-d107582114f0" containerName="payload-processing"
Oct 30 22:17:13.854876 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:13.854847    5510 memory_manager.go:365] "RemoveStaleState removing state" podUID="d3670624-670c-4d8c-9444-3dcdde37c2b0" containerName="payload-pre-processing"
Oct 30 22:17:13.858817 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:13.858799    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-xv7c7"
Oct 30 22:17:13.954848 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:13.954812    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/21c0fef2-49cd-4734-b402-f1cdeb031ac8-istio-envoy\") pod \"maas-default-gateway-openshift-default-85544d4cd7-xv7c7\" (UID: \"21c0fef2-49cd-4734-b402-f1cdeb031ac8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-xv7c7"
Oct 30 22:17:13.954848 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:13.954858    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/21c0fef2-49cd-4734-b402-f1cdeb031ac8-workload-certs\") pod \"maas-default-gateway-openshift-default-85544d4cd7-xv7c7\" (UID: \"21c0fef2-49cd-4734-b402-f1cdeb031ac8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-xv7c7"
Oct 30 22:17:13.955143 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:13.954959    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/21c0fef2-49cd-4734-b402-f1cdeb031ac8-credential-socket\") pod \"maas-default-gateway-openshift-default-85544d4cd7-xv7c7\" (UID: \"21c0fef2-49cd-4734-b402-f1cdeb031ac8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-xv7c7"
Oct 30 22:17:13.955143 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:13.955026    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/21c0fef2-49cd-4734-b402-f1cdeb031ac8-istio-data\") pod \"maas-default-gateway-openshift-default-85544d4cd7-xv7c7\" (UID: \"21c0fef2-49cd-4734-b402-f1cdeb031ac8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-xv7c7"
Oct 30 22:17:13.955143 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:13.955055    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/21c0fef2-49cd-4734-b402-f1cdeb031ac8-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-85544d4cd7-xv7c7\" (UID: \"21c0fef2-49cd-4734-b402-f1cdeb031ac8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-xv7c7"
Oct 30 22:17:13.955276 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:13.955219    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/21c0fef2-49cd-4734-b402-f1cdeb031ac8-istio-token\") pod \"maas-default-gateway-openshift-default-85544d4cd7-xv7c7\" (UID: \"21c0fef2-49cd-4734-b402-f1cdeb031ac8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-xv7c7"
Oct 30 22:17:13.955276 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:13.955244    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ltb4x\" (UniqueName: \"kubernetes.io/projected/21c0fef2-49cd-4734-b402-f1cdeb031ac8-kube-api-access-ltb4x\") pod \"maas-default-gateway-openshift-default-85544d4cd7-xv7c7\" (UID: \"21c0fef2-49cd-4734-b402-f1cdeb031ac8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-xv7c7"
Oct 30 22:17:13.955372 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:13.955353    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/21c0fef2-49cd-4734-b402-f1cdeb031ac8-workload-socket\") pod \"maas-default-gateway-openshift-default-85544d4cd7-xv7c7\" (UID: \"21c0fef2-49cd-4734-b402-f1cdeb031ac8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-xv7c7"
Oct 30 22:17:13.955415 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:13.955381    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/21c0fef2-49cd-4734-b402-f1cdeb031ac8-istio-podinfo\") pod \"maas-default-gateway-openshift-default-85544d4cd7-xv7c7\" (UID: \"21c0fef2-49cd-4734-b402-f1cdeb031ac8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-xv7c7"
Oct 30 22:17:13.968269 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:13.968225    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-xv7c7"]
Oct 30 22:17:14.057319 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:14.057279    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/21c0fef2-49cd-4734-b402-f1cdeb031ac8-credential-socket\") pod \"maas-default-gateway-openshift-default-85544d4cd7-xv7c7\" (UID: \"21c0fef2-49cd-4734-b402-f1cdeb031ac8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-xv7c7"
Oct 30 22:17:14.057514 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:14.057403    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/21c0fef2-49cd-4734-b402-f1cdeb031ac8-istio-data\") pod \"maas-default-gateway-openshift-default-85544d4cd7-xv7c7\" (UID: \"21c0fef2-49cd-4734-b402-f1cdeb031ac8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-xv7c7"
Oct 30 22:17:14.057514 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:14.057444    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/21c0fef2-49cd-4734-b402-f1cdeb031ac8-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-85544d4cd7-xv7c7\" (UID: \"21c0fef2-49cd-4734-b402-f1cdeb031ac8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-xv7c7"
Oct 30 22:17:14.057645 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:14.057567    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/21c0fef2-49cd-4734-b402-f1cdeb031ac8-istio-token\") pod \"maas-default-gateway-openshift-default-85544d4cd7-xv7c7\" (UID: \"21c0fef2-49cd-4734-b402-f1cdeb031ac8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-xv7c7"
Oct 30 22:17:14.057645 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:14.057604    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ltb4x\" (UniqueName: \"kubernetes.io/projected/21c0fef2-49cd-4734-b402-f1cdeb031ac8-kube-api-access-ltb4x\") pod \"maas-default-gateway-openshift-default-85544d4cd7-xv7c7\" (UID: \"21c0fef2-49cd-4734-b402-f1cdeb031ac8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-xv7c7"
Oct 30 22:17:14.057802 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:14.057764    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/21c0fef2-49cd-4734-b402-f1cdeb031ac8-workload-socket\") pod \"maas-default-gateway-openshift-default-85544d4cd7-xv7c7\" (UID: \"21c0fef2-49cd-4734-b402-f1cdeb031ac8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-xv7c7"
Oct 30 22:17:14.057859 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:14.057798    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/21c0fef2-49cd-4734-b402-f1cdeb031ac8-istio-podinfo\") pod \"maas-default-gateway-openshift-default-85544d4cd7-xv7c7\" (UID: \"21c0fef2-49cd-4734-b402-f1cdeb031ac8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-xv7c7"
Oct 30 22:17:14.057859 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:14.057817    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/21c0fef2-49cd-4734-b402-f1cdeb031ac8-istio-data\") pod \"maas-default-gateway-openshift-default-85544d4cd7-xv7c7\" (UID: \"21c0fef2-49cd-4734-b402-f1cdeb031ac8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-xv7c7"
Oct 30 22:17:14.058032 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:14.058014    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/21c0fef2-49cd-4734-b402-f1cdeb031ac8-istio-envoy\") pod \"maas-default-gateway-openshift-default-85544d4cd7-xv7c7\" (UID: \"21c0fef2-49cd-4734-b402-f1cdeb031ac8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-xv7c7"
Oct 30 22:17:14.058085 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:14.058061    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/21c0fef2-49cd-4734-b402-f1cdeb031ac8-workload-certs\") pod \"maas-default-gateway-openshift-default-85544d4cd7-xv7c7\" (UID: \"21c0fef2-49cd-4734-b402-f1cdeb031ac8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-xv7c7"
Oct 30 22:17:14.058140 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:14.058076    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/21c0fef2-49cd-4734-b402-f1cdeb031ac8-workload-socket\") pod \"maas-default-gateway-openshift-default-85544d4cd7-xv7c7\" (UID: \"21c0fef2-49cd-4734-b402-f1cdeb031ac8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-xv7c7"
Oct 30 22:17:14.058257 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:14.058236    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/21c0fef2-49cd-4734-b402-f1cdeb031ac8-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-85544d4cd7-xv7c7\" (UID: \"21c0fef2-49cd-4734-b402-f1cdeb031ac8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-xv7c7"
Oct 30 22:17:14.058378 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:14.058358    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/21c0fef2-49cd-4734-b402-f1cdeb031ac8-credential-socket\") pod \"maas-default-gateway-openshift-default-85544d4cd7-xv7c7\" (UID: \"21c0fef2-49cd-4734-b402-f1cdeb031ac8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-xv7c7"
Oct 30 22:17:14.058591 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:14.058559    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/21c0fef2-49cd-4734-b402-f1cdeb031ac8-workload-certs\") pod \"maas-default-gateway-openshift-default-85544d4cd7-xv7c7\" (UID: \"21c0fef2-49cd-4734-b402-f1cdeb031ac8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-xv7c7"
Oct 30 22:17:14.060300 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:14.060278    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/21c0fef2-49cd-4734-b402-f1cdeb031ac8-istio-envoy\") pod \"maas-default-gateway-openshift-default-85544d4cd7-xv7c7\" (UID: \"21c0fef2-49cd-4734-b402-f1cdeb031ac8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-xv7c7"
Oct 30 22:17:14.060778 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:14.060757    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/21c0fef2-49cd-4734-b402-f1cdeb031ac8-istio-podinfo\") pod \"maas-default-gateway-openshift-default-85544d4cd7-xv7c7\" (UID: \"21c0fef2-49cd-4734-b402-f1cdeb031ac8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-xv7c7"
Oct 30 22:17:14.120024 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:14.119949    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/21c0fef2-49cd-4734-b402-f1cdeb031ac8-istio-token\") pod \"maas-default-gateway-openshift-default-85544d4cd7-xv7c7\" (UID: \"21c0fef2-49cd-4734-b402-f1cdeb031ac8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-xv7c7"
Oct 30 22:17:14.120182 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:14.120156    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ltb4x\" (UniqueName: \"kubernetes.io/projected/21c0fef2-49cd-4734-b402-f1cdeb031ac8-kube-api-access-ltb4x\") pod \"maas-default-gateway-openshift-default-85544d4cd7-xv7c7\" (UID: \"21c0fef2-49cd-4734-b402-f1cdeb031ac8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-xv7c7"
Oct 30 22:17:14.157066 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:14.157035    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-ff716ef6-f89c8bd9c-mhgqb"]
Oct 30 22:17:14.172244 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:14.172215    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-xv7c7"
Oct 30 22:17:14.273585 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:14.272392    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-ff716ef6-5574d46599-svwhz"]
Oct 30 22:17:14.600639 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:14.600597    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:17:14.623861 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:14.623812    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:17:14.624576 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:14.624525    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:17:14.624748 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:14.624596    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:17:14.627623 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:14.627598    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-xv7c7"]
Oct 30 22:17:14.818689 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:14.813093    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:17:15.041069 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:15.041032    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-xv7c7" event={"ID":"21c0fef2-49cd-4734-b402-f1cdeb031ac8","Type":"ContainerStarted","Data":"4a7ab40fdddf291e6c6022b41eb55c5dcb262680b18dc93ef9a1447cde8a7616"}
Oct 30 22:17:15.041451 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:15.041092    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-xv7c7" event={"ID":"21c0fef2-49cd-4734-b402-f1cdeb031ac8","Type":"ContainerStarted","Data":"5912920fc39403e38e45c4d02ebe937e09ff944960005edc4b3aa9eb8bb73f45"}
Oct 30 22:17:15.167835 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:15.167743    5510 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-xv7c7" podStartSLOduration=2.167726688 podStartE2EDuration="2.167726688s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:17:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:17:15.164835627 +0000 UTC m=+1615.725645651" watchObservedRunningTime="2026-10-30 22:17:15.167726688 +0000 UTC m=+1615.728536693"
Oct 30 22:17:15.173168 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:15.173139    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-xv7c7"
Oct 30 22:17:15.174933 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:15.174907    5510 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-85544d4cd7-xv7c7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.98:15021/healthz/ready\": dial tcp 10.133.0.98:15021: connect: connection refused" start-of-body=
Oct 30 22:17:15.175062 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:15.174950    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-xv7c7" podUID="21c0fef2-49cd-4734-b402-f1cdeb031ac8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.98:15021/healthz/ready\": dial tcp 10.133.0.98:15021: connect: connection refused"
Oct 30 22:17:15.604679 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:15.602295    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:17:16.178689 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:16.178162    5510 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-85544d4cd7-xv7c7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.98:15021/healthz/ready\": dial tcp 10.133.0.98:15021: connect: connection refused" start-of-body=
Oct 30 22:17:16.178689 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:16.178274    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-xv7c7" podUID="21c0fef2-49cd-4734-b402-f1cdeb031ac8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.98:15021/healthz/ready\": dial tcp 10.133.0.98:15021: connect: connection refused"
Oct 30 22:17:17.174686 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:17.173899    5510 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-85544d4cd7-xv7c7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.98:15021/healthz/ready\": dial tcp 10.133.0.98:15021: connect: connection refused" start-of-body=
Oct 30 22:17:17.174686 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:17.173983    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-xv7c7" podUID="21c0fef2-49cd-4734-b402-f1cdeb031ac8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.98:15021/healthz/ready\": dial tcp 10.133.0.98:15021: connect: connection refused"
Oct 30 22:17:18.185681 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:18.176007    5510 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-85544d4cd7-xv7c7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.98:15021/healthz/ready\": dial tcp 10.133.0.98:15021: connect: connection refused" start-of-body=
Oct 30 22:17:18.185681 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:18.176120    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-xv7c7" podUID="21c0fef2-49cd-4734-b402-f1cdeb031ac8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.98:15021/healthz/ready\": dial tcp 10.133.0.98:15021: connect: connection refused"
Oct 30 22:17:18.372682 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:18.371740    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-7bb1c2/distinct-2-w3-7bb1c2-kserve-c549df844-zt595" podUID="1916e87c-a1a4-4967-8640-8bbcd9d12903" containerName="main" probeResult="failure" output="Get \"https://10.133.0.73:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:17:19.159682 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:19.158784    5510 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-ff716ef6-f89c8bd9c-mhgqb" podUID="46927be5-d550-469c-baa4-f9f124952a84" containerName="payload-processing" containerID="cri-o://ea3dd327695220c042deb86ce05d2a8eb1619962e730615341b71329fb7ed09a" gracePeriod=25
Oct 30 22:17:19.179192 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:19.179153    5510 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-85544d4cd7-xv7c7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.98:15021/healthz/ready\": dial tcp 10.133.0.98:15021: connect: connection refused" start-of-body=
Oct 30 22:17:19.179477 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:19.179443    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-xv7c7" podUID="21c0fef2-49cd-4734-b402-f1cdeb031ac8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.98:15021/healthz/ready\": dial tcp 10.133.0.98:15021: connect: connection refused"
Oct 30 22:17:19.293927 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:19.293828    5510 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-ff716ef6-5574d46599-svwhz" podUID="1a1e6300-d0d1-41c7-b089-b7f27eafe2bb" containerName="payload-pre-processing" containerID="cri-o://68643faca0840d83d21ca5655dc89b59b7bd51cc57041ef16cbe04be87eccc22" gracePeriod=25
Oct 30 22:17:20.122683 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:20.117747    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-fb0b93/premium-w6-fb0b93-kserve-5cb9948c54-w7pnk" podUID="290ae7a2-de37-4af3-82ae-a8e855b83bb9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:17:20.131857 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:20.131813    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-779f32/distinct-w0-779f32-kserve-6dcbc948c7-2vzdf" podUID="e17cedd4-7d48-42b3-b127-b6192c1b3795" containerName="main" probeResult="failure" output="Get \"https://10.133.0.65:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:17:20.215048 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:20.214938    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-bc714d/distinct-w2-bc714d-kserve-66cfb968dd-jvgqj" podUID="9e9b11b8-1cc9-4fc8-8097-1afcffa8c422" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:17:20.364921 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:20.364866    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-c9dbe6/distinct-w5-c9dbe6-kserve-7599fd86d9-bdhxd" podUID="e903db90-f2cd-4244-b00b-f99b21873f6e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.67:8000/ready\": context deadline exceeded"
Oct 30 22:17:20.401697 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:20.370305    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-fb0b93/distinct-w6-fb0b93-kserve-58db6b6757-6mww8" podUID="8ee6579b-acea-4492-862b-9883f70d00c9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:17:20.401697 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:20.370594    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-7bb1c2/distinct-2-w3-7bb1c2-kserve-c549df844-zt595" podUID="1916e87c-a1a4-4967-8640-8bbcd9d12903" containerName="main" probeResult="failure" output="Get \"https://10.133.0.73:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:17:20.432680 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:20.413065    5510 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w5-c9dbe6/distinct-w5-c9dbe6-kserve-7599fd86d9-bdhxd" podUID="e903db90-f2cd-4244-b00b-f99b21873f6e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.67:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:17:20.951730 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:17:20.941093    5510 kubelet.go:2711] "Housekeeping took longer than expected" err="housekeeping took too long" expected="1s" actual="1.362s"
Oct 30 22:17:20.951730 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:20.941143    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-72b5472b-544d6795d5-vtsjp"]
Oct 30 22:17:20.951730 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:20.941322    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-72b5472b-5dfd8db78c-zwg2j"]
Oct 30 22:17:21.106203 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:21.106163    5510 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-85544d4cd7-xv7c7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.98:15021/healthz/ready\": dial tcp 10.133.0.98:15021: connect: connection refused" start-of-body=
Oct 30 22:17:21.106488 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:21.106452    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-xv7c7" podUID="21c0fef2-49cd-4734-b402-f1cdeb031ac8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.98:15021/healthz/ready\": dial tcp 10.133.0.98:15021: connect: connection refused"
Oct 30 22:17:21.246307 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:21.243528    5510 generic.go:356] "Generic (PLEG): container finished" podID="46927be5-d550-469c-baa4-f9f124952a84" containerID="ea3dd327695220c042deb86ce05d2a8eb1619962e730615341b71329fb7ed09a" exitCode=0
Oct 30 22:17:21.340677 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:21.319797    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-ff716ef6-f89c8bd9c-mhgqb" event={"ID":"46927be5-d550-469c-baa4-f9f124952a84","Type":"ContainerDied","Data":"ea3dd327695220c042deb86ce05d2a8eb1619962e730615341b71329fb7ed09a"}
Oct 30 22:17:21.648678 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:21.648464    5510 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-85544d4cd7-xv7c7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.98:15021/healthz/ready\": dial tcp 10.133.0.98:15021: connect: connection refused" start-of-body=
Oct 30 22:17:21.648678 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:21.648595    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-xv7c7" podUID="21c0fef2-49cd-4734-b402-f1cdeb031ac8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.98:15021/healthz/ready\": dial tcp 10.133.0.98:15021: connect: connection refused"
Oct 30 22:17:22.061110 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:22.061060    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-7bb1c2/distinct-2-w3-7bb1c2-kserve-c549df844-zt595" podUID="1916e87c-a1a4-4967-8640-8bbcd9d12903" containerName="main" probeResult="failure" output="Get \"https://10.133.0.73:8000/ready\": context deadline exceeded"
Oct 30 22:17:22.062540 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:22.062508    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-779f32/distinct-2-w0-779f32-kserve-857ffdf778-gk2h9" podUID="ef500cdd-cc52-45b0-8dd7-1896897acc46" containerName="main" probeResult="failure" output="Get \"https://10.133.0.74:8000/ready\": context deadline exceeded"
Oct 30 22:17:22.152982 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:22.145262    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-779f32/distinct-w0-779f32-kserve-6dcbc948c7-2vzdf" podUID="e17cedd4-7d48-42b3-b127-b6192c1b3795" containerName="main" probeResult="failure" output="Get \"https://10.133.0.65:8000/ready\": context deadline exceeded"
Oct 30 22:17:22.152982 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:22.145578    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-c9dbe6/distinct-w5-c9dbe6-kserve-7599fd86d9-bdhxd" podUID="e903db90-f2cd-4244-b00b-f99b21873f6e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.67:8000/ready\": context deadline exceeded"
Oct 30 22:17:22.152982 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:22.145808    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-42mzp" podUID="9f899391-6025-4f44-9553-a82cd5e25e69" containerName="main" probeResult="failure" output="Get \"https://10.133.0.33:8000/ready\": context deadline exceeded"
Oct 30 22:17:22.152982 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:22.145974    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-fb0b93/distinct-w6-fb0b93-kserve-58db6b6757-6mww8" podUID="8ee6579b-acea-4492-862b-9883f70d00c9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.59:8000/ready\": context deadline exceeded"
Oct 30 22:17:22.293779 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:22.293639    5510 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-85544d4cd7-xv7c7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.98:15021/healthz/ready\": dial tcp 10.133.0.98:15021: connect: connection refused" start-of-body=
Oct 30 22:17:22.294013 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:22.293987    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-xv7c7" podUID="21c0fef2-49cd-4734-b402-f1cdeb031ac8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.98:15021/healthz/ready\": dial tcp 10.133.0.98:15021: connect: connection refused"
Oct 30 22:17:22.333541 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:22.333401    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:17:22.338007 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:22.337978    5510 generic.go:356] "Generic (PLEG): container finished" podID="1a1e6300-d0d1-41c7-b089-b7f27eafe2bb" containerID="68643faca0840d83d21ca5655dc89b59b7bd51cc57041ef16cbe04be87eccc22" exitCode=0
Oct 30 22:17:22.338507 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:22.338463    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-ff716ef6-5574d46599-svwhz" event={"ID":"1a1e6300-d0d1-41c7-b089-b7f27eafe2bb","Type":"ContainerDied","Data":"68643faca0840d83d21ca5655dc89b59b7bd51cc57041ef16cbe04be87eccc22"}
Oct 30 22:17:22.338625 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:22.338548    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-7bb1c2/distinct-2-w3-7bb1c2-kserve-c549df844-zt595"
Oct 30 22:17:22.338625 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:22.338598    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-779f32/distinct-w0-779f32-kserve-6dcbc948c7-2vzdf"
Oct 30 22:17:22.338746 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:22.338632    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-c9dbe6/distinct-w5-c9dbe6-kserve-7599fd86d9-bdhxd"
Oct 30 22:17:22.338746 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:22.338689    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-fb0b93/distinct-w6-fb0b93-kserve-58db6b6757-6mww8"
Oct 30 22:17:22.338746 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:22.338727    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-fb0b93/distinct-w6-fb0b93-kserve-58db6b6757-6mww8"
Oct 30 22:17:22.340576 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:22.340560    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-ff716ef6-f89c8bd9c-mhgqb"
Oct 30 22:17:22.340788 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:22.340775    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-ff716ef6-5574d46599-svwhz"
Oct 30 22:17:22.413542 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:22.411768    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-7bb1c2/distinct-2-w3-7bb1c2-kserve-c549df844-zt595"
Oct 30 22:17:22.418506 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:22.418475    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-779f32/distinct-w0-779f32-kserve-6dcbc948c7-2vzdf"
Oct 30 22:17:22.420345 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:22.420313    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-c9dbe6/distinct-w5-c9dbe6-kserve-7599fd86d9-bdhxd"
Oct 30 22:17:22.517438 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:22.517407    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1a1e6300-d0d1-41c7-b089-b7f27eafe2bb-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1a1e6300-d0d1-41c7-b089-b7f27eafe2bb-plugins-config-volume\") pod \"1a1e6300-d0d1-41c7-b089-b7f27eafe2bb\" (UID: \"1a1e6300-d0d1-41c7-b089-b7f27eafe2bb\") "
Oct 30 22:17:22.517576 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:22.517560    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1a1e6300-d0d1-41c7-b089-b7f27eafe2bb-kube-api-access-dmwc6\" (UniqueName: \"kubernetes.io/projected/1a1e6300-d0d1-41c7-b089-b7f27eafe2bb-kube-api-access-dmwc6\") pod \"1a1e6300-d0d1-41c7-b089-b7f27eafe2bb\" (UID: \"1a1e6300-d0d1-41c7-b089-b7f27eafe2bb\") "
Oct 30 22:17:22.517736 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:22.517718    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/46927be5-d550-469c-baa4-f9f124952a84-kube-api-access-l4jkk\" (UniqueName: \"kubernetes.io/projected/46927be5-d550-469c-baa4-f9f124952a84-kube-api-access-l4jkk\") pod \"46927be5-d550-469c-baa4-f9f124952a84\" (UID: \"46927be5-d550-469c-baa4-f9f124952a84\") "
Oct 30 22:17:22.517843 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:22.517814    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1a1e6300-d0d1-41c7-b089-b7f27eafe2bb-plugins-config-volume" pod "1a1e6300-d0d1-41c7-b089-b7f27eafe2bb" (UID: "1a1e6300-d0d1-41c7-b089-b7f27eafe2bb"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:17:22.517904 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:22.517870    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/46927be5-d550-469c-baa4-f9f124952a84-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/46927be5-d550-469c-baa4-f9f124952a84-plugins-config-volume\") pod \"46927be5-d550-469c-baa4-f9f124952a84\" (UID: \"46927be5-d550-469c-baa4-f9f124952a84\") "
Oct 30 22:17:22.518381 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:22.518361    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/46927be5-d550-469c-baa4-f9f124952a84-plugins-config-volume" pod "46927be5-d550-469c-baa4-f9f124952a84" (UID: "46927be5-d550-469c-baa4-f9f124952a84"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:17:22.518631 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:22.518617    5510 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/46927be5-d550-469c-baa4-f9f124952a84-plugins-config-volume\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:17:22.518737 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:22.518634    5510 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1a1e6300-d0d1-41c7-b089-b7f27eafe2bb-plugins-config-volume\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:17:22.519869 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:22.519835    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1a1e6300-d0d1-41c7-b089-b7f27eafe2bb-kube-api-access-dmwc6" pod "1a1e6300-d0d1-41c7-b089-b7f27eafe2bb" (UID: "1a1e6300-d0d1-41c7-b089-b7f27eafe2bb"). InnerVolumeSpecName "kube-api-access-dmwc6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:17:22.519935 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:22.519922    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/46927be5-d550-469c-baa4-f9f124952a84-kube-api-access-l4jkk" pod "46927be5-d550-469c-baa4-f9f124952a84" (UID: "46927be5-d550-469c-baa4-f9f124952a84"). InnerVolumeSpecName "kube-api-access-l4jkk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:17:22.603505 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:22.603373    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:17:22.620043 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:22.620012    5510 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dmwc6\" (UniqueName: \"kubernetes.io/projected/1a1e6300-d0d1-41c7-b089-b7f27eafe2bb-kube-api-access-dmwc6\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:17:22.620043 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:22.620040    5510 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l4jkk\" (UniqueName: \"kubernetes.io/projected/46927be5-d550-469c-baa4-f9f124952a84-kube-api-access-l4jkk\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:17:22.994390 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:22.994352    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:17:23.287736 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:23.284878    5510 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-85544d4cd7-xv7c7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.98:15021/healthz/ready\": dial tcp 10.133.0.98:15021: connect: connection refused" start-of-body=
Oct 30 22:17:23.287736 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:23.284962    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-xv7c7" podUID="21c0fef2-49cd-4734-b402-f1cdeb031ac8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.98:15021/healthz/ready\": dial tcp 10.133.0.98:15021: connect: connection refused"
Oct 30 22:17:23.363525 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:23.362992    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-ff716ef6-f89c8bd9c-mhgqb"
Oct 30 22:17:23.363525 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:23.362997    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-ff716ef6-f89c8bd9c-mhgqb" event={"ID":"46927be5-d550-469c-baa4-f9f124952a84","Type":"ContainerDied","Data":"1b00206ea6bd65594d463e7f22118eb1fe289f986da6b6fc3db7852aec58896f"}
Oct 30 22:17:23.363525 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:23.363117    5510 scope.go:122] "RemoveContainer" containerID="ea3dd327695220c042deb86ce05d2a8eb1619962e730615341b71329fb7ed09a"
Oct 30 22:17:23.369582 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:23.369559    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-ff716ef6-5574d46599-svwhz"
Oct 30 22:17:23.369854 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:23.369825    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-ff716ef6-5574d46599-svwhz" event={"ID":"1a1e6300-d0d1-41c7-b089-b7f27eafe2bb","Type":"ContainerDied","Data":"56f042d14ba82abbfefec7b2d91692ca6f8a5b77afa82194e0931b1462516757"}
Oct 30 22:17:23.379818 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:23.379794    5510 scope.go:122] "RemoveContainer" containerID="68643faca0840d83d21ca5655dc89b59b7bd51cc57041ef16cbe04be87eccc22"
Oct 30 22:17:23.816795 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:23.816760    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-ff716ef6-5574d46599-svwhz"]
Oct 30 22:17:23.821180 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:23.821152    5510 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-ff716ef6-5574d46599-svwhz"]
Oct 30 22:17:23.933050 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:23.933016    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-ff716ef6-f89c8bd9c-mhgqb"]
Oct 30 22:17:23.962463 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:23.962433    5510 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-ff716ef6-f89c8bd9c-mhgqb"]
Oct 30 22:17:24.173118 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:24.173026    5510 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-85544d4cd7-xv7c7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.98:15021/healthz/ready\": dial tcp 10.133.0.98:15021: connect: connection refused" start-of-body=
Oct 30 22:17:24.173118 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:24.173095    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-xv7c7" podUID="21c0fef2-49cd-4734-b402-f1cdeb031ac8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.98:15021/healthz/ready\": dial tcp 10.133.0.98:15021: connect: connection refused"
Oct 30 22:17:25.555990 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:25.555953    5510 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1a1e6300-d0d1-41c7-b089-b7f27eafe2bb" path="/var/lib/kubelet/pods/1a1e6300-d0d1-41c7-b089-b7f27eafe2bb/volumes"
Oct 30 22:17:25.557281 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:25.557254    5510 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="46927be5-d550-469c-baa4-f9f124952a84" path="/var/lib/kubelet/pods/46927be5-d550-469c-baa4-f9f124952a84/volumes"
Oct 30 22:17:25.898879 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:25.898783    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-xv7c7"
Oct 30 22:17:25.899217 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:25.899192    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-xv7c7"
Oct 30 22:17:25.900213 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:25.900189    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-xv7c7"
Oct 30 22:17:26.677160 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:26.677113    5510 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-72b5472b-544d6795d5-vtsjp" podUID="8dc39be6-e92f-4d15-afca-1035e750f9a6" containerName="payload-processing" containerID="cri-o://b6fc1c053eac4b69f90904dade68a5afd4acfbc17bad6fa881b5e301926f8d85" gracePeriod=25
Oct 30 22:17:27.006273 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:27.006239    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-72b5472b-544d6795d5-vtsjp"
Oct 30 22:17:27.034515 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:27.034485    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8dc39be6-e92f-4d15-afca-1035e750f9a6-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8dc39be6-e92f-4d15-afca-1035e750f9a6-plugins-config-volume\") pod \"8dc39be6-e92f-4d15-afca-1035e750f9a6\" (UID: \"8dc39be6-e92f-4d15-afca-1035e750f9a6\") "
Oct 30 22:17:27.034737 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:27.034722    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8dc39be6-e92f-4d15-afca-1035e750f9a6-kube-api-access-5vp5w\" (UniqueName: \"kubernetes.io/projected/8dc39be6-e92f-4d15-afca-1035e750f9a6-kube-api-access-5vp5w\") pod \"8dc39be6-e92f-4d15-afca-1035e750f9a6\" (UID: \"8dc39be6-e92f-4d15-afca-1035e750f9a6\") "
Oct 30 22:17:27.034851 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:27.034829    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8dc39be6-e92f-4d15-afca-1035e750f9a6-plugins-config-volume" pod "8dc39be6-e92f-4d15-afca-1035e750f9a6" (UID: "8dc39be6-e92f-4d15-afca-1035e750f9a6"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:17:27.035355 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:27.035326    5510 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8dc39be6-e92f-4d15-afca-1035e750f9a6-plugins-config-volume\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:17:27.036900 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:27.036878    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8dc39be6-e92f-4d15-afca-1035e750f9a6-kube-api-access-5vp5w" pod "8dc39be6-e92f-4d15-afca-1035e750f9a6" (UID: "8dc39be6-e92f-4d15-afca-1035e750f9a6"). InnerVolumeSpecName "kube-api-access-5vp5w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:17:27.137130 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:27.137091    5510 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5vp5w\" (UniqueName: \"kubernetes.io/projected/8dc39be6-e92f-4d15-afca-1035e750f9a6-kube-api-access-5vp5w\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:17:27.239048 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:27.238964    5510 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-72b5472b-5dfd8db78c-zwg2j" podUID="2679ae9b-9620-4340-8701-aab7b4cc1e13" containerName="payload-pre-processing" containerID="cri-o://0f09b6a5919282acb849777ab5f6a5ffe90c65b391e19c455b58a6d33528b32c" gracePeriod=25
Oct 30 22:17:27.415068 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:27.415031    5510 generic.go:356] "Generic (PLEG): container finished" podID="2679ae9b-9620-4340-8701-aab7b4cc1e13" containerID="0f09b6a5919282acb849777ab5f6a5ffe90c65b391e19c455b58a6d33528b32c" exitCode=0
Oct 30 22:17:27.415225 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:27.415109    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-72b5472b-5dfd8db78c-zwg2j" event={"ID":"2679ae9b-9620-4340-8701-aab7b4cc1e13","Type":"ContainerDied","Data":"0f09b6a5919282acb849777ab5f6a5ffe90c65b391e19c455b58a6d33528b32c"}
Oct 30 22:17:27.416759 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:27.416725    5510 generic.go:356] "Generic (PLEG): container finished" podID="8dc39be6-e92f-4d15-afca-1035e750f9a6" containerID="b6fc1c053eac4b69f90904dade68a5afd4acfbc17bad6fa881b5e301926f8d85" exitCode=0
Oct 30 22:17:27.416899 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:27.416772    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-72b5472b-544d6795d5-vtsjp" event={"ID":"8dc39be6-e92f-4d15-afca-1035e750f9a6","Type":"ContainerDied","Data":"b6fc1c053eac4b69f90904dade68a5afd4acfbc17bad6fa881b5e301926f8d85"}
Oct 30 22:17:27.416899 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:27.416822    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-72b5472b-544d6795d5-vtsjp" event={"ID":"8dc39be6-e92f-4d15-afca-1035e750f9a6","Type":"ContainerDied","Data":"01b8aa077a50c8ec8ddeebeccc812bb9b10b8f94349a343882da62bb4ae5bac0"}
Oct 30 22:17:27.416899 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:27.416779    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-72b5472b-544d6795d5-vtsjp"
Oct 30 22:17:27.416899 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:27.416850    5510 scope.go:122] "RemoveContainer" containerID="b6fc1c053eac4b69f90904dade68a5afd4acfbc17bad6fa881b5e301926f8d85"
Oct 30 22:17:27.429276 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:27.429248    5510 scope.go:122] "RemoveContainer" containerID="b6fc1c053eac4b69f90904dade68a5afd4acfbc17bad6fa881b5e301926f8d85"
Oct 30 22:17:27.429596 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:17:27.429576    5510 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b6fc1c053eac4b69f90904dade68a5afd4acfbc17bad6fa881b5e301926f8d85\": container with ID starting with b6fc1c053eac4b69f90904dade68a5afd4acfbc17bad6fa881b5e301926f8d85 not found: ID does not exist" containerID="b6fc1c053eac4b69f90904dade68a5afd4acfbc17bad6fa881b5e301926f8d85"
Oct 30 22:17:27.429702 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:27.429603    5510 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b6fc1c053eac4b69f90904dade68a5afd4acfbc17bad6fa881b5e301926f8d85"} err="failed to get container status \"b6fc1c053eac4b69f90904dade68a5afd4acfbc17bad6fa881b5e301926f8d85\": rpc error: code = NotFound desc = could not find container \"b6fc1c053eac4b69f90904dade68a5afd4acfbc17bad6fa881b5e301926f8d85\": container with ID starting with b6fc1c053eac4b69f90904dade68a5afd4acfbc17bad6fa881b5e301926f8d85 not found: ID does not exist"
Oct 30 22:17:27.453278 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:27.453124    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:17:27.537184 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:27.537151    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-72b5472b-5dfd8db78c-zwg2j"
Oct 30 22:17:27.604120 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:27.604083    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-72b5472b-544d6795d5-vtsjp"]
Oct 30 22:17:27.622914 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:27.622886    5510 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-72b5472b-544d6795d5-vtsjp"]
Oct 30 22:17:27.646008 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:27.645971    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2679ae9b-9620-4340-8701-aab7b4cc1e13-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2679ae9b-9620-4340-8701-aab7b4cc1e13-plugins-config-volume\") pod \"2679ae9b-9620-4340-8701-aab7b4cc1e13\" (UID: \"2679ae9b-9620-4340-8701-aab7b4cc1e13\") "
Oct 30 22:17:27.646300 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:27.646275    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2679ae9b-9620-4340-8701-aab7b4cc1e13-kube-api-access-8rqdr\" (UniqueName: \"kubernetes.io/projected/2679ae9b-9620-4340-8701-aab7b4cc1e13-kube-api-access-8rqdr\") pod \"2679ae9b-9620-4340-8701-aab7b4cc1e13\" (UID: \"2679ae9b-9620-4340-8701-aab7b4cc1e13\") "
Oct 30 22:17:27.646400 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:27.646354    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2679ae9b-9620-4340-8701-aab7b4cc1e13-plugins-config-volume" pod "2679ae9b-9620-4340-8701-aab7b4cc1e13" (UID: "2679ae9b-9620-4340-8701-aab7b4cc1e13"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:17:27.647824 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:27.647763    5510 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2679ae9b-9620-4340-8701-aab7b4cc1e13-plugins-config-volume\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:17:27.649311 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:27.649287    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2679ae9b-9620-4340-8701-aab7b4cc1e13-kube-api-access-8rqdr" pod "2679ae9b-9620-4340-8701-aab7b4cc1e13" (UID: "2679ae9b-9620-4340-8701-aab7b4cc1e13"). InnerVolumeSpecName "kube-api-access-8rqdr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:17:27.750292 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:27.749834    5510 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8rqdr\" (UniqueName: \"kubernetes.io/projected/2679ae9b-9620-4340-8701-aab7b4cc1e13-kube-api-access-8rqdr\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:17:27.924735 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:27.916893    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:17:28.426673 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:28.426621    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-72b5472b-5dfd8db78c-zwg2j" event={"ID":"2679ae9b-9620-4340-8701-aab7b4cc1e13","Type":"ContainerDied","Data":"656d1d75990d86a3f9b37633f6bac4937f97780e40e2f649d3768b64ed6c2ccd"}
Oct 30 22:17:28.426992 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:28.426679    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-72b5472b-5dfd8db78c-zwg2j"
Oct 30 22:17:28.426992 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:28.426773    5510 scope.go:122] "RemoveContainer" containerID="0f09b6a5919282acb849777ab5f6a5ffe90c65b391e19c455b58a6d33528b32c"
Oct 30 22:17:28.595411 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:28.595375    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-72b5472b-5dfd8db78c-zwg2j"]
Oct 30 22:17:28.646040 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:28.646001    5510 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-72b5472b-5dfd8db78c-zwg2j"]
Oct 30 22:17:29.555484 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:29.555448    5510 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2679ae9b-9620-4340-8701-aab7b4cc1e13" path="/var/lib/kubelet/pods/2679ae9b-9620-4340-8701-aab7b4cc1e13/volumes"
Oct 30 22:17:29.556019 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:29.556002    5510 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8dc39be6-e92f-4d15-afca-1035e750f9a6" path="/var/lib/kubelet/pods/8dc39be6-e92f-4d15-afca-1035e750f9a6/volumes"
Oct 30 22:17:32.533876 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:32.533824    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:17:37.217799 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:37.217753    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:17:38.422181 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:17:38.422145    5510 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6gjbf" containerName="extractor"
Oct 30 22:17:41.216708 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:41.216649    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:17:42.709284 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:42.709244    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:17:42.958562 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:42.958489    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:17:43.154156 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:43.154115    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:17:43.405670 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:43.405571    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-67184f2a-gw-openshift-default-5cb54945b9-5c64x"]
Oct 30 22:17:43.405851 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:43.405827    5510 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-67184f2a-gw-openshift-default-5cb54945b9-5c64x" podUID="f067f5fc-05db-4807-982f-21e7a5997acc" containerName="istio-proxy" containerID="cri-o://ce5d937cbad5658849effa727bf0135def5e85e56e4cd5e7d24d4e4a8318f8cb" gracePeriod=30
Oct 30 22:17:43.457785 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:43.457748    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:17:44.115077 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:44.115044    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:17:44.870595 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:44.870560    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:17:45.252094 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:45.252003    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:17:45.513736 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:45.512833    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:17:45.587792 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:45.587737    5510 patch_prober.go:29] interesting pod/e2e-migrate-67184f2a-gw-openshift-default-5cb54945b9-5c64x container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 22:17:45.588105 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:45.588073    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-migrate-67184f2a-gw-openshift-default-5cb54945b9-5c64x" podUID="f067f5fc-05db-4807-982f-21e7a5997acc" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 22:17:45.777438 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:45.777398    5510 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w0-c42c0a-openshift-default-558b6f6d6f-7rcsl"]
Oct 30 22:17:45.778971 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:45.778883    5510 cpu_manager.go:414] "containerMap: removing container" podUID="8dc39be6-e92f-4d15-afca-1035e750f9a6" containerName="payload-processing"
Oct 30 22:17:45.779169 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:45.779149    5510 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8dc39be6-e92f-4d15-afca-1035e750f9a6" containerName="payload-processing"
Oct 30 22:17:45.779286 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:45.779275    5510 cpu_manager.go:414] "containerMap: removing container" podUID="2679ae9b-9620-4340-8701-aab7b4cc1e13" containerName="payload-pre-processing"
Oct 30 22:17:45.779367 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:45.779356    5510 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2679ae9b-9620-4340-8701-aab7b4cc1e13" containerName="payload-pre-processing"
Oct 30 22:17:45.779489 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:45.779478    5510 cpu_manager.go:414] "containerMap: removing container" podUID="1a1e6300-d0d1-41c7-b089-b7f27eafe2bb" containerName="payload-pre-processing"
Oct 30 22:17:45.779573 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:45.779562    5510 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1a1e6300-d0d1-41c7-b089-b7f27eafe2bb" containerName="payload-pre-processing"
Oct 30 22:17:45.779718 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:45.779707    5510 cpu_manager.go:414] "containerMap: removing container" podUID="46927be5-d550-469c-baa4-f9f124952a84" containerName="payload-processing"
Oct 30 22:17:45.779807 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:45.779797    5510 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="46927be5-d550-469c-baa4-f9f124952a84" containerName="payload-processing"
Oct 30 22:17:45.780057 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:45.780044    5510 memory_manager.go:365] "RemoveStaleState removing state" podUID="46927be5-d550-469c-baa4-f9f124952a84" containerName="payload-processing"
Oct 30 22:17:45.780186 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:45.780140    5510 memory_manager.go:365] "RemoveStaleState removing state" podUID="8dc39be6-e92f-4d15-afca-1035e750f9a6" containerName="payload-processing"
Oct 30 22:17:45.780280 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:45.780268    5510 memory_manager.go:365] "RemoveStaleState removing state" podUID="2679ae9b-9620-4340-8701-aab7b4cc1e13" containerName="payload-pre-processing"
Oct 30 22:17:45.780361 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:45.780352    5510 memory_manager.go:365] "RemoveStaleState removing state" podUID="1a1e6300-d0d1-41c7-b089-b7f27eafe2bb" containerName="payload-pre-processing"
Oct 30 22:17:45.807305 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:45.800148    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w0-c42c0a-openshift-default-558b6f6d6f-7rcsl"
Oct 30 22:17:45.836969 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:45.836939    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w0-c42c0a-openshift-default-558b6f6d6f-7rcsl"]
Oct 30 22:17:45.882820 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:45.882782    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w0-c42c0a-openshift-default-dockercfg-n4qmj\""
Oct 30 22:17:45.926771 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:45.926732    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3a76b5b9-88cf-4f7f-9d14-fd725017b62f-workload-certs\") pod \"e2e-shared-a-w0-c42c0a-openshift-default-558b6f6d6f-7rcsl\" (UID: \"3a76b5b9-88cf-4f7f-9d14-fd725017b62f\") " pod="openshift-ingress/e2e-shared-a-w0-c42c0a-openshift-default-558b6f6d6f-7rcsl"
Oct 30 22:17:45.926983 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:45.926781    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a76b5b9-88cf-4f7f-9d14-fd725017b62f-credential-socket\") pod \"e2e-shared-a-w0-c42c0a-openshift-default-558b6f6d6f-7rcsl\" (UID: \"3a76b5b9-88cf-4f7f-9d14-fd725017b62f\") " pod="openshift-ingress/e2e-shared-a-w0-c42c0a-openshift-default-558b6f6d6f-7rcsl"
Oct 30 22:17:45.926983 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:45.926896    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3a76b5b9-88cf-4f7f-9d14-fd725017b62f-istio-token\") pod \"e2e-shared-a-w0-c42c0a-openshift-default-558b6f6d6f-7rcsl\" (UID: \"3a76b5b9-88cf-4f7f-9d14-fd725017b62f\") " pod="openshift-ingress/e2e-shared-a-w0-c42c0a-openshift-default-558b6f6d6f-7rcsl"
Oct 30 22:17:45.927138 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:45.927002    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3a76b5b9-88cf-4f7f-9d14-fd725017b62f-istio-envoy\") pod \"e2e-shared-a-w0-c42c0a-openshift-default-558b6f6d6f-7rcsl\" (UID: \"3a76b5b9-88cf-4f7f-9d14-fd725017b62f\") " pod="openshift-ingress/e2e-shared-a-w0-c42c0a-openshift-default-558b6f6d6f-7rcsl"
Oct 30 22:17:45.927138 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:45.927078    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fpzw4\" (UniqueName: \"kubernetes.io/projected/3a76b5b9-88cf-4f7f-9d14-fd725017b62f-kube-api-access-fpzw4\") pod \"e2e-shared-a-w0-c42c0a-openshift-default-558b6f6d6f-7rcsl\" (UID: \"3a76b5b9-88cf-4f7f-9d14-fd725017b62f\") " pod="openshift-ingress/e2e-shared-a-w0-c42c0a-openshift-default-558b6f6d6f-7rcsl"
Oct 30 22:17:45.927253 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:45.927169    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3a76b5b9-88cf-4f7f-9d14-fd725017b62f-istio-podinfo\") pod \"e2e-shared-a-w0-c42c0a-openshift-default-558b6f6d6f-7rcsl\" (UID: \"3a76b5b9-88cf-4f7f-9d14-fd725017b62f\") " pod="openshift-ingress/e2e-shared-a-w0-c42c0a-openshift-default-558b6f6d6f-7rcsl"
Oct 30 22:17:45.927308 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:45.927257    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3a76b5b9-88cf-4f7f-9d14-fd725017b62f-istio-data\") pod \"e2e-shared-a-w0-c42c0a-openshift-default-558b6f6d6f-7rcsl\" (UID: \"3a76b5b9-88cf-4f7f-9d14-fd725017b62f\") " pod="openshift-ingress/e2e-shared-a-w0-c42c0a-openshift-default-558b6f6d6f-7rcsl"
Oct 30 22:17:45.927430 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:45.927326    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a76b5b9-88cf-4f7f-9d14-fd725017b62f-workload-socket\") pod \"e2e-shared-a-w0-c42c0a-openshift-default-558b6f6d6f-7rcsl\" (UID: \"3a76b5b9-88cf-4f7f-9d14-fd725017b62f\") " pod="openshift-ingress/e2e-shared-a-w0-c42c0a-openshift-default-558b6f6d6f-7rcsl"
Oct 30 22:17:45.927430 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:45.927380    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3a76b5b9-88cf-4f7f-9d14-fd725017b62f-istiod-ca-cert\") pod \"e2e-shared-a-w0-c42c0a-openshift-default-558b6f6d6f-7rcsl\" (UID: \"3a76b5b9-88cf-4f7f-9d14-fd725017b62f\") " pod="openshift-ingress/e2e-shared-a-w0-c42c0a-openshift-default-558b6f6d6f-7rcsl"
Oct 30 22:17:46.038598 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:46.038489    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3a76b5b9-88cf-4f7f-9d14-fd725017b62f-istio-envoy\") pod \"e2e-shared-a-w0-c42c0a-openshift-default-558b6f6d6f-7rcsl\" (UID: \"3a76b5b9-88cf-4f7f-9d14-fd725017b62f\") " pod="openshift-ingress/e2e-shared-a-w0-c42c0a-openshift-default-558b6f6d6f-7rcsl"
Oct 30 22:17:46.038938 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:46.038907    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fpzw4\" (UniqueName: \"kubernetes.io/projected/3a76b5b9-88cf-4f7f-9d14-fd725017b62f-kube-api-access-fpzw4\") pod \"e2e-shared-a-w0-c42c0a-openshift-default-558b6f6d6f-7rcsl\" (UID: \"3a76b5b9-88cf-4f7f-9d14-fd725017b62f\") " pod="openshift-ingress/e2e-shared-a-w0-c42c0a-openshift-default-558b6f6d6f-7rcsl"
Oct 30 22:17:46.039109 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:46.039094    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3a76b5b9-88cf-4f7f-9d14-fd725017b62f-istio-podinfo\") pod \"e2e-shared-a-w0-c42c0a-openshift-default-558b6f6d6f-7rcsl\" (UID: \"3a76b5b9-88cf-4f7f-9d14-fd725017b62f\") " pod="openshift-ingress/e2e-shared-a-w0-c42c0a-openshift-default-558b6f6d6f-7rcsl"
Oct 30 22:17:46.039287 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:46.039273    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3a76b5b9-88cf-4f7f-9d14-fd725017b62f-istio-data\") pod \"e2e-shared-a-w0-c42c0a-openshift-default-558b6f6d6f-7rcsl\" (UID: \"3a76b5b9-88cf-4f7f-9d14-fd725017b62f\") " pod="openshift-ingress/e2e-shared-a-w0-c42c0a-openshift-default-558b6f6d6f-7rcsl"
Oct 30 22:17:46.039668 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:46.039631    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a76b5b9-88cf-4f7f-9d14-fd725017b62f-workload-socket\") pod \"e2e-shared-a-w0-c42c0a-openshift-default-558b6f6d6f-7rcsl\" (UID: \"3a76b5b9-88cf-4f7f-9d14-fd725017b62f\") " pod="openshift-ingress/e2e-shared-a-w0-c42c0a-openshift-default-558b6f6d6f-7rcsl"
Oct 30 22:17:46.039860 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:46.039841    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3a76b5b9-88cf-4f7f-9d14-fd725017b62f-istiod-ca-cert\") pod \"e2e-shared-a-w0-c42c0a-openshift-default-558b6f6d6f-7rcsl\" (UID: \"3a76b5b9-88cf-4f7f-9d14-fd725017b62f\") " pod="openshift-ingress/e2e-shared-a-w0-c42c0a-openshift-default-558b6f6d6f-7rcsl"
Oct 30 22:17:46.040189 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:46.040165    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3a76b5b9-88cf-4f7f-9d14-fd725017b62f-workload-certs\") pod \"e2e-shared-a-w0-c42c0a-openshift-default-558b6f6d6f-7rcsl\" (UID: \"3a76b5b9-88cf-4f7f-9d14-fd725017b62f\") " pod="openshift-ingress/e2e-shared-a-w0-c42c0a-openshift-default-558b6f6d6f-7rcsl"
Oct 30 22:17:46.040339 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:46.040324    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a76b5b9-88cf-4f7f-9d14-fd725017b62f-credential-socket\") pod \"e2e-shared-a-w0-c42c0a-openshift-default-558b6f6d6f-7rcsl\" (UID: \"3a76b5b9-88cf-4f7f-9d14-fd725017b62f\") " pod="openshift-ingress/e2e-shared-a-w0-c42c0a-openshift-default-558b6f6d6f-7rcsl"
Oct 30 22:17:46.040476 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:46.040462    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3a76b5b9-88cf-4f7f-9d14-fd725017b62f-istio-token\") pod \"e2e-shared-a-w0-c42c0a-openshift-default-558b6f6d6f-7rcsl\" (UID: \"3a76b5b9-88cf-4f7f-9d14-fd725017b62f\") " pod="openshift-ingress/e2e-shared-a-w0-c42c0a-openshift-default-558b6f6d6f-7rcsl"
Oct 30 22:17:46.045684 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:46.041852    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a76b5b9-88cf-4f7f-9d14-fd725017b62f-workload-socket\") pod \"e2e-shared-a-w0-c42c0a-openshift-default-558b6f6d6f-7rcsl\" (UID: \"3a76b5b9-88cf-4f7f-9d14-fd725017b62f\") " pod="openshift-ingress/e2e-shared-a-w0-c42c0a-openshift-default-558b6f6d6f-7rcsl"
Oct 30 22:17:46.045927 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:46.044434    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3a76b5b9-88cf-4f7f-9d14-fd725017b62f-istiod-ca-cert\") pod \"e2e-shared-a-w0-c42c0a-openshift-default-558b6f6d6f-7rcsl\" (UID: \"3a76b5b9-88cf-4f7f-9d14-fd725017b62f\") " pod="openshift-ingress/e2e-shared-a-w0-c42c0a-openshift-default-558b6f6d6f-7rcsl"
Oct 30 22:17:46.046034 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:46.044682    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3a76b5b9-88cf-4f7f-9d14-fd725017b62f-workload-certs\") pod \"e2e-shared-a-w0-c42c0a-openshift-default-558b6f6d6f-7rcsl\" (UID: \"3a76b5b9-88cf-4f7f-9d14-fd725017b62f\") " pod="openshift-ingress/e2e-shared-a-w0-c42c0a-openshift-default-558b6f6d6f-7rcsl"
Oct 30 22:17:46.046118 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:46.044915    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a76b5b9-88cf-4f7f-9d14-fd725017b62f-credential-socket\") pod \"e2e-shared-a-w0-c42c0a-openshift-default-558b6f6d6f-7rcsl\" (UID: \"3a76b5b9-88cf-4f7f-9d14-fd725017b62f\") " pod="openshift-ingress/e2e-shared-a-w0-c42c0a-openshift-default-558b6f6d6f-7rcsl"
Oct 30 22:17:46.046192 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:46.045170    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3a76b5b9-88cf-4f7f-9d14-fd725017b62f-istio-data\") pod \"e2e-shared-a-w0-c42c0a-openshift-default-558b6f6d6f-7rcsl\" (UID: \"3a76b5b9-88cf-4f7f-9d14-fd725017b62f\") " pod="openshift-ingress/e2e-shared-a-w0-c42c0a-openshift-default-558b6f6d6f-7rcsl"
Oct 30 22:17:46.061510 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:46.061467    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:17:46.096127 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:46.083570    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3a76b5b9-88cf-4f7f-9d14-fd725017b62f-istio-podinfo\") pod \"e2e-shared-a-w0-c42c0a-openshift-default-558b6f6d6f-7rcsl\" (UID: \"3a76b5b9-88cf-4f7f-9d14-fd725017b62f\") " pod="openshift-ingress/e2e-shared-a-w0-c42c0a-openshift-default-558b6f6d6f-7rcsl"
Oct 30 22:17:46.096127 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:46.096011    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3a76b5b9-88cf-4f7f-9d14-fd725017b62f-istio-envoy\") pod \"e2e-shared-a-w0-c42c0a-openshift-default-558b6f6d6f-7rcsl\" (UID: \"3a76b5b9-88cf-4f7f-9d14-fd725017b62f\") " pod="openshift-ingress/e2e-shared-a-w0-c42c0a-openshift-default-558b6f6d6f-7rcsl"
Oct 30 22:17:46.141402 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:46.141362    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fpzw4\" (UniqueName: \"kubernetes.io/projected/3a76b5b9-88cf-4f7f-9d14-fd725017b62f-kube-api-access-fpzw4\") pod \"e2e-shared-a-w0-c42c0a-openshift-default-558b6f6d6f-7rcsl\" (UID: \"3a76b5b9-88cf-4f7f-9d14-fd725017b62f\") " pod="openshift-ingress/e2e-shared-a-w0-c42c0a-openshift-default-558b6f6d6f-7rcsl"
Oct 30 22:17:46.155730 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:46.151592    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3a76b5b9-88cf-4f7f-9d14-fd725017b62f-istio-token\") pod \"e2e-shared-a-w0-c42c0a-openshift-default-558b6f6d6f-7rcsl\" (UID: \"3a76b5b9-88cf-4f7f-9d14-fd725017b62f\") " pod="openshift-ingress/e2e-shared-a-w0-c42c0a-openshift-default-558b6f6d6f-7rcsl"
Oct 30 22:17:46.311411 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:46.311297    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:17:46.437466 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:46.437434    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w0-c42c0a-openshift-default-558b6f6d6f-7rcsl"
Oct 30 22:17:46.876582 ip-10-0-136-210 kubenswrapper[5510]: W1030 22:17:46.876553    5510 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a76b5b9_88cf_4f7f_9d14_fd725017b62f.slice/crio-f7c15ebc7bdeb825ba84162e90afe48dcfbe5a9d923930cecd984d28c41ddf15 WatchSource:0}: Error finding container f7c15ebc7bdeb825ba84162e90afe48dcfbe5a9d923930cecd984d28c41ddf15: Status 404 returned error can't find the container with id f7c15ebc7bdeb825ba84162e90afe48dcfbe5a9d923930cecd984d28c41ddf15
Oct 30 22:17:46.879134 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:46.879100    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:17:46.879229 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:46.879166    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:17:46.879229 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:46.879197    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:17:46.879690 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:46.879669    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w0-c42c0a-openshift-default-558b6f6d6f-7rcsl"]
Oct 30 22:17:47.574729 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:47.574692    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-ff716ef6-6c87f5cf68-vrk8m"]
Oct 30 22:17:47.575203 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:47.574903    5510 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ff716ef6-6c87f5cf68-vrk8m" podUID="fa1f9fc4-483e-4423-9242-8cd97885bb41" containerName="maas-api" containerID="cri-o://74df5e6dbd1fbb7998d0ff7e9a5fd00fac8d13c1c245e3fcfd470afd3405f280" gracePeriod=30
Oct 30 22:17:47.701393 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:47.701354    5510 generic.go:356] "Generic (PLEG): container finished" podID="fa1f9fc4-483e-4423-9242-8cd97885bb41" containerID="74df5e6dbd1fbb7998d0ff7e9a5fd00fac8d13c1c245e3fcfd470afd3405f280" exitCode=0
Oct 30 22:17:47.701588 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:47.701565    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ff716ef6-6c87f5cf68-vrk8m" event={"ID":"fa1f9fc4-483e-4423-9242-8cd97885bb41","Type":"ContainerDied","Data":"74df5e6dbd1fbb7998d0ff7e9a5fd00fac8d13c1c245e3fcfd470afd3405f280"}
Oct 30 22:17:47.703635 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:47.703576    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w0-c42c0a-openshift-default-558b6f6d6f-7rcsl" event={"ID":"3a76b5b9-88cf-4f7f-9d14-fd725017b62f","Type":"ContainerStarted","Data":"f9f8ec7c0b142468c3f7862ce226aa9aa72fbaeb489447aefd7e17bb2921aae2"}
Oct 30 22:17:47.703784 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:47.703643    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w0-c42c0a-openshift-default-558b6f6d6f-7rcsl" event={"ID":"3a76b5b9-88cf-4f7f-9d14-fd725017b62f","Type":"ContainerStarted","Data":"f7c15ebc7bdeb825ba84162e90afe48dcfbe5a9d923930cecd984d28c41ddf15"}
Oct 30 22:17:47.710332 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:47.710286    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:17:48.281965 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:48.281897    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ff716ef6-6c87f5cf68-vrk8m"
Oct 30 22:17:48.330449 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:48.330114    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/fa1f9fc4-483e-4423-9242-8cd97885bb41-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fa1f9fc4-483e-4423-9242-8cd97885bb41-maas-api-tls\") pod \"fa1f9fc4-483e-4423-9242-8cd97885bb41\" (UID: \"fa1f9fc4-483e-4423-9242-8cd97885bb41\") "
Oct 30 22:17:48.330449 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:48.330284    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fa1f9fc4-483e-4423-9242-8cd97885bb41-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fa1f9fc4-483e-4423-9242-8cd97885bb41-openshift-service-ca\") pod \"fa1f9fc4-483e-4423-9242-8cd97885bb41\" (UID: \"fa1f9fc4-483e-4423-9242-8cd97885bb41\") "
Oct 30 22:17:48.330449 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:48.330422    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fa1f9fc4-483e-4423-9242-8cd97885bb41-kube-api-access-qwdz9\" (UniqueName: \"kubernetes.io/projected/fa1f9fc4-483e-4423-9242-8cd97885bb41-kube-api-access-qwdz9\") pod \"fa1f9fc4-483e-4423-9242-8cd97885bb41\" (UID: \"fa1f9fc4-483e-4423-9242-8cd97885bb41\") "
Oct 30 22:17:48.330709 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:48.330468    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/fa1f9fc4-483e-4423-9242-8cd97885bb41-metrics-certs\" (UniqueName: \"kubernetes.io/secret/fa1f9fc4-483e-4423-9242-8cd97885bb41-metrics-certs\") pod \"fa1f9fc4-483e-4423-9242-8cd97885bb41\" (UID: \"fa1f9fc4-483e-4423-9242-8cd97885bb41\") "
Oct 30 22:17:48.331398 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:48.331360    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fa1f9fc4-483e-4423-9242-8cd97885bb41-openshift-service-ca" pod "fa1f9fc4-483e-4423-9242-8cd97885bb41" (UID: "fa1f9fc4-483e-4423-9242-8cd97885bb41"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:17:48.334697 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:48.333881    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fa1f9fc4-483e-4423-9242-8cd97885bb41-maas-api-tls" pod "fa1f9fc4-483e-4423-9242-8cd97885bb41" (UID: "fa1f9fc4-483e-4423-9242-8cd97885bb41"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:17:48.334697 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:48.333961    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fa1f9fc4-483e-4423-9242-8cd97885bb41-kube-api-access-qwdz9" pod "fa1f9fc4-483e-4423-9242-8cd97885bb41" (UID: "fa1f9fc4-483e-4423-9242-8cd97885bb41"). InnerVolumeSpecName "kube-api-access-qwdz9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:17:48.334697 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:48.334042    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fa1f9fc4-483e-4423-9242-8cd97885bb41-metrics-certs" pod "fa1f9fc4-483e-4423-9242-8cd97885bb41" (UID: "fa1f9fc4-483e-4423-9242-8cd97885bb41"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:17:48.406016 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:48.405846    5510 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w0-c42c0a-openshift-default-558b6f6d6f-7rcsl" podStartSLOduration=3.405827416 podStartE2EDuration="3.405827416s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:17:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:17:47.895065139 +0000 UTC m=+1648.455875145" watchObservedRunningTime="2026-10-30 22:17:48.405827416 +0000 UTC m=+1648.966637422"
Oct 30 22:17:48.435355 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:48.435314    5510 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fa1f9fc4-483e-4423-9242-8cd97885bb41-openshift-service-ca\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:17:48.435355 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:48.435357    5510 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qwdz9\" (UniqueName: \"kubernetes.io/projected/fa1f9fc4-483e-4423-9242-8cd97885bb41-kube-api-access-qwdz9\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:17:48.435587 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:48.435373    5510 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fa1f9fc4-483e-4423-9242-8cd97885bb41-metrics-certs\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:17:48.435587 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:48.435388    5510 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fa1f9fc4-483e-4423-9242-8cd97885bb41-maas-api-tls\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:17:48.438378 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:48.438351    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w0-c42c0a-openshift-default-558b6f6d6f-7rcsl"
Oct 30 22:17:48.445713 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:48.445685    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w0-c42c0a-openshift-default-558b6f6d6f-7rcsl"
Oct 30 22:17:48.565097 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:48.565056    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:17:48.683789 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:48.683761    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-67184f2a-gw-openshift-default-5cb54945b9-5c64x"
Oct 30 22:17:48.713813 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:48.713774    5510 generic.go:356] "Generic (PLEG): container finished" podID="f067f5fc-05db-4807-982f-21e7a5997acc" containerID="ce5d937cbad5658849effa727bf0135def5e85e56e4cd5e7d24d4e4a8318f8cb" exitCode=0
Oct 30 22:17:48.713975 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:48.713848    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-67184f2a-gw-openshift-default-5cb54945b9-5c64x"
Oct 30 22:17:48.713975 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:48.713859    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-67184f2a-gw-openshift-default-5cb54945b9-5c64x" event={"ID":"f067f5fc-05db-4807-982f-21e7a5997acc","Type":"ContainerDied","Data":"ce5d937cbad5658849effa727bf0135def5e85e56e4cd5e7d24d4e4a8318f8cb"}
Oct 30 22:17:48.713975 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:48.713924    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-67184f2a-gw-openshift-default-5cb54945b9-5c64x" event={"ID":"f067f5fc-05db-4807-982f-21e7a5997acc","Type":"ContainerDied","Data":"b5fd0cae1043845ebf8ea572edf1a3890e743aef23f8579cb3589b6a282d56e9"}
Oct 30 22:17:48.713975 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:48.713956    5510 scope.go:122] "RemoveContainer" containerID="ce5d937cbad5658849effa727bf0135def5e85e56e4cd5e7d24d4e4a8318f8cb"
Oct 30 22:17:48.716080 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:48.716048    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ff716ef6-6c87f5cf68-vrk8m" event={"ID":"fa1f9fc4-483e-4423-9242-8cd97885bb41","Type":"ContainerDied","Data":"371125ca7fd6c35b70a6780db3344d38d084f3b0fd203975ca464febcab25fbf"}
Oct 30 22:17:48.716204 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:48.716060    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ff716ef6-6c87f5cf68-vrk8m"
Oct 30 22:17:48.716360 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:48.716342    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w0-c42c0a-openshift-default-558b6f6d6f-7rcsl"
Oct 30 22:17:48.717428 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:48.717410    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w0-c42c0a-openshift-default-558b6f6d6f-7rcsl"
Oct 30 22:17:48.726899 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:48.726874    5510 scope.go:122] "RemoveContainer" containerID="9ef24aa63fe5461c2c7c015615611beddd21df935aa93d916eb5ba896912143f"
Oct 30 22:17:48.738602 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:48.738577    5510 scope.go:122] "RemoveContainer" containerID="ce5d937cbad5658849effa727bf0135def5e85e56e4cd5e7d24d4e4a8318f8cb"
Oct 30 22:17:48.739035 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:17:48.739010    5510 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ce5d937cbad5658849effa727bf0135def5e85e56e4cd5e7d24d4e4a8318f8cb\": container with ID starting with ce5d937cbad5658849effa727bf0135def5e85e56e4cd5e7d24d4e4a8318f8cb not found: ID does not exist" containerID="ce5d937cbad5658849effa727bf0135def5e85e56e4cd5e7d24d4e4a8318f8cb"
Oct 30 22:17:48.739156 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:48.739039    5510 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ce5d937cbad5658849effa727bf0135def5e85e56e4cd5e7d24d4e4a8318f8cb"} err="failed to get container status \"ce5d937cbad5658849effa727bf0135def5e85e56e4cd5e7d24d4e4a8318f8cb\": rpc error: code = NotFound desc = could not find container \"ce5d937cbad5658849effa727bf0135def5e85e56e4cd5e7d24d4e4a8318f8cb\": container with ID starting with ce5d937cbad5658849effa727bf0135def5e85e56e4cd5e7d24d4e4a8318f8cb not found: ID does not exist"
Oct 30 22:17:48.739156 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:48.739059    5510 scope.go:122] "RemoveContainer" containerID="9ef24aa63fe5461c2c7c015615611beddd21df935aa93d916eb5ba896912143f"
Oct 30 22:17:48.739388 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:17:48.739372    5510 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9ef24aa63fe5461c2c7c015615611beddd21df935aa93d916eb5ba896912143f\": container with ID starting with 9ef24aa63fe5461c2c7c015615611beddd21df935aa93d916eb5ba896912143f not found: ID does not exist" containerID="9ef24aa63fe5461c2c7c015615611beddd21df935aa93d916eb5ba896912143f"
Oct 30 22:17:48.739450 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:48.739389    5510 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9ef24aa63fe5461c2c7c015615611beddd21df935aa93d916eb5ba896912143f"} err="failed to get container status \"9ef24aa63fe5461c2c7c015615611beddd21df935aa93d916eb5ba896912143f\": rpc error: code = NotFound desc = could not find container \"9ef24aa63fe5461c2c7c015615611beddd21df935aa93d916eb5ba896912143f\": container with ID starting with 9ef24aa63fe5461c2c7c015615611beddd21df935aa93d916eb5ba896912143f not found: ID does not exist"
Oct 30 22:17:48.739450 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:48.739406    5510 scope.go:122] "RemoveContainer" containerID="74df5e6dbd1fbb7998d0ff7e9a5fd00fac8d13c1c245e3fcfd470afd3405f280"
Oct 30 22:17:48.739450 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:48.739408    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f067f5fc-05db-4807-982f-21e7a5997acc-istio-token\" (UniqueName: \"kubernetes.io/projected/f067f5fc-05db-4807-982f-21e7a5997acc-istio-token\") pod \"f067f5fc-05db-4807-982f-21e7a5997acc\" (UID: \"f067f5fc-05db-4807-982f-21e7a5997acc\") "
Oct 30 22:17:48.739607 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:48.739454    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f067f5fc-05db-4807-982f-21e7a5997acc-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f067f5fc-05db-4807-982f-21e7a5997acc-istiod-ca-cert\") pod \"f067f5fc-05db-4807-982f-21e7a5997acc\" (UID: \"f067f5fc-05db-4807-982f-21e7a5997acc\") "
Oct 30 22:17:48.739607 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:48.739499    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f067f5fc-05db-4807-982f-21e7a5997acc-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f067f5fc-05db-4807-982f-21e7a5997acc-istio-envoy\") pod \"f067f5fc-05db-4807-982f-21e7a5997acc\" (UID: \"f067f5fc-05db-4807-982f-21e7a5997acc\") "
Oct 30 22:17:48.739607 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:48.739534    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f067f5fc-05db-4807-982f-21e7a5997acc-kube-api-access-p6g2v\" (UniqueName: \"kubernetes.io/projected/f067f5fc-05db-4807-982f-21e7a5997acc-kube-api-access-p6g2v\") pod \"f067f5fc-05db-4807-982f-21e7a5997acc\" (UID: \"f067f5fc-05db-4807-982f-21e7a5997acc\") "
Oct 30 22:17:48.739607 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:48.739571    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f067f5fc-05db-4807-982f-21e7a5997acc-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f067f5fc-05db-4807-982f-21e7a5997acc-workload-certs\") pod \"f067f5fc-05db-4807-982f-21e7a5997acc\" (UID: \"f067f5fc-05db-4807-982f-21e7a5997acc\") "
Oct 30 22:17:48.739830 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:48.739639    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f067f5fc-05db-4807-982f-21e7a5997acc-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f067f5fc-05db-4807-982f-21e7a5997acc-istio-data\") pod \"f067f5fc-05db-4807-982f-21e7a5997acc\" (UID: \"f067f5fc-05db-4807-982f-21e7a5997acc\") "
Oct 30 22:17:48.739880 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:48.739863    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f067f5fc-05db-4807-982f-21e7a5997acc-istiod-ca-cert" pod "f067f5fc-05db-4807-982f-21e7a5997acc" (UID: "f067f5fc-05db-4807-982f-21e7a5997acc"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:17:48.740090 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:48.740069    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f067f5fc-05db-4807-982f-21e7a5997acc-workload-certs" pod "f067f5fc-05db-4807-982f-21e7a5997acc" (UID: "f067f5fc-05db-4807-982f-21e7a5997acc"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:17:48.740155 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:48.740093    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/f067f5fc-05db-4807-982f-21e7a5997acc-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f067f5fc-05db-4807-982f-21e7a5997acc-istio-podinfo\") pod \"f067f5fc-05db-4807-982f-21e7a5997acc\" (UID: \"f067f5fc-05db-4807-982f-21e7a5997acc\") "
Oct 30 22:17:48.740266 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:48.740250    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f067f5fc-05db-4807-982f-21e7a5997acc-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f067f5fc-05db-4807-982f-21e7a5997acc-workload-socket\") pod \"f067f5fc-05db-4807-982f-21e7a5997acc\" (UID: \"f067f5fc-05db-4807-982f-21e7a5997acc\") "
Oct 30 22:17:48.740325 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:48.740282    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f067f5fc-05db-4807-982f-21e7a5997acc-istio-data" pod "f067f5fc-05db-4807-982f-21e7a5997acc" (UID: "f067f5fc-05db-4807-982f-21e7a5997acc"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:17:48.740554 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:48.740351    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f067f5fc-05db-4807-982f-21e7a5997acc-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f067f5fc-05db-4807-982f-21e7a5997acc-credential-socket\") pod \"f067f5fc-05db-4807-982f-21e7a5997acc\" (UID: \"f067f5fc-05db-4807-982f-21e7a5997acc\") "
Oct 30 22:17:48.740618 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:48.740567    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f067f5fc-05db-4807-982f-21e7a5997acc-workload-socket" pod "f067f5fc-05db-4807-982f-21e7a5997acc" (UID: "f067f5fc-05db-4807-982f-21e7a5997acc"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:17:48.740876 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:48.740849    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f067f5fc-05db-4807-982f-21e7a5997acc-credential-socket" pod "f067f5fc-05db-4807-982f-21e7a5997acc" (UID: "f067f5fc-05db-4807-982f-21e7a5997acc"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:17:48.741463 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:48.741382    5510 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f067f5fc-05db-4807-982f-21e7a5997acc-workload-socket\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:17:48.741463 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:48.741408    5510 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f067f5fc-05db-4807-982f-21e7a5997acc-credential-socket\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:17:48.741463 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:48.741423    5510 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f067f5fc-05db-4807-982f-21e7a5997acc-istiod-ca-cert\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:17:48.741463 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:48.741438    5510 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f067f5fc-05db-4807-982f-21e7a5997acc-workload-certs\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:17:48.741463 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:48.741452    5510 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f067f5fc-05db-4807-982f-21e7a5997acc-istio-data\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:17:48.742232 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:48.742206    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f067f5fc-05db-4807-982f-21e7a5997acc-istio-envoy" pod "f067f5fc-05db-4807-982f-21e7a5997acc" (UID: "f067f5fc-05db-4807-982f-21e7a5997acc"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:17:48.742371 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:48.742348    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f067f5fc-05db-4807-982f-21e7a5997acc-istio-token" pod "f067f5fc-05db-4807-982f-21e7a5997acc" (UID: "f067f5fc-05db-4807-982f-21e7a5997acc"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:17:48.742729 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:48.742698    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f067f5fc-05db-4807-982f-21e7a5997acc-istio-podinfo" pod "f067f5fc-05db-4807-982f-21e7a5997acc" (UID: "f067f5fc-05db-4807-982f-21e7a5997acc"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:17:48.742966 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:48.742943    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f067f5fc-05db-4807-982f-21e7a5997acc-kube-api-access-p6g2v" pod "f067f5fc-05db-4807-982f-21e7a5997acc" (UID: "f067f5fc-05db-4807-982f-21e7a5997acc"). InnerVolumeSpecName "kube-api-access-p6g2v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:17:48.850874 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:48.850589    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:17:48.854731 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:48.851259    5510 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f067f5fc-05db-4807-982f-21e7a5997acc-istio-token\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:17:48.854731 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:48.851289    5510 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f067f5fc-05db-4807-982f-21e7a5997acc-istio-envoy\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:17:48.854731 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:48.851304    5510 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p6g2v\" (UniqueName: \"kubernetes.io/projected/f067f5fc-05db-4807-982f-21e7a5997acc-kube-api-access-p6g2v\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:17:48.854731 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:48.851318    5510 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f067f5fc-05db-4807-982f-21e7a5997acc-istio-podinfo\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:17:49.207648 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:49.207594    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:17:49.338811 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:49.336680    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-ff716ef6-6c87f5cf68-vrk8m"]
Oct 30 22:17:49.363824 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:49.363790    5510 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-ff716ef6-6c87f5cf68-vrk8m"]
Oct 30 22:17:49.465284 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:49.465195    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-67184f2a-gw-openshift-default-5cb54945b9-5c64x"]
Oct 30 22:17:49.469511 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:49.469481    5510 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-67184f2a-gw-openshift-default-5cb54945b9-5c64x"]
Oct 30 22:17:49.652678 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:49.646270    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:17:49.669681 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:49.669007    5510 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f067f5fc-05db-4807-982f-21e7a5997acc" path="/var/lib/kubelet/pods/f067f5fc-05db-4807-982f-21e7a5997acc/volumes"
Oct 30 22:17:49.669886 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:49.669805    5510 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fa1f9fc4-483e-4423-9242-8cd97885bb41" path="/var/lib/kubelet/pods/fa1f9fc4-483e-4423-9242-8cd97885bb41/volumes"
Oct 30 22:17:49.966824 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:49.966783    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:17:50.992628 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:50.992593    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:17:51.339048 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:51.339013    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:17:51.614289 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:51.614204    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:17:51.874110 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:51.874035    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-72b5472b-openshift-default-6c5c978fc5-xs68s"]
Oct 30 22:17:51.874283 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:51.874255    5510 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-72b5472b-openshift-default-6c5c978fc5-xs68s" podUID="ebf683b5-5605-4790-813e-3f06b8f8c38f" containerName="istio-proxy" containerID="cri-o://820b206c67fa43a19deb4fd4d8194dd69ebc402da24c0acdfdd026efa12a50e8" gracePeriod=30
Oct 30 22:17:51.898738 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:51.898708    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:17:52.199052 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:52.198964    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:17:52.559010 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:52.558961    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:17:53.055776 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:53.055739    5510 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-bee2db53-openshift-default-6b56b8cdb9-jv7mv"]
Oct 30 22:17:53.056901 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:53.056872    5510 cpu_manager.go:414] "containerMap: removing container" podUID="fa1f9fc4-483e-4423-9242-8cd97885bb41" containerName="maas-api"
Oct 30 22:17:53.056901 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:53.056900    5510 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fa1f9fc4-483e-4423-9242-8cd97885bb41" containerName="maas-api"
Oct 30 22:17:53.057144 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:53.056915    5510 cpu_manager.go:414] "containerMap: removing container" podUID="f067f5fc-05db-4807-982f-21e7a5997acc" containerName="istio-proxy"
Oct 30 22:17:53.057144 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:53.056922    5510 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f067f5fc-05db-4807-982f-21e7a5997acc" containerName="istio-proxy"
Oct 30 22:17:53.057144 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:53.056937    5510 cpu_manager.go:414] "containerMap: removing container" podUID="f067f5fc-05db-4807-982f-21e7a5997acc" containerName="istio-proxy"
Oct 30 22:17:53.057144 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:53.056944    5510 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f067f5fc-05db-4807-982f-21e7a5997acc" containerName="istio-proxy"
Oct 30 22:17:53.057362 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:53.057185    5510 memory_manager.go:365] "RemoveStaleState removing state" podUID="f067f5fc-05db-4807-982f-21e7a5997acc" containerName="istio-proxy"
Oct 30 22:17:53.057362 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:53.057226    5510 memory_manager.go:365] "RemoveStaleState removing state" podUID="fa1f9fc4-483e-4423-9242-8cd97885bb41" containerName="maas-api"
Oct 30 22:17:53.062229 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:53.062172    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-bee2db53-openshift-default-6b56b8cdb9-jv7mv"
Oct 30 22:17:53.146060 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:53.145624    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/01891eb3-3336-4e31-a26a-9a1ecb935f9c-workload-certs\") pod \"e2e-derive-bee2db53-openshift-default-6b56b8cdb9-jv7mv\" (UID: \"01891eb3-3336-4e31-a26a-9a1ecb935f9c\") " pod="openshift-ingress/e2e-derive-bee2db53-openshift-default-6b56b8cdb9-jv7mv"
Oct 30 22:17:53.146424 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:53.146398    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/01891eb3-3336-4e31-a26a-9a1ecb935f9c-istiod-ca-cert\") pod \"e2e-derive-bee2db53-openshift-default-6b56b8cdb9-jv7mv\" (UID: \"01891eb3-3336-4e31-a26a-9a1ecb935f9c\") " pod="openshift-ingress/e2e-derive-bee2db53-openshift-default-6b56b8cdb9-jv7mv"
Oct 30 22:17:53.146645 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:53.146620    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/01891eb3-3336-4e31-a26a-9a1ecb935f9c-credential-socket\") pod \"e2e-derive-bee2db53-openshift-default-6b56b8cdb9-jv7mv\" (UID: \"01891eb3-3336-4e31-a26a-9a1ecb935f9c\") " pod="openshift-ingress/e2e-derive-bee2db53-openshift-default-6b56b8cdb9-jv7mv"
Oct 30 22:17:53.177827 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:53.177795    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/01891eb3-3336-4e31-a26a-9a1ecb935f9c-workload-socket\") pod \"e2e-derive-bee2db53-openshift-default-6b56b8cdb9-jv7mv\" (UID: \"01891eb3-3336-4e31-a26a-9a1ecb935f9c\") " pod="openshift-ingress/e2e-derive-bee2db53-openshift-default-6b56b8cdb9-jv7mv"
Oct 30 22:17:53.182586 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:53.182540    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/01891eb3-3336-4e31-a26a-9a1ecb935f9c-istio-envoy\") pod \"e2e-derive-bee2db53-openshift-default-6b56b8cdb9-jv7mv\" (UID: \"01891eb3-3336-4e31-a26a-9a1ecb935f9c\") " pod="openshift-ingress/e2e-derive-bee2db53-openshift-default-6b56b8cdb9-jv7mv"
Oct 30 22:17:53.182963 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:53.182937    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/01891eb3-3336-4e31-a26a-9a1ecb935f9c-istio-token\") pod \"e2e-derive-bee2db53-openshift-default-6b56b8cdb9-jv7mv\" (UID: \"01891eb3-3336-4e31-a26a-9a1ecb935f9c\") " pod="openshift-ingress/e2e-derive-bee2db53-openshift-default-6b56b8cdb9-jv7mv"
Oct 30 22:17:53.183161 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:53.183140    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qmnpj\" (UniqueName: \"kubernetes.io/projected/01891eb3-3336-4e31-a26a-9a1ecb935f9c-kube-api-access-qmnpj\") pod \"e2e-derive-bee2db53-openshift-default-6b56b8cdb9-jv7mv\" (UID: \"01891eb3-3336-4e31-a26a-9a1ecb935f9c\") " pod="openshift-ingress/e2e-derive-bee2db53-openshift-default-6b56b8cdb9-jv7mv"
Oct 30 22:17:53.183337 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:53.183319    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/01891eb3-3336-4e31-a26a-9a1ecb935f9c-istio-podinfo\") pod \"e2e-derive-bee2db53-openshift-default-6b56b8cdb9-jv7mv\" (UID: \"01891eb3-3336-4e31-a26a-9a1ecb935f9c\") " pod="openshift-ingress/e2e-derive-bee2db53-openshift-default-6b56b8cdb9-jv7mv"
Oct 30 22:17:53.183458 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:53.183429    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/01891eb3-3336-4e31-a26a-9a1ecb935f9c-istio-data\") pod \"e2e-derive-bee2db53-openshift-default-6b56b8cdb9-jv7mv\" (UID: \"01891eb3-3336-4e31-a26a-9a1ecb935f9c\") " pod="openshift-ingress/e2e-derive-bee2db53-openshift-default-6b56b8cdb9-jv7mv"
Oct 30 22:17:53.183801 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:53.181859    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-bee2db53-openshift-default-dockercfg-22rps\""
Oct 30 22:17:53.195042 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:53.195012    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-bee2db53-openshift-default-6b56b8cdb9-jv7mv"]
Oct 30 22:17:53.227468 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:53.227369    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:17:53.313738 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:53.293634    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/01891eb3-3336-4e31-a26a-9a1ecb935f9c-workload-socket\") pod \"e2e-derive-bee2db53-openshift-default-6b56b8cdb9-jv7mv\" (UID: \"01891eb3-3336-4e31-a26a-9a1ecb935f9c\") " pod="openshift-ingress/e2e-derive-bee2db53-openshift-default-6b56b8cdb9-jv7mv"
Oct 30 22:17:53.313738 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:53.293729    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/01891eb3-3336-4e31-a26a-9a1ecb935f9c-istio-envoy\") pod \"e2e-derive-bee2db53-openshift-default-6b56b8cdb9-jv7mv\" (UID: \"01891eb3-3336-4e31-a26a-9a1ecb935f9c\") " pod="openshift-ingress/e2e-derive-bee2db53-openshift-default-6b56b8cdb9-jv7mv"
Oct 30 22:17:53.313738 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:53.293798    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/01891eb3-3336-4e31-a26a-9a1ecb935f9c-istio-token\") pod \"e2e-derive-bee2db53-openshift-default-6b56b8cdb9-jv7mv\" (UID: \"01891eb3-3336-4e31-a26a-9a1ecb935f9c\") " pod="openshift-ingress/e2e-derive-bee2db53-openshift-default-6b56b8cdb9-jv7mv"
Oct 30 22:17:53.313738 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:53.293865    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qmnpj\" (UniqueName: \"kubernetes.io/projected/01891eb3-3336-4e31-a26a-9a1ecb935f9c-kube-api-access-qmnpj\") pod \"e2e-derive-bee2db53-openshift-default-6b56b8cdb9-jv7mv\" (UID: \"01891eb3-3336-4e31-a26a-9a1ecb935f9c\") " pod="openshift-ingress/e2e-derive-bee2db53-openshift-default-6b56b8cdb9-jv7mv"
Oct 30 22:17:53.313738 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:53.293934    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/01891eb3-3336-4e31-a26a-9a1ecb935f9c-istio-podinfo\") pod \"e2e-derive-bee2db53-openshift-default-6b56b8cdb9-jv7mv\" (UID: \"01891eb3-3336-4e31-a26a-9a1ecb935f9c\") " pod="openshift-ingress/e2e-derive-bee2db53-openshift-default-6b56b8cdb9-jv7mv"
Oct 30 22:17:53.313738 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:53.293963    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/01891eb3-3336-4e31-a26a-9a1ecb935f9c-istio-data\") pod \"e2e-derive-bee2db53-openshift-default-6b56b8cdb9-jv7mv\" (UID: \"01891eb3-3336-4e31-a26a-9a1ecb935f9c\") " pod="openshift-ingress/e2e-derive-bee2db53-openshift-default-6b56b8cdb9-jv7mv"
Oct 30 22:17:53.313738 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:53.294050    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/01891eb3-3336-4e31-a26a-9a1ecb935f9c-workload-certs\") pod \"e2e-derive-bee2db53-openshift-default-6b56b8cdb9-jv7mv\" (UID: \"01891eb3-3336-4e31-a26a-9a1ecb935f9c\") " pod="openshift-ingress/e2e-derive-bee2db53-openshift-default-6b56b8cdb9-jv7mv"
Oct 30 22:17:53.313738 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:53.294169    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/01891eb3-3336-4e31-a26a-9a1ecb935f9c-istiod-ca-cert\") pod \"e2e-derive-bee2db53-openshift-default-6b56b8cdb9-jv7mv\" (UID: \"01891eb3-3336-4e31-a26a-9a1ecb935f9c\") " pod="openshift-ingress/e2e-derive-bee2db53-openshift-default-6b56b8cdb9-jv7mv"
Oct 30 22:17:53.313738 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:53.294267    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/01891eb3-3336-4e31-a26a-9a1ecb935f9c-credential-socket\") pod \"e2e-derive-bee2db53-openshift-default-6b56b8cdb9-jv7mv\" (UID: \"01891eb3-3336-4e31-a26a-9a1ecb935f9c\") " pod="openshift-ingress/e2e-derive-bee2db53-openshift-default-6b56b8cdb9-jv7mv"
Oct 30 22:17:53.313738 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:53.294817    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/01891eb3-3336-4e31-a26a-9a1ecb935f9c-credential-socket\") pod \"e2e-derive-bee2db53-openshift-default-6b56b8cdb9-jv7mv\" (UID: \"01891eb3-3336-4e31-a26a-9a1ecb935f9c\") " pod="openshift-ingress/e2e-derive-bee2db53-openshift-default-6b56b8cdb9-jv7mv"
Oct 30 22:17:53.335751 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:53.315914    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/01891eb3-3336-4e31-a26a-9a1ecb935f9c-workload-certs\") pod \"e2e-derive-bee2db53-openshift-default-6b56b8cdb9-jv7mv\" (UID: \"01891eb3-3336-4e31-a26a-9a1ecb935f9c\") " pod="openshift-ingress/e2e-derive-bee2db53-openshift-default-6b56b8cdb9-jv7mv"
Oct 30 22:17:53.335751 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:53.316294    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/01891eb3-3336-4e31-a26a-9a1ecb935f9c-istio-data\") pod \"e2e-derive-bee2db53-openshift-default-6b56b8cdb9-jv7mv\" (UID: \"01891eb3-3336-4e31-a26a-9a1ecb935f9c\") " pod="openshift-ingress/e2e-derive-bee2db53-openshift-default-6b56b8cdb9-jv7mv"
Oct 30 22:17:53.335751 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:53.317177    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/01891eb3-3336-4e31-a26a-9a1ecb935f9c-istiod-ca-cert\") pod \"e2e-derive-bee2db53-openshift-default-6b56b8cdb9-jv7mv\" (UID: \"01891eb3-3336-4e31-a26a-9a1ecb935f9c\") " pod="openshift-ingress/e2e-derive-bee2db53-openshift-default-6b56b8cdb9-jv7mv"
Oct 30 22:17:53.364248 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:53.339031    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/01891eb3-3336-4e31-a26a-9a1ecb935f9c-workload-socket\") pod \"e2e-derive-bee2db53-openshift-default-6b56b8cdb9-jv7mv\" (UID: \"01891eb3-3336-4e31-a26a-9a1ecb935f9c\") " pod="openshift-ingress/e2e-derive-bee2db53-openshift-default-6b56b8cdb9-jv7mv"
Oct 30 22:17:53.482730 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:53.481753    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/01891eb3-3336-4e31-a26a-9a1ecb935f9c-istio-envoy\") pod \"e2e-derive-bee2db53-openshift-default-6b56b8cdb9-jv7mv\" (UID: \"01891eb3-3336-4e31-a26a-9a1ecb935f9c\") " pod="openshift-ingress/e2e-derive-bee2db53-openshift-default-6b56b8cdb9-jv7mv"
Oct 30 22:17:53.545946 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:53.544748    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/01891eb3-3336-4e31-a26a-9a1ecb935f9c-istio-podinfo\") pod \"e2e-derive-bee2db53-openshift-default-6b56b8cdb9-jv7mv\" (UID: \"01891eb3-3336-4e31-a26a-9a1ecb935f9c\") " pod="openshift-ingress/e2e-derive-bee2db53-openshift-default-6b56b8cdb9-jv7mv"
Oct 30 22:17:53.587725 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:53.587604    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:17:53.642165 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:53.642131    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qmnpj\" (UniqueName: \"kubernetes.io/projected/01891eb3-3336-4e31-a26a-9a1ecb935f9c-kube-api-access-qmnpj\") pod \"e2e-derive-bee2db53-openshift-default-6b56b8cdb9-jv7mv\" (UID: \"01891eb3-3336-4e31-a26a-9a1ecb935f9c\") " pod="openshift-ingress/e2e-derive-bee2db53-openshift-default-6b56b8cdb9-jv7mv"
Oct 30 22:17:53.645381 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:53.645275    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/01891eb3-3336-4e31-a26a-9a1ecb935f9c-istio-token\") pod \"e2e-derive-bee2db53-openshift-default-6b56b8cdb9-jv7mv\" (UID: \"01891eb3-3336-4e31-a26a-9a1ecb935f9c\") " pod="openshift-ingress/e2e-derive-bee2db53-openshift-default-6b56b8cdb9-jv7mv"
Oct 30 22:17:53.714268 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:53.714231    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-bee2db53-openshift-default-6b56b8cdb9-jv7mv"
Oct 30 22:17:54.178185 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:54.159377    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:17:54.281680 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:54.274910    5510 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-ac5024ad-6bfdb47b7-fxbsd"]
Oct 30 22:17:54.290675 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:54.285529    5510 memory_manager.go:365] "RemoveStaleState removing state" podUID="f067f5fc-05db-4807-982f-21e7a5997acc" containerName="istio-proxy"
Oct 30 22:17:54.314678 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:54.311466    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-ac5024ad-6bfdb47b7-fxbsd"
Oct 30 22:17:54.420678 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:54.418225    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-ac5024ad-dockercfg-fxb62\""
Oct 30 22:17:54.432019 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:54.431566    5510 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-ac5024ad\""
Oct 30 22:17:54.439729 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:54.439101    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:17:54.465741 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:54.460416    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jlzgt\" (UniqueName: \"kubernetes.io/projected/a08bf006-75ff-4597-8ec3-b6d8da49d808-kube-api-access-jlzgt\") pod \"payload-pre-processing-e2e-ait-ac5024ad-6bfdb47b7-fxbsd\" (UID: \"a08bf006-75ff-4597-8ec3-b6d8da49d808\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-ac5024ad-6bfdb47b7-fxbsd"
Oct 30 22:17:54.476706 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:54.474076    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a08bf006-75ff-4597-8ec3-b6d8da49d808-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-ac5024ad-6bfdb47b7-fxbsd\" (UID: \"a08bf006-75ff-4597-8ec3-b6d8da49d808\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-ac5024ad-6bfdb47b7-fxbsd"
Oct 30 22:17:54.581103 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:54.578460    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a08bf006-75ff-4597-8ec3-b6d8da49d808-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-ac5024ad-6bfdb47b7-fxbsd\" (UID: \"a08bf006-75ff-4597-8ec3-b6d8da49d808\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-ac5024ad-6bfdb47b7-fxbsd"
Oct 30 22:17:54.581103 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:54.578697    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jlzgt\" (UniqueName: \"kubernetes.io/projected/a08bf006-75ff-4597-8ec3-b6d8da49d808-kube-api-access-jlzgt\") pod \"payload-pre-processing-e2e-ait-ac5024ad-6bfdb47b7-fxbsd\" (UID: \"a08bf006-75ff-4597-8ec3-b6d8da49d808\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-ac5024ad-6bfdb47b7-fxbsd"
Oct 30 22:17:54.603856 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:54.588175    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a08bf006-75ff-4597-8ec3-b6d8da49d808-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-ac5024ad-6bfdb47b7-fxbsd\" (UID: \"a08bf006-75ff-4597-8ec3-b6d8da49d808\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-ac5024ad-6bfdb47b7-fxbsd"
Oct 30 22:17:54.644683 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:54.631046    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-ac5024ad-6bfdb47b7-fxbsd"]
Oct 30 22:17:54.689681 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:54.686209    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jlzgt\" (UniqueName: \"kubernetes.io/projected/a08bf006-75ff-4597-8ec3-b6d8da49d808-kube-api-access-jlzgt\") pod \"payload-pre-processing-e2e-ait-ac5024ad-6bfdb47b7-fxbsd\" (UID: \"a08bf006-75ff-4597-8ec3-b6d8da49d808\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-ac5024ad-6bfdb47b7-fxbsd"
Oct 30 22:17:54.700720 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:54.696797    5510 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-ac5024ad-6685bb6d9f-prktq"]
Oct 30 22:17:54.707592 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:54.707563    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-ac5024ad-6685bb6d9f-prktq"]
Oct 30 22:17:54.707927 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:54.707908    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-ac5024ad-6685bb6d9f-prktq"
Oct 30 22:17:54.763676 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:54.760166    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-ac5024ad-6bfdb47b7-fxbsd"
Oct 30 22:17:54.839260 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:54.839145    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g75ph\" (UniqueName: \"kubernetes.io/projected/d9e4f1df-768a-4b77-aa2f-0fa8f8d2818f-kube-api-access-g75ph\") pod \"payload-processing-e2e-ait-ac5024ad-6685bb6d9f-prktq\" (UID: \"d9e4f1df-768a-4b77-aa2f-0fa8f8d2818f\") " pod="openshift-ingress/payload-processing-e2e-ait-ac5024ad-6685bb6d9f-prktq"
Oct 30 22:17:54.839884 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:54.839852    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d9e4f1df-768a-4b77-aa2f-0fa8f8d2818f-plugins-config-volume\") pod \"payload-processing-e2e-ait-ac5024ad-6685bb6d9f-prktq\" (UID: \"d9e4f1df-768a-4b77-aa2f-0fa8f8d2818f\") " pod="openshift-ingress/payload-processing-e2e-ait-ac5024ad-6685bb6d9f-prktq"
Oct 30 22:17:54.973124 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:54.947044    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d9e4f1df-768a-4b77-aa2f-0fa8f8d2818f-plugins-config-volume\") pod \"payload-processing-e2e-ait-ac5024ad-6685bb6d9f-prktq\" (UID: \"d9e4f1df-768a-4b77-aa2f-0fa8f8d2818f\") " pod="openshift-ingress/payload-processing-e2e-ait-ac5024ad-6685bb6d9f-prktq"
Oct 30 22:17:55.096014 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:55.078300    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g75ph\" (UniqueName: \"kubernetes.io/projected/d9e4f1df-768a-4b77-aa2f-0fa8f8d2818f-kube-api-access-g75ph\") pod \"payload-processing-e2e-ait-ac5024ad-6685bb6d9f-prktq\" (UID: \"d9e4f1df-768a-4b77-aa2f-0fa8f8d2818f\") " pod="openshift-ingress/payload-processing-e2e-ait-ac5024ad-6685bb6d9f-prktq"
Oct 30 22:17:55.096014 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:55.084642    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d9e4f1df-768a-4b77-aa2f-0fa8f8d2818f-plugins-config-volume\") pod \"payload-processing-e2e-ait-ac5024ad-6685bb6d9f-prktq\" (UID: \"d9e4f1df-768a-4b77-aa2f-0fa8f8d2818f\") " pod="openshift-ingress/payload-processing-e2e-ait-ac5024ad-6685bb6d9f-prktq"
Oct 30 22:17:55.148705 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:55.148674    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g75ph\" (UniqueName: \"kubernetes.io/projected/d9e4f1df-768a-4b77-aa2f-0fa8f8d2818f-kube-api-access-g75ph\") pod \"payload-processing-e2e-ait-ac5024ad-6685bb6d9f-prktq\" (UID: \"d9e4f1df-768a-4b77-aa2f-0fa8f8d2818f\") " pod="openshift-ingress/payload-processing-e2e-ait-ac5024ad-6685bb6d9f-prktq"
Oct 30 22:17:55.338531 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:55.338492    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-ac5024ad-6685bb6d9f-prktq"
Oct 30 22:17:55.387584 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:55.387389    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-bee2db53-openshift-default-6b56b8cdb9-jv7mv"]
Oct 30 22:17:55.454754 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:55.454691    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:17:55.454876 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:55.454852    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:17:55.454960 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:55.454938    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:17:55.462945 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:55.458990    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-ac5024ad-6bfdb47b7-fxbsd"]
Oct 30 22:17:55.731127 ip-10-0-136-210 kubenswrapper[5510]: W1030 22:17:55.731094    5510 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9e4f1df_768a_4b77_aa2f_0fa8f8d2818f.slice/crio-3710688cb4212dd120a49b9a0edbe1f5c47c7c1e38d748ba235271fa10199b08 WatchSource:0}: Error finding container 3710688cb4212dd120a49b9a0edbe1f5c47c7c1e38d748ba235271fa10199b08: Status 404 returned error can't find the container with id 3710688cb4212dd120a49b9a0edbe1f5c47c7c1e38d748ba235271fa10199b08
Oct 30 22:17:55.734249 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:55.734197    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-ac5024ad-6685bb6d9f-prktq"]
Oct 30 22:17:56.202066 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:56.202030    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-bee2db53-openshift-default-6b56b8cdb9-jv7mv" event={"ID":"01891eb3-3336-4e31-a26a-9a1ecb935f9c","Type":"ContainerStarted","Data":"15c70c82c935624bfc0094fd21db370e95de86bdda73f8cb176f92455e7fd968"}
Oct 30 22:17:56.202268 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:56.202096    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-bee2db53-openshift-default-6b56b8cdb9-jv7mv" event={"ID":"01891eb3-3336-4e31-a26a-9a1ecb935f9c","Type":"ContainerStarted","Data":"62dd22d27a87229f702e34b1cebbfb63f2f8f55e7a892ae634ac637937a8ec18"}
Oct 30 22:17:56.203804 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:56.203773    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-ac5024ad-6bfdb47b7-fxbsd" event={"ID":"a08bf006-75ff-4597-8ec3-b6d8da49d808","Type":"ContainerStarted","Data":"f5c9187b5e793a4f18a050a02b0fec68abc6f42ccf50ade2705392380d872267"}
Oct 30 22:17:56.203958 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:56.203818    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-ac5024ad-6bfdb47b7-fxbsd" event={"ID":"a08bf006-75ff-4597-8ec3-b6d8da49d808","Type":"ContainerStarted","Data":"71ca3d9fd1e755ed1138ac1606f10bc229bbbd4712d09c2ec5b4f50f61450dc1"}
Oct 30 22:17:56.203958 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:56.203867    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-ac5024ad-6bfdb47b7-fxbsd"
Oct 30 22:17:56.205547 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:56.205523    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-ac5024ad-6685bb6d9f-prktq" event={"ID":"d9e4f1df-768a-4b77-aa2f-0fa8f8d2818f","Type":"ContainerStarted","Data":"20745cc50eabc223ae637cbf5076590e1064bca5f33fd8ece001ae21b83a81b6"}
Oct 30 22:17:56.205704 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:56.205564    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-ac5024ad-6685bb6d9f-prktq" event={"ID":"d9e4f1df-768a-4b77-aa2f-0fa8f8d2818f","Type":"ContainerStarted","Data":"3710688cb4212dd120a49b9a0edbe1f5c47c7c1e38d748ba235271fa10199b08"}
Oct 30 22:17:56.205704 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:56.205593    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-ac5024ad-6685bb6d9f-prktq"
Oct 30 22:17:56.328127 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:56.328072    5510 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-bee2db53-openshift-default-6b56b8cdb9-jv7mv" podStartSLOduration=3.328058589 podStartE2EDuration="3.328058589s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:17:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:17:56.326595202 +0000 UTC m=+1656.887405209" watchObservedRunningTime="2026-10-30 22:17:56.328058589 +0000 UTC m=+1656.888868594"
Oct 30 22:17:56.508729 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:56.500289    5510 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-ac5024ad-6bfdb47b7-fxbsd" podStartSLOduration=2.500267511 podStartE2EDuration="2.500267511s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:17:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:17:56.457016505 +0000 UTC m=+1657.017826512" watchObservedRunningTime="2026-10-30 22:17:56.500267511 +0000 UTC m=+1657.061077517"
Oct 30 22:17:56.737680 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:56.720295    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-bee2db53-openshift-default-6b56b8cdb9-jv7mv"
Oct 30 22:17:56.738906 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:56.738877    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-bee2db53-openshift-default-6b56b8cdb9-jv7mv"
Oct 30 22:17:56.745591 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:56.744492    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:17:56.962056 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:56.943330    5510 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-ac5024ad-6685bb6d9f-prktq" podStartSLOduration=2.943308416 podStartE2EDuration="2.943308416s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:17:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:17:56.60140599 +0000 UTC m=+1657.162215994" watchObservedRunningTime="2026-10-30 22:17:56.943308416 +0000 UTC m=+1657.504118422"
Oct 30 22:17:57.193733 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:57.170605    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:17:57.330124 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:57.329132    5510 generic.go:356] "Generic (PLEG): container finished" podID="ebf683b5-5605-4790-813e-3f06b8f8c38f" containerID="820b206c67fa43a19deb4fd4d8194dd69ebc402da24c0acdfdd026efa12a50e8" exitCode=0
Oct 30 22:17:57.331065 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:57.331023    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-72b5472b-openshift-default-6c5c978fc5-xs68s" event={"ID":"ebf683b5-5605-4790-813e-3f06b8f8c38f","Type":"ContainerDied","Data":"820b206c67fa43a19deb4fd4d8194dd69ebc402da24c0acdfdd026efa12a50e8"}
Oct 30 22:17:57.331293 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:57.331262    5510 scope.go:122] "RemoveContainer" containerID="0e581049efefcb2f4e1c0c85388ccfce3199fe27e9e1336b26e6ba65e0e0c000"
Oct 30 22:17:57.334261 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:57.334244    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-bee2db53-openshift-default-6b56b8cdb9-jv7mv"
Oct 30 22:17:57.339157 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:57.339137    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-bee2db53-openshift-default-6b56b8cdb9-jv7mv"
Oct 30 22:17:57.598995 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:57.592618    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:17:57.997382 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:57.997355    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-72b5472b-openshift-default-6c5c978fc5-xs68s"
Oct 30 22:17:58.192818 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:58.192786    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ebf683b5-5605-4790-813e-3f06b8f8c38f-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ebf683b5-5605-4790-813e-3f06b8f8c38f-workload-certs\") pod \"ebf683b5-5605-4790-813e-3f06b8f8c38f\" (UID: \"ebf683b5-5605-4790-813e-3f06b8f8c38f\") "
Oct 30 22:17:58.193011 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:58.192913    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ebf683b5-5605-4790-813e-3f06b8f8c38f-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ebf683b5-5605-4790-813e-3f06b8f8c38f-istio-data\") pod \"ebf683b5-5605-4790-813e-3f06b8f8c38f\" (UID: \"ebf683b5-5605-4790-813e-3f06b8f8c38f\") "
Oct 30 22:17:58.193011 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:58.192961    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ebf683b5-5605-4790-813e-3f06b8f8c38f-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ebf683b5-5605-4790-813e-3f06b8f8c38f-istiod-ca-cert\") pod \"ebf683b5-5605-4790-813e-3f06b8f8c38f\" (UID: \"ebf683b5-5605-4790-813e-3f06b8f8c38f\") "
Oct 30 22:17:58.193011 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:58.192982    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ebf683b5-5605-4790-813e-3f06b8f8c38f-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ebf683b5-5605-4790-813e-3f06b8f8c38f-workload-socket\") pod \"ebf683b5-5605-4790-813e-3f06b8f8c38f\" (UID: \"ebf683b5-5605-4790-813e-3f06b8f8c38f\") "
Oct 30 22:17:58.193011 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:58.193006    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ebf683b5-5605-4790-813e-3f06b8f8c38f-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ebf683b5-5605-4790-813e-3f06b8f8c38f-credential-socket\") pod \"ebf683b5-5605-4790-813e-3f06b8f8c38f\" (UID: \"ebf683b5-5605-4790-813e-3f06b8f8c38f\") "
Oct 30 22:17:58.193232 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:58.193039    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ebf683b5-5605-4790-813e-3f06b8f8c38f-kube-api-access-jml4m\" (UniqueName: \"kubernetes.io/projected/ebf683b5-5605-4790-813e-3f06b8f8c38f-kube-api-access-jml4m\") pod \"ebf683b5-5605-4790-813e-3f06b8f8c38f\" (UID: \"ebf683b5-5605-4790-813e-3f06b8f8c38f\") "
Oct 30 22:17:58.193232 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:58.193081    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ebf683b5-5605-4790-813e-3f06b8f8c38f-workload-certs" pod "ebf683b5-5605-4790-813e-3f06b8f8c38f" (UID: "ebf683b5-5605-4790-813e-3f06b8f8c38f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:17:58.193232 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:58.193112    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ebf683b5-5605-4790-813e-3f06b8f8c38f-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ebf683b5-5605-4790-813e-3f06b8f8c38f-istio-envoy\") pod \"ebf683b5-5605-4790-813e-3f06b8f8c38f\" (UID: \"ebf683b5-5605-4790-813e-3f06b8f8c38f\") "
Oct 30 22:17:58.193232 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:58.193214    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/ebf683b5-5605-4790-813e-3f06b8f8c38f-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ebf683b5-5605-4790-813e-3f06b8f8c38f-istio-podinfo\") pod \"ebf683b5-5605-4790-813e-3f06b8f8c38f\" (UID: \"ebf683b5-5605-4790-813e-3f06b8f8c38f\") "
Oct 30 22:17:58.193456 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:58.193252    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ebf683b5-5605-4790-813e-3f06b8f8c38f-istio-data" pod "ebf683b5-5605-4790-813e-3f06b8f8c38f" (UID: "ebf683b5-5605-4790-813e-3f06b8f8c38f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:17:58.193456 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:58.193269    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ebf683b5-5605-4790-813e-3f06b8f8c38f-workload-socket" pod "ebf683b5-5605-4790-813e-3f06b8f8c38f" (UID: "ebf683b5-5605-4790-813e-3f06b8f8c38f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:17:58.193456 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:58.193374    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ebf683b5-5605-4790-813e-3f06b8f8c38f-credential-socket" pod "ebf683b5-5605-4790-813e-3f06b8f8c38f" (UID: "ebf683b5-5605-4790-813e-3f06b8f8c38f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:17:58.193456 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:58.193387    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ebf683b5-5605-4790-813e-3f06b8f8c38f-istio-token\" (UniqueName: \"kubernetes.io/projected/ebf683b5-5605-4790-813e-3f06b8f8c38f-istio-token\") pod \"ebf683b5-5605-4790-813e-3f06b8f8c38f\" (UID: \"ebf683b5-5605-4790-813e-3f06b8f8c38f\") "
Oct 30 22:17:58.193705 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:58.193384    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ebf683b5-5605-4790-813e-3f06b8f8c38f-istiod-ca-cert" pod "ebf683b5-5605-4790-813e-3f06b8f8c38f" (UID: "ebf683b5-5605-4790-813e-3f06b8f8c38f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:17:58.194163 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:58.194139    5510 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ebf683b5-5605-4790-813e-3f06b8f8c38f-workload-certs\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:17:58.194227 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:58.194171    5510 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ebf683b5-5605-4790-813e-3f06b8f8c38f-istio-data\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:17:58.194227 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:58.194186    5510 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ebf683b5-5605-4790-813e-3f06b8f8c38f-istiod-ca-cert\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:17:58.194227 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:58.194201    5510 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ebf683b5-5605-4790-813e-3f06b8f8c38f-workload-socket\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:17:58.194227 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:58.194217    5510 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ebf683b5-5605-4790-813e-3f06b8f8c38f-credential-socket\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:17:58.195564 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:58.195543    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ebf683b5-5605-4790-813e-3f06b8f8c38f-istio-token" pod "ebf683b5-5605-4790-813e-3f06b8f8c38f" (UID: "ebf683b5-5605-4790-813e-3f06b8f8c38f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:17:58.195690 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:58.195570    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ebf683b5-5605-4790-813e-3f06b8f8c38f-kube-api-access-jml4m" pod "ebf683b5-5605-4790-813e-3f06b8f8c38f" (UID: "ebf683b5-5605-4790-813e-3f06b8f8c38f"). InnerVolumeSpecName "kube-api-access-jml4m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:17:58.195838 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:58.195820    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ebf683b5-5605-4790-813e-3f06b8f8c38f-istio-envoy" pod "ebf683b5-5605-4790-813e-3f06b8f8c38f" (UID: "ebf683b5-5605-4790-813e-3f06b8f8c38f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:17:58.195876 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:58.195855    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ebf683b5-5605-4790-813e-3f06b8f8c38f-istio-podinfo" pod "ebf683b5-5605-4790-813e-3f06b8f8c38f" (UID: "ebf683b5-5605-4790-813e-3f06b8f8c38f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:17:58.295514 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:58.295477    5510 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ebf683b5-5605-4790-813e-3f06b8f8c38f-istio-token\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:17:58.295514 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:58.295512    5510 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jml4m\" (UniqueName: \"kubernetes.io/projected/ebf683b5-5605-4790-813e-3f06b8f8c38f-kube-api-access-jml4m\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:17:58.295755 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:58.295527    5510 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ebf683b5-5605-4790-813e-3f06b8f8c38f-istio-envoy\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:17:58.295755 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:58.295541    5510 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ebf683b5-5605-4790-813e-3f06b8f8c38f-istio-podinfo\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:17:58.338456 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:58.338425    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-72b5472b-openshift-default-6c5c978fc5-xs68s"
Oct 30 22:17:58.338651 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:58.338453    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-72b5472b-openshift-default-6c5c978fc5-xs68s" event={"ID":"ebf683b5-5605-4790-813e-3f06b8f8c38f","Type":"ContainerDied","Data":"0da2a89ce6767841f3a34e17f23c0fa62dc660410deaf724d5dc82e0c8c8c4c9"}
Oct 30 22:17:58.338651 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:58.338521    5510 scope.go:122] "RemoveContainer" containerID="820b206c67fa43a19deb4fd4d8194dd69ebc402da24c0acdfdd026efa12a50e8"
Oct 30 22:17:58.507707 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:58.507648    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:17:58.509015 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:58.508988    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-72b5472b-openshift-default-6c5c978fc5-xs68s"]
Oct 30 22:17:58.515802 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:58.515779    5510 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-72b5472b-openshift-default-6c5c978fc5-xs68s"]
Oct 30 22:17:58.759573 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:58.759490    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:17:59.556719 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:17:59.556645    5510 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ebf683b5-5605-4790-813e-3f06b8f8c38f" path="/var/lib/kubelet/pods/ebf683b5-5605-4790-813e-3f06b8f8c38f/volumes"
Oct 30 22:18:01.567142 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:18:01.567087    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:01.899784 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:18:01.898579    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:02.193732 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:18:02.193601    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:02.305678 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:18:02.302934    5510 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-bee2db53-78cf85d8ff-w2jnq"]
Oct 30 22:18:02.305678 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:18:02.304274    5510 cpu_manager.go:414] "containerMap: removing container" podUID="ebf683b5-5605-4790-813e-3f06b8f8c38f" containerName="istio-proxy"
Oct 30 22:18:02.305678 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:18:02.304295    5510 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ebf683b5-5605-4790-813e-3f06b8f8c38f" containerName="istio-proxy"
Oct 30 22:18:02.305678 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:18:02.304350    5510 cpu_manager.go:414] "containerMap: removing container" podUID="ebf683b5-5605-4790-813e-3f06b8f8c38f" containerName="istio-proxy"
Oct 30 22:18:02.305678 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:18:02.304357    5510 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ebf683b5-5605-4790-813e-3f06b8f8c38f" containerName="istio-proxy"
Oct 30 22:18:02.305678 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:18:02.304575    5510 memory_manager.go:365] "RemoveStaleState removing state" podUID="ebf683b5-5605-4790-813e-3f06b8f8c38f" containerName="istio-proxy"
Oct 30 22:18:02.305678 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:18:02.304599    5510 memory_manager.go:365] "RemoveStaleState removing state" podUID="ebf683b5-5605-4790-813e-3f06b8f8c38f" containerName="istio-proxy"
Oct 30 22:18:02.347646 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:18:02.346751    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-bee2db53-78cf85d8ff-w2jnq"
Oct 30 22:18:02.372682 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:18:02.368691    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-bee2db53-78cf85d8ff-w2jnq"]
Oct 30 22:18:02.520732 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:18:02.426219    5510 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-bee2db53\""
Oct 30 22:18:02.520732 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:18:02.477768    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/236afc02-33f3-40f9-b5ee-16816eac2442-plugins-config-volume\") pod \"payload-processing-e2e-derive-bee2db53-78cf85d8ff-w2jnq\" (UID: \"236afc02-33f3-40f9-b5ee-16816eac2442\") " pod="openshift-ingress/payload-processing-e2e-derive-bee2db53-78cf85d8ff-w2jnq"
Oct 30 22:18:02.522887 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:18:02.521325    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-82qmq\" (UniqueName: \"kubernetes.io/projected/236afc02-33f3-40f9-b5ee-16816eac2442-kube-api-access-82qmq\") pod \"payload-processing-e2e-derive-bee2db53-78cf85d8ff-w2jnq\" (UID: \"236afc02-33f3-40f9-b5ee-16816eac2442\") " pod="openshift-ingress/payload-processing-e2e-derive-bee2db53-78cf85d8ff-w2jnq"
Oct 30 22:18:02.611698 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:18:02.610246    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-bee2db53-dockercfg-5rdsl\""
Oct 30 22:18:02.640692 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:18:02.629400    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/236afc02-33f3-40f9-b5ee-16816eac2442-plugins-config-volume\") pod \"payload-processing-e2e-derive-bee2db53-78cf85d8ff-w2jnq\" (UID: \"236afc02-33f3-40f9-b5ee-16816eac2442\") " pod="openshift-ingress/payload-processing-e2e-derive-bee2db53-78cf85d8ff-w2jnq"
Oct 30 22:18:02.640692 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:18:02.629499    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-82qmq\" (UniqueName: \"kubernetes.io/projected/236afc02-33f3-40f9-b5ee-16816eac2442-kube-api-access-82qmq\") pod \"payload-processing-e2e-derive-bee2db53-78cf85d8ff-w2jnq\" (UID: \"236afc02-33f3-40f9-b5ee-16816eac2442\") " pod="openshift-ingress/payload-processing-e2e-derive-bee2db53-78cf85d8ff-w2jnq"
Oct 30 22:18:02.640692 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:18:02.631384    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/236afc02-33f3-40f9-b5ee-16816eac2442-plugins-config-volume\") pod \"payload-processing-e2e-derive-bee2db53-78cf85d8ff-w2jnq\" (UID: \"236afc02-33f3-40f9-b5ee-16816eac2442\") " pod="openshift-ingress/payload-processing-e2e-derive-bee2db53-78cf85d8ff-w2jnq"
Oct 30 22:18:02.746868 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:18:02.743367    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-82qmq\" (UniqueName: \"kubernetes.io/projected/236afc02-33f3-40f9-b5ee-16816eac2442-kube-api-access-82qmq\") pod \"payload-processing-e2e-derive-bee2db53-78cf85d8ff-w2jnq\" (UID: \"236afc02-33f3-40f9-b5ee-16816eac2442\") " pod="openshift-ingress/payload-processing-e2e-derive-bee2db53-78cf85d8ff-w2jnq"
Oct 30 22:18:03.024204 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:18:03.024175    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-bee2db53-78cf85d8ff-w2jnq"
Oct 30 22:18:03.425011 ip-10-0-136-210 kubenswrapper[5510]: W1030 22:18:03.424982    5510 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod236afc02_33f3_40f9_b5ee_16816eac2442.slice/crio-d612df8495b9cae815b6ee3e4f63950f21035eb6c5cb4fc0e5b8b9b93992ee01 WatchSource:0}: Error finding container d612df8495b9cae815b6ee3e4f63950f21035eb6c5cb4fc0e5b8b9b93992ee01: Status 404 returned error can't find the container with id d612df8495b9cae815b6ee3e4f63950f21035eb6c5cb4fc0e5b8b9b93992ee01
Oct 30 22:18:03.428166 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:18:03.428136    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-bee2db53-78cf85d8ff-w2jnq"]
Oct 30 22:18:03.517692 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:18:03.514429    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:03.634017 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:18:03.633980    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-bee2db53-78cf85d8ff-w2jnq" event={"ID":"236afc02-33f3-40f9-b5ee-16816eac2442","Type":"ContainerStarted","Data":"d612df8495b9cae815b6ee3e4f63950f21035eb6c5cb4fc0e5b8b9b93992ee01"}
Oct 30 22:18:04.642979 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:18:04.642947    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-bee2db53-78cf85d8ff-w2jnq" event={"ID":"236afc02-33f3-40f9-b5ee-16816eac2442","Type":"ContainerStarted","Data":"236ed3024d7f71f5032ffbb6b27644990cbf44b44d8a7c685b566516957f9e0b"}
Oct 30 22:18:04.643480 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:18:04.643010    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-derive-bee2db53-78cf85d8ff-w2jnq"
Oct 30 22:18:04.770250 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:18:04.770192    5510 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-derive-bee2db53-78cf85d8ff-w2jnq" podStartSLOduration=2.770178638 podStartE2EDuration="2.770178638s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:18:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:18:04.767021618 +0000 UTC m=+1665.327831642" watchObservedRunningTime="2026-10-30 22:18:04.770178638 +0000 UTC m=+1665.330988643"
Oct 30 22:18:07.333975 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:18:07.333942    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-ac5024ad-6685bb6d9f-prktq"
Oct 30 22:18:07.335215 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:18:07.335194    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-ac5024ad-6bfdb47b7-fxbsd"
Oct 30 22:18:08.421639 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:18:08.421605    5510 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6gjbf" containerName="extractor"
Oct 30 22:18:08.785330 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:18:08.785291    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:09.494935 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:18:09.494894    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:13.413457 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:18:13.413317    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:15.654284 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:18:15.654255    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-derive-bee2db53-78cf85d8ff-w2jnq"
Oct 30 22:18:18.023677 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:18:18.023237    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:22.726035 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:18:22.725949    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:27.483686 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:18:27.483227    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:27.928378 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:18:27.928343    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:31.719328 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:18:31.719289    5510 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-ac5024ad-6dcc474c55-dh4j9"]
Oct 30 22:18:31.724240 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:18:31.724217    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ac5024ad-6dcc474c55-dh4j9"
Oct 30 22:18:31.779207 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:18:31.779172    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-ac5024ad-serving-cert\""
Oct 30 22:18:31.824709 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:18:31.824669    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-ac5024ad-6dcc474c55-dh4j9"]
Oct 30 22:18:31.888152 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:18:31.888098    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e58f98c-97c4-4bfa-b934-56414dfd53c7-openshift-service-ca\") pod \"maas-api-e2e-ait-ac5024ad-6dcc474c55-dh4j9\" (UID: \"4e58f98c-97c4-4bfa-b934-56414dfd53c7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ac5024ad-6dcc474c55-dh4j9"
Oct 30 22:18:31.888358 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:18:31.888174    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4e58f98c-97c4-4bfa-b934-56414dfd53c7-metrics-certs\") pod \"maas-api-e2e-ait-ac5024ad-6dcc474c55-dh4j9\" (UID: \"4e58f98c-97c4-4bfa-b934-56414dfd53c7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ac5024ad-6dcc474c55-dh4j9"
Oct 30 22:18:31.888453 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:18:31.888409    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nc2kh\" (UniqueName: \"kubernetes.io/projected/4e58f98c-97c4-4bfa-b934-56414dfd53c7-kube-api-access-nc2kh\") pod \"maas-api-e2e-ait-ac5024ad-6dcc474c55-dh4j9\" (UID: \"4e58f98c-97c4-4bfa-b934-56414dfd53c7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ac5024ad-6dcc474c55-dh4j9"
Oct 30 22:18:31.888719 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:18:31.888694    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4e58f98c-97c4-4bfa-b934-56414dfd53c7-maas-api-tls\") pod \"maas-api-e2e-ait-ac5024ad-6dcc474c55-dh4j9\" (UID: \"4e58f98c-97c4-4bfa-b934-56414dfd53c7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ac5024ad-6dcc474c55-dh4j9"
Oct 30 22:18:32.015732 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:18:31.997550    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4e58f98c-97c4-4bfa-b934-56414dfd53c7-maas-api-tls\") pod \"maas-api-e2e-ait-ac5024ad-6dcc474c55-dh4j9\" (UID: \"4e58f98c-97c4-4bfa-b934-56414dfd53c7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ac5024ad-6dcc474c55-dh4j9"
Oct 30 22:18:32.031942 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:18:32.031906    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e58f98c-97c4-4bfa-b934-56414dfd53c7-openshift-service-ca\") pod \"maas-api-e2e-ait-ac5024ad-6dcc474c55-dh4j9\" (UID: \"4e58f98c-97c4-4bfa-b934-56414dfd53c7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ac5024ad-6dcc474c55-dh4j9"
Oct 30 22:18:32.032212 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:18:32.032193    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4e58f98c-97c4-4bfa-b934-56414dfd53c7-metrics-certs\") pod \"maas-api-e2e-ait-ac5024ad-6dcc474c55-dh4j9\" (UID: \"4e58f98c-97c4-4bfa-b934-56414dfd53c7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ac5024ad-6dcc474c55-dh4j9"
Oct 30 22:18:32.038678 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:18:32.032488    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nc2kh\" (UniqueName: \"kubernetes.io/projected/4e58f98c-97c4-4bfa-b934-56414dfd53c7-kube-api-access-nc2kh\") pod \"maas-api-e2e-ait-ac5024ad-6dcc474c55-dh4j9\" (UID: \"4e58f98c-97c4-4bfa-b934-56414dfd53c7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ac5024ad-6dcc474c55-dh4j9"
Oct 30 22:18:32.042729 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:18:32.035467    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e58f98c-97c4-4bfa-b934-56414dfd53c7-openshift-service-ca\") pod \"maas-api-e2e-ait-ac5024ad-6dcc474c55-dh4j9\" (UID: \"4e58f98c-97c4-4bfa-b934-56414dfd53c7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ac5024ad-6dcc474c55-dh4j9"
Oct 30 22:18:32.042729 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:18:32.042213    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4e58f98c-97c4-4bfa-b934-56414dfd53c7-maas-api-tls\") pod \"maas-api-e2e-ait-ac5024ad-6dcc474c55-dh4j9\" (UID: \"4e58f98c-97c4-4bfa-b934-56414dfd53c7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ac5024ad-6dcc474c55-dh4j9"
Oct 30 22:18:32.042729 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:18:32.042552    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4e58f98c-97c4-4bfa-b934-56414dfd53c7-metrics-certs\") pod \"maas-api-e2e-ait-ac5024ad-6dcc474c55-dh4j9\" (UID: \"4e58f98c-97c4-4bfa-b934-56414dfd53c7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ac5024ad-6dcc474c55-dh4j9"
Oct 30 22:18:32.049939 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:18:32.049911    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:32.119483 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:18:32.119441    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nc2kh\" (UniqueName: \"kubernetes.io/projected/4e58f98c-97c4-4bfa-b934-56414dfd53c7-kube-api-access-nc2kh\") pod \"maas-api-e2e-ait-ac5024ad-6dcc474c55-dh4j9\" (UID: \"4e58f98c-97c4-4bfa-b934-56414dfd53c7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ac5024ad-6dcc474c55-dh4j9"
Oct 30 22:18:32.338925 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:18:32.338893    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ac5024ad-6dcc474c55-dh4j9"
Oct 30 22:18:32.425497 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:18:32.419573    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:32.724928 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:18:32.724896    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-ac5024ad-6dcc474c55-dh4j9"]
Oct 30 22:18:32.726374 ip-10-0-136-210 kubenswrapper[5510]: W1030 22:18:32.726352    5510 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e58f98c_97c4_4bfa_b934_56414dfd53c7.slice/crio-dd6548dabee491d368d1b7ec7f598869d65b1c5db5a64f9547a4950c7fd9cd3a WatchSource:0}: Error finding container dd6548dabee491d368d1b7ec7f598869d65b1c5db5a64f9547a4950c7fd9cd3a: Status 404 returned error can't find the container with id dd6548dabee491d368d1b7ec7f598869d65b1c5db5a64f9547a4950c7fd9cd3a
Oct 30 22:18:32.852971 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:18:32.852933    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ac5024ad-6dcc474c55-dh4j9" event={"ID":"4e58f98c-97c4-4bfa-b934-56414dfd53c7","Type":"ContainerStarted","Data":"dd6548dabee491d368d1b7ec7f598869d65b1c5db5a64f9547a4950c7fd9cd3a"}
Oct 30 22:18:33.862173 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:18:33.862081    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ac5024ad-6dcc474c55-dh4j9" event={"ID":"4e58f98c-97c4-4bfa-b934-56414dfd53c7","Type":"ContainerStarted","Data":"b5efb250cbd4d90d3459ae1661b81d10407931f0de37454fe8b43d0c5abe6a72"}
Oct 30 22:18:33.862560 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:18:33.862206    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ac5024ad-6dcc474c55-dh4j9"
Oct 30 22:18:34.005186 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:18:34.005130    5510 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ac5024ad-6dcc474c55-dh4j9" podStartSLOduration=2.16131898 podStartE2EDuration="3.005113385s" totalImagesPullingTime="843.794405ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:18:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:18:34.00418599 +0000 UTC m=+1694.564995995" watchObservedRunningTime="2026-10-30 22:18:34.005113385 +0000 UTC m=+1694.565923390"
Oct 30 22:18:36.857460 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:18:36.857421    5510 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-c42c0a-8545457dcf-ghrwb"]
Oct 30 22:18:36.862492 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:18:36.862469    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-c42c0a-8545457dcf-ghrwb"
Oct 30 22:18:36.919357 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:18:36.919312    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w0-c42c0a-serving-cert\""
Oct 30 22:18:36.948294 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:18:36.948258    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/42db3ad0-7d90-4073-9ad9-19b6b5590551-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w0-c42c0a-8545457dcf-ghrwb\" (UID: \"42db3ad0-7d90-4073-9ad9-19b6b5590551\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-c42c0a-8545457dcf-ghrwb"
Oct 30 22:18:36.948488 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:18:36.948343    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bqsfn\" (UniqueName: \"kubernetes.io/projected/42db3ad0-7d90-4073-9ad9-19b6b5590551-kube-api-access-bqsfn\") pod \"maas-api-e2e-shared-a-w0-c42c0a-8545457dcf-ghrwb\" (UID: \"42db3ad0-7d90-4073-9ad9-19b6b5590551\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-c42c0a-8545457dcf-ghrwb"
Oct 30 22:18:36.948592 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:18:36.948565    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/42db3ad0-7d90-4073-9ad9-19b6b5590551-maas-api-tls\") pod \"maas-api-e2e-shared-a-w0-c42c0a-8545457dcf-ghrwb\" (UID: \"42db3ad0-7d90-4073-9ad9-19b6b5590551\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-c42c0a-8545457dcf-ghrwb"
Oct 30 22:18:36.948647 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:18:36.948633    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/42db3ad0-7d90-4073-9ad9-19b6b5590551-metrics-certs\") pod \"maas-api-e2e-shared-a-w0-c42c0a-8545457dcf-ghrwb\" (UID: \"42db3ad0-7d90-4073-9ad9-19b6b5590551\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-c42c0a-8545457dcf-ghrwb"
Oct 30 22:18:36.972317 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:18:36.972281    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-c42c0a-8545457dcf-ghrwb"]
Oct 30 22:18:37.069159 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:18:37.057000    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/42db3ad0-7d90-4073-9ad9-19b6b5590551-maas-api-tls\") pod \"maas-api-e2e-shared-a-w0-c42c0a-8545457dcf-ghrwb\" (UID: \"42db3ad0-7d90-4073-9ad9-19b6b5590551\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-c42c0a-8545457dcf-ghrwb"
Oct 30 22:18:37.069573 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:18:37.063430    5510 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-c42c0a-serving-cert: secret "maas-api-e2e-shared-a-w0-c42c0a-serving-cert" not found
Oct 30 22:18:37.069789 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:18:37.069773    5510 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/42db3ad0-7d90-4073-9ad9-19b6b5590551-maas-api-tls podName:42db3ad0-7d90-4073-9ad9-19b6b5590551 nodeName:}" failed. No retries permitted until 2026-10-30 22:18:37.569742472 +0000 UTC m=+1698.130552458 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/42db3ad0-7d90-4073-9ad9-19b6b5590551-maas-api-tls") pod "maas-api-e2e-shared-a-w0-c42c0a-8545457dcf-ghrwb" (UID: "42db3ad0-7d90-4073-9ad9-19b6b5590551") : secret "maas-api-e2e-shared-a-w0-c42c0a-serving-cert" not found
Oct 30 22:18:37.070197 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:18:37.070173    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/42db3ad0-7d90-4073-9ad9-19b6b5590551-metrics-certs\") pod \"maas-api-e2e-shared-a-w0-c42c0a-8545457dcf-ghrwb\" (UID: \"42db3ad0-7d90-4073-9ad9-19b6b5590551\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-c42c0a-8545457dcf-ghrwb"
Oct 30 22:18:37.070588 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:18:37.070566    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/42db3ad0-7d90-4073-9ad9-19b6b5590551-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w0-c42c0a-8545457dcf-ghrwb\" (UID: \"42db3ad0-7d90-4073-9ad9-19b6b5590551\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-c42c0a-8545457dcf-ghrwb"
Oct 30 22:18:37.092678 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:18:37.070776    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bqsfn\" (UniqueName: \"kubernetes.io/projected/42db3ad0-7d90-4073-9ad9-19b6b5590551-kube-api-access-bqsfn\") pod \"maas-api-e2e-shared-a-w0-c42c0a-8545457dcf-ghrwb\" (UID: \"42db3ad0-7d90-4073-9ad9-19b6b5590551\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-c42c0a-8545457dcf-ghrwb"
Oct 30 22:18:37.093284 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:18:37.071705    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/42db3ad0-7d90-4073-9ad9-19b6b5590551-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w0-c42c0a-8545457dcf-ghrwb\" (UID: \"42db3ad0-7d90-4073-9ad9-19b6b5590551\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-c42c0a-8545457dcf-ghrwb"
Oct 30 22:18:37.093473 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:18:37.084645    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/42db3ad0-7d90-4073-9ad9-19b6b5590551-metrics-certs\") pod \"maas-api-e2e-shared-a-w0-c42c0a-8545457dcf-ghrwb\" (UID: \"42db3ad0-7d90-4073-9ad9-19b6b5590551\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-c42c0a-8545457dcf-ghrwb"
Oct 30 22:18:37.121754 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:18:37.121646    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:37.161733 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:18:37.157716    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bqsfn\" (UniqueName: \"kubernetes.io/projected/42db3ad0-7d90-4073-9ad9-19b6b5590551-kube-api-access-bqsfn\") pod \"maas-api-e2e-shared-a-w0-c42c0a-8545457dcf-ghrwb\" (UID: \"42db3ad0-7d90-4073-9ad9-19b6b5590551\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-c42c0a-8545457dcf-ghrwb"
Oct 30 22:18:37.504049 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:18:37.503953    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:37.603610 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:18:37.603577    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/42db3ad0-7d90-4073-9ad9-19b6b5590551-maas-api-tls\") pod \"maas-api-e2e-shared-a-w0-c42c0a-8545457dcf-ghrwb\" (UID: \"42db3ad0-7d90-4073-9ad9-19b6b5590551\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-c42c0a-8545457dcf-ghrwb"
Oct 30 22:18:37.606868 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:18:37.606424    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/42db3ad0-7d90-4073-9ad9-19b6b5590551-maas-api-tls\") pod \"maas-api-e2e-shared-a-w0-c42c0a-8545457dcf-ghrwb\" (UID: \"42db3ad0-7d90-4073-9ad9-19b6b5590551\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-c42c0a-8545457dcf-ghrwb"
Oct 30 22:18:37.778196 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:18:37.778163    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-c42c0a-8545457dcf-ghrwb"
Oct 30 22:18:38.168886 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:18:38.168848    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-c42c0a-8545457dcf-ghrwb"]
Oct 30 22:18:38.169312 ip-10-0-136-210 kubenswrapper[5510]: W1030 22:18:38.169189    5510 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42db3ad0_7d90_4073_9ad9_19b6b5590551.slice/crio-66bb191d3d5035ced3affb6aaacb96d2dd24840adfe561a620eff06240ff4d06 WatchSource:0}: Error finding container 66bb191d3d5035ced3affb6aaacb96d2dd24840adfe561a620eff06240ff4d06: Status 404 returned error can't find the container with id 66bb191d3d5035ced3affb6aaacb96d2dd24840adfe561a620eff06240ff4d06
Oct 30 22:18:38.422925 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:18:38.422838    5510 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6gjbf" containerName="extractor"
Oct 30 22:18:38.925975 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:18:38.925934    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-c42c0a-8545457dcf-ghrwb" event={"ID":"42db3ad0-7d90-4073-9ad9-19b6b5590551","Type":"ContainerStarted","Data":"66bb191d3d5035ced3affb6aaacb96d2dd24840adfe561a620eff06240ff4d06"}
Oct 30 22:18:39.962867 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:18:39.962831    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ac5024ad-6dcc474c55-dh4j9"
Oct 30 22:18:39.993094 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:18:39.991093    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-c42c0a-8545457dcf-ghrwb" event={"ID":"42db3ad0-7d90-4073-9ad9-19b6b5590551","Type":"ContainerStarted","Data":"20947ec152a579120ffc74a77d841e88e7990da4c29221f4bfe0994ac2ef78bf"}
Oct 30 22:18:39.993094 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:18:39.992180    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-c42c0a-8545457dcf-ghrwb"
Oct 30 22:18:40.193187 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:18:40.193129    5510 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-c42c0a-8545457dcf-ghrwb" podStartSLOduration=3.2754439140000002 podStartE2EDuration="4.193107873s" totalImagesPullingTime="917.663959ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:18:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:18:40.19217153 +0000 UTC m=+1700.752981537" watchObservedRunningTime="2026-10-30 22:18:40.193107873 +0000 UTC m=+1700.753917879"
Oct 30 22:18:42.128932 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:18:42.128894    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:46.527795 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:18:46.527760    5510 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-bee2db53-7ffd6dbfcb-s7jf4"]
Oct 30 22:18:46.532547 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:18:46.532525    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-bee2db53-7ffd6dbfcb-s7jf4"
Oct 30 22:18:46.583561 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:18:46.583529    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-bee2db53-serving-cert\""
Oct 30 22:18:46.610627 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:18:46.610591    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9678c16a-f6f3-4055-af08-f3d0adaab85f-metrics-certs\") pod \"maas-api-e2e-derive-bee2db53-7ffd6dbfcb-s7jf4\" (UID: \"9678c16a-f6f3-4055-af08-f3d0adaab85f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-bee2db53-7ffd6dbfcb-s7jf4"
Oct 30 22:18:46.610848 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:18:46.610825    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9678c16a-f6f3-4055-af08-f3d0adaab85f-openshift-service-ca\") pod \"maas-api-e2e-derive-bee2db53-7ffd6dbfcb-s7jf4\" (UID: \"9678c16a-f6f3-4055-af08-f3d0adaab85f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-bee2db53-7ffd6dbfcb-s7jf4"
Oct 30 22:18:46.611064 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:18:46.611042    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9678c16a-f6f3-4055-af08-f3d0adaab85f-maas-api-tls\") pod \"maas-api-e2e-derive-bee2db53-7ffd6dbfcb-s7jf4\" (UID: \"9678c16a-f6f3-4055-af08-f3d0adaab85f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-bee2db53-7ffd6dbfcb-s7jf4"
Oct 30 22:18:46.611273 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:18:46.611251    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p6mbm\" (UniqueName: \"kubernetes.io/projected/9678c16a-f6f3-4055-af08-f3d0adaab85f-kube-api-access-p6mbm\") pod \"maas-api-e2e-derive-bee2db53-7ffd6dbfcb-s7jf4\" (UID: \"9678c16a-f6f3-4055-af08-f3d0adaab85f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-bee2db53-7ffd6dbfcb-s7jf4"
Oct 30 22:18:46.636959 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:18:46.636924    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-bee2db53-7ffd6dbfcb-s7jf4"]
Oct 30 22:18:46.713278 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:18:46.713245    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9678c16a-f6f3-4055-af08-f3d0adaab85f-openshift-service-ca\") pod \"maas-api-e2e-derive-bee2db53-7ffd6dbfcb-s7jf4\" (UID: \"9678c16a-f6f3-4055-af08-f3d0adaab85f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-bee2db53-7ffd6dbfcb-s7jf4"
Oct 30 22:18:46.713442 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:18:46.713385    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9678c16a-f6f3-4055-af08-f3d0adaab85f-maas-api-tls\") pod \"maas-api-e2e-derive-bee2db53-7ffd6dbfcb-s7jf4\" (UID: \"9678c16a-f6f3-4055-af08-f3d0adaab85f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-bee2db53-7ffd6dbfcb-s7jf4"
Oct 30 22:18:46.713492 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:18:46.713450    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p6mbm\" (UniqueName: \"kubernetes.io/projected/9678c16a-f6f3-4055-af08-f3d0adaab85f-kube-api-access-p6mbm\") pod \"maas-api-e2e-derive-bee2db53-7ffd6dbfcb-s7jf4\" (UID: \"9678c16a-f6f3-4055-af08-f3d0adaab85f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-bee2db53-7ffd6dbfcb-s7jf4"
Oct 30 22:18:46.713538 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:18:46.713511    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9678c16a-f6f3-4055-af08-f3d0adaab85f-metrics-certs\") pod \"maas-api-e2e-derive-bee2db53-7ffd6dbfcb-s7jf4\" (UID: \"9678c16a-f6f3-4055-af08-f3d0adaab85f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-bee2db53-7ffd6dbfcb-s7jf4"
Oct 30 22:18:46.713599 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:18:46.713530    5510 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-bee2db53-serving-cert: secret "maas-api-e2e-derive-bee2db53-serving-cert" not found
Oct 30 22:18:46.713599 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:18:46.713591    5510 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9678c16a-f6f3-4055-af08-f3d0adaab85f-maas-api-tls podName:9678c16a-f6f3-4055-af08-f3d0adaab85f nodeName:}" failed. No retries permitted until 2026-10-30 22:18:47.213574557 +0000 UTC m=+1707.774384548 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/9678c16a-f6f3-4055-af08-f3d0adaab85f-maas-api-tls") pod "maas-api-e2e-derive-bee2db53-7ffd6dbfcb-s7jf4" (UID: "9678c16a-f6f3-4055-af08-f3d0adaab85f") : secret "maas-api-e2e-derive-bee2db53-serving-cert" not found
Oct 30 22:18:46.713984 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:18:46.713966    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9678c16a-f6f3-4055-af08-f3d0adaab85f-openshift-service-ca\") pod \"maas-api-e2e-derive-bee2db53-7ffd6dbfcb-s7jf4\" (UID: \"9678c16a-f6f3-4055-af08-f3d0adaab85f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-bee2db53-7ffd6dbfcb-s7jf4"
Oct 30 22:18:46.715950 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:18:46.715929    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9678c16a-f6f3-4055-af08-f3d0adaab85f-metrics-certs\") pod \"maas-api-e2e-derive-bee2db53-7ffd6dbfcb-s7jf4\" (UID: \"9678c16a-f6f3-4055-af08-f3d0adaab85f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-bee2db53-7ffd6dbfcb-s7jf4"
Oct 30 22:18:46.845742 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:18:46.843975    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p6mbm\" (UniqueName: \"kubernetes.io/projected/9678c16a-f6f3-4055-af08-f3d0adaab85f-kube-api-access-p6mbm\") pod \"maas-api-e2e-derive-bee2db53-7ffd6dbfcb-s7jf4\" (UID: \"9678c16a-f6f3-4055-af08-f3d0adaab85f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-bee2db53-7ffd6dbfcb-s7jf4"
Oct 30 22:18:46.925678 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:18:46.925109    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:47.039936 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:18:47.039888    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-c42c0a-8545457dcf-ghrwb"
Oct 30 22:18:47.215955 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:18:47.215858    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:47.238061 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:18:47.238016    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9678c16a-f6f3-4055-af08-f3d0adaab85f-maas-api-tls\") pod \"maas-api-e2e-derive-bee2db53-7ffd6dbfcb-s7jf4\" (UID: \"9678c16a-f6f3-4055-af08-f3d0adaab85f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-bee2db53-7ffd6dbfcb-s7jf4"
Oct 30 22:18:47.240572 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:18:47.240548    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9678c16a-f6f3-4055-af08-f3d0adaab85f-maas-api-tls\") pod \"maas-api-e2e-derive-bee2db53-7ffd6dbfcb-s7jf4\" (UID: \"9678c16a-f6f3-4055-af08-f3d0adaab85f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-bee2db53-7ffd6dbfcb-s7jf4"
Oct 30 22:18:47.446974 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:18:47.446946    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-bee2db53-7ffd6dbfcb-s7jf4"
Oct 30 22:18:47.829892 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:18:47.829857    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-bee2db53-7ffd6dbfcb-s7jf4"]
Oct 30 22:18:47.830269 ip-10-0-136-210 kubenswrapper[5510]: W1030 22:18:47.830108    5510 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9678c16a_f6f3_4055_af08_f3d0adaab85f.slice/crio-a19176bf510f9f98a5ce90d8e9ef81717831e6a8e2f25e87e766afaf087ee2ef WatchSource:0}: Error finding container a19176bf510f9f98a5ce90d8e9ef81717831e6a8e2f25e87e766afaf087ee2ef: Status 404 returned error can't find the container with id a19176bf510f9f98a5ce90d8e9ef81717831e6a8e2f25e87e766afaf087ee2ef
Oct 30 22:18:48.099786 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:18:48.099704    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-bee2db53-7ffd6dbfcb-s7jf4" event={"ID":"9678c16a-f6f3-4055-af08-f3d0adaab85f","Type":"ContainerStarted","Data":"a19176bf510f9f98a5ce90d8e9ef81717831e6a8e2f25e87e766afaf087ee2ef"}
Oct 30 22:18:50.282987 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:18:50.282941    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-bee2db53-7ffd6dbfcb-s7jf4" event={"ID":"9678c16a-f6f3-4055-af08-f3d0adaab85f","Type":"ContainerStarted","Data":"f9038a0a29975a8ce832dc81d33872d5fe7c68a69c0e4fe2e2158ca83aea36e7"}
Oct 30 22:18:50.283590 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:18:50.283568    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-bee2db53-7ffd6dbfcb-s7jf4"
Oct 30 22:18:50.397257 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:18:50.397192    5510 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-bee2db53-7ffd6dbfcb-s7jf4" podStartSLOduration=3.18521019 podStartE2EDuration="4.397175002s" totalImagesPullingTime="1.211964812s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:18:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:18:50.396338333 +0000 UTC m=+1710.957148397" watchObservedRunningTime="2026-10-30 22:18:50.397175002 +0000 UTC m=+1710.957985008"
Oct 30 22:18:52.320647 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:18:52.320597    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:56.310398 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:18:56.310361    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-bee2db53-7ffd6dbfcb-s7jf4"
Oct 30 22:18:57.406383 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:18:57.404902    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:02.128727 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:19:02.128507    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:06.755628 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:19:06.755559    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:08.421921 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:19:08.421875    5510 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6gjbf" containerName="extractor"
Oct 30 22:19:11.637681 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:19:11.637568    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:16.153393 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:19:16.153353    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:20.939514 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:19:20.939475    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:25.517364 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:19:25.517323    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:28.365789 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:19:28.365746    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:29.625782 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:19:29.625740    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:30.526180 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:19:30.526137    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:31.782601 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:19:31.782563    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:32.445421 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:19:32.445384    5510 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-4da07739-openshift-default-6bd77c67d9-7wknn"]
Oct 30 22:19:32.634694 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:19:32.615323    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-4da07739-openshift-default-6bd77c67d9-7wknn"]
Oct 30 22:19:32.634694 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:19:32.615462    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-4da07739-openshift-default-6bd77c67d9-7wknn"
Oct 30 22:19:32.670703 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:19:32.654144    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:32.704025 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:19:32.703932    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-4da07739-openshift-default-dockercfg-8vsnh\""
Oct 30 22:19:32.755681 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:19:32.743523    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f4d4ae01-a1b3-41ea-9e54-88edbe45642c-istio-envoy\") pod \"e2e-mt-4da07739-openshift-default-6bd77c67d9-7wknn\" (UID: \"f4d4ae01-a1b3-41ea-9e54-88edbe45642c\") " pod="openshift-ingress/e2e-mt-4da07739-openshift-default-6bd77c67d9-7wknn"
Oct 30 22:19:32.778464 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:19:32.778420    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f4d4ae01-a1b3-41ea-9e54-88edbe45642c-istio-data\") pod \"e2e-mt-4da07739-openshift-default-6bd77c67d9-7wknn\" (UID: \"f4d4ae01-a1b3-41ea-9e54-88edbe45642c\") " pod="openshift-ingress/e2e-mt-4da07739-openshift-default-6bd77c67d9-7wknn"
Oct 30 22:19:32.778814 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:19:32.778782    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4d4ae01-a1b3-41ea-9e54-88edbe45642c-credential-socket\") pod \"e2e-mt-4da07739-openshift-default-6bd77c67d9-7wknn\" (UID: \"f4d4ae01-a1b3-41ea-9e54-88edbe45642c\") " pod="openshift-ingress/e2e-mt-4da07739-openshift-default-6bd77c67d9-7wknn"
Oct 30 22:19:32.778999 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:19:32.778980    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f4d4ae01-a1b3-41ea-9e54-88edbe45642c-istiod-ca-cert\") pod \"e2e-mt-4da07739-openshift-default-6bd77c67d9-7wknn\" (UID: \"f4d4ae01-a1b3-41ea-9e54-88edbe45642c\") " pod="openshift-ingress/e2e-mt-4da07739-openshift-default-6bd77c67d9-7wknn"
Oct 30 22:19:32.779194 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:19:32.779175    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f4d4ae01-a1b3-41ea-9e54-88edbe45642c-istio-token\") pod \"e2e-mt-4da07739-openshift-default-6bd77c67d9-7wknn\" (UID: \"f4d4ae01-a1b3-41ea-9e54-88edbe45642c\") " pod="openshift-ingress/e2e-mt-4da07739-openshift-default-6bd77c67d9-7wknn"
Oct 30 22:19:32.779362 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:19:32.779345    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f4d4ae01-a1b3-41ea-9e54-88edbe45642c-istio-podinfo\") pod \"e2e-mt-4da07739-openshift-default-6bd77c67d9-7wknn\" (UID: \"f4d4ae01-a1b3-41ea-9e54-88edbe45642c\") " pod="openshift-ingress/e2e-mt-4da07739-openshift-default-6bd77c67d9-7wknn"
Oct 30 22:19:32.779590 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:19:32.779574    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f4d4ae01-a1b3-41ea-9e54-88edbe45642c-workload-certs\") pod \"e2e-mt-4da07739-openshift-default-6bd77c67d9-7wknn\" (UID: \"f4d4ae01-a1b3-41ea-9e54-88edbe45642c\") " pod="openshift-ingress/e2e-mt-4da07739-openshift-default-6bd77c67d9-7wknn"
Oct 30 22:19:32.779792 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:19:32.779770    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w5stm\" (UniqueName: \"kubernetes.io/projected/f4d4ae01-a1b3-41ea-9e54-88edbe45642c-kube-api-access-w5stm\") pod \"e2e-mt-4da07739-openshift-default-6bd77c67d9-7wknn\" (UID: \"f4d4ae01-a1b3-41ea-9e54-88edbe45642c\") " pod="openshift-ingress/e2e-mt-4da07739-openshift-default-6bd77c67d9-7wknn"
Oct 30 22:19:32.790683 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:19:32.779914    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4d4ae01-a1b3-41ea-9e54-88edbe45642c-workload-socket\") pod \"e2e-mt-4da07739-openshift-default-6bd77c67d9-7wknn\" (UID: \"f4d4ae01-a1b3-41ea-9e54-88edbe45642c\") " pod="openshift-ingress/e2e-mt-4da07739-openshift-default-6bd77c67d9-7wknn"
Oct 30 22:19:32.942685 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:19:32.938897    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f4d4ae01-a1b3-41ea-9e54-88edbe45642c-istio-podinfo\") pod \"e2e-mt-4da07739-openshift-default-6bd77c67d9-7wknn\" (UID: \"f4d4ae01-a1b3-41ea-9e54-88edbe45642c\") " pod="openshift-ingress/e2e-mt-4da07739-openshift-default-6bd77c67d9-7wknn"
Oct 30 22:19:32.942685 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:19:32.939035    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f4d4ae01-a1b3-41ea-9e54-88edbe45642c-workload-certs\") pod \"e2e-mt-4da07739-openshift-default-6bd77c67d9-7wknn\" (UID: \"f4d4ae01-a1b3-41ea-9e54-88edbe45642c\") " pod="openshift-ingress/e2e-mt-4da07739-openshift-default-6bd77c67d9-7wknn"
Oct 30 22:19:32.942685 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:19:32.939111    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w5stm\" (UniqueName: \"kubernetes.io/projected/f4d4ae01-a1b3-41ea-9e54-88edbe45642c-kube-api-access-w5stm\") pod \"e2e-mt-4da07739-openshift-default-6bd77c67d9-7wknn\" (UID: \"f4d4ae01-a1b3-41ea-9e54-88edbe45642c\") " pod="openshift-ingress/e2e-mt-4da07739-openshift-default-6bd77c67d9-7wknn"
Oct 30 22:19:32.942685 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:19:32.939142    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4d4ae01-a1b3-41ea-9e54-88edbe45642c-workload-socket\") pod \"e2e-mt-4da07739-openshift-default-6bd77c67d9-7wknn\" (UID: \"f4d4ae01-a1b3-41ea-9e54-88edbe45642c\") " pod="openshift-ingress/e2e-mt-4da07739-openshift-default-6bd77c67d9-7wknn"
Oct 30 22:19:32.942685 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:19:32.939409    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f4d4ae01-a1b3-41ea-9e54-88edbe45642c-istio-envoy\") pod \"e2e-mt-4da07739-openshift-default-6bd77c67d9-7wknn\" (UID: \"f4d4ae01-a1b3-41ea-9e54-88edbe45642c\") " pod="openshift-ingress/e2e-mt-4da07739-openshift-default-6bd77c67d9-7wknn"
Oct 30 22:19:32.942685 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:19:32.939457    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f4d4ae01-a1b3-41ea-9e54-88edbe45642c-istio-data\") pod \"e2e-mt-4da07739-openshift-default-6bd77c67d9-7wknn\" (UID: \"f4d4ae01-a1b3-41ea-9e54-88edbe45642c\") " pod="openshift-ingress/e2e-mt-4da07739-openshift-default-6bd77c67d9-7wknn"
Oct 30 22:19:32.942685 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:19:32.939501    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4d4ae01-a1b3-41ea-9e54-88edbe45642c-credential-socket\") pod \"e2e-mt-4da07739-openshift-default-6bd77c67d9-7wknn\" (UID: \"f4d4ae01-a1b3-41ea-9e54-88edbe45642c\") " pod="openshift-ingress/e2e-mt-4da07739-openshift-default-6bd77c67d9-7wknn"
Oct 30 22:19:32.942685 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:19:32.939526    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f4d4ae01-a1b3-41ea-9e54-88edbe45642c-istiod-ca-cert\") pod \"e2e-mt-4da07739-openshift-default-6bd77c67d9-7wknn\" (UID: \"f4d4ae01-a1b3-41ea-9e54-88edbe45642c\") " pod="openshift-ingress/e2e-mt-4da07739-openshift-default-6bd77c67d9-7wknn"
Oct 30 22:19:32.942685 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:19:32.939596    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f4d4ae01-a1b3-41ea-9e54-88edbe45642c-istio-token\") pod \"e2e-mt-4da07739-openshift-default-6bd77c67d9-7wknn\" (UID: \"f4d4ae01-a1b3-41ea-9e54-88edbe45642c\") " pod="openshift-ingress/e2e-mt-4da07739-openshift-default-6bd77c67d9-7wknn"
Oct 30 22:19:32.942685 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:19:32.942173    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4d4ae01-a1b3-41ea-9e54-88edbe45642c-credential-socket\") pod \"e2e-mt-4da07739-openshift-default-6bd77c67d9-7wknn\" (UID: \"f4d4ae01-a1b3-41ea-9e54-88edbe45642c\") " pod="openshift-ingress/e2e-mt-4da07739-openshift-default-6bd77c67d9-7wknn"
Oct 30 22:19:32.942685 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:19:32.942422    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f4d4ae01-a1b3-41ea-9e54-88edbe45642c-istio-data\") pod \"e2e-mt-4da07739-openshift-default-6bd77c67d9-7wknn\" (UID: \"f4d4ae01-a1b3-41ea-9e54-88edbe45642c\") " pod="openshift-ingress/e2e-mt-4da07739-openshift-default-6bd77c67d9-7wknn"
Oct 30 22:19:32.953775 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:19:32.953061    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f4d4ae01-a1b3-41ea-9e54-88edbe45642c-workload-certs\") pod \"e2e-mt-4da07739-openshift-default-6bd77c67d9-7wknn\" (UID: \"f4d4ae01-a1b3-41ea-9e54-88edbe45642c\") " pod="openshift-ingress/e2e-mt-4da07739-openshift-default-6bd77c67d9-7wknn"
Oct 30 22:19:32.953775 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:19:32.953323    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4d4ae01-a1b3-41ea-9e54-88edbe45642c-workload-socket\") pod \"e2e-mt-4da07739-openshift-default-6bd77c67d9-7wknn\" (UID: \"f4d4ae01-a1b3-41ea-9e54-88edbe45642c\") " pod="openshift-ingress/e2e-mt-4da07739-openshift-default-6bd77c67d9-7wknn"
Oct 30 22:19:32.958994 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:19:32.953973    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f4d4ae01-a1b3-41ea-9e54-88edbe45642c-istiod-ca-cert\") pod \"e2e-mt-4da07739-openshift-default-6bd77c67d9-7wknn\" (UID: \"f4d4ae01-a1b3-41ea-9e54-88edbe45642c\") " pod="openshift-ingress/e2e-mt-4da07739-openshift-default-6bd77c67d9-7wknn"
Oct 30 22:19:32.958994 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:19:32.954585    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f4d4ae01-a1b3-41ea-9e54-88edbe45642c-istio-podinfo\") pod \"e2e-mt-4da07739-openshift-default-6bd77c67d9-7wknn\" (UID: \"f4d4ae01-a1b3-41ea-9e54-88edbe45642c\") " pod="openshift-ingress/e2e-mt-4da07739-openshift-default-6bd77c67d9-7wknn"
Oct 30 22:19:32.961123 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:19:32.960078    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f4d4ae01-a1b3-41ea-9e54-88edbe45642c-istio-envoy\") pod \"e2e-mt-4da07739-openshift-default-6bd77c67d9-7wknn\" (UID: \"f4d4ae01-a1b3-41ea-9e54-88edbe45642c\") " pod="openshift-ingress/e2e-mt-4da07739-openshift-default-6bd77c67d9-7wknn"
Oct 30 22:19:33.015491 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:19:33.015419    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f4d4ae01-a1b3-41ea-9e54-88edbe45642c-istio-token\") pod \"e2e-mt-4da07739-openshift-default-6bd77c67d9-7wknn\" (UID: \"f4d4ae01-a1b3-41ea-9e54-88edbe45642c\") " pod="openshift-ingress/e2e-mt-4da07739-openshift-default-6bd77c67d9-7wknn"
Oct 30 22:19:33.022194 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:19:33.022123    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w5stm\" (UniqueName: \"kubernetes.io/projected/f4d4ae01-a1b3-41ea-9e54-88edbe45642c-kube-api-access-w5stm\") pod \"e2e-mt-4da07739-openshift-default-6bd77c67d9-7wknn\" (UID: \"f4d4ae01-a1b3-41ea-9e54-88edbe45642c\") " pod="openshift-ingress/e2e-mt-4da07739-openshift-default-6bd77c67d9-7wknn"
Oct 30 22:19:33.104227 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:19:33.104152    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:33.252240 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:19:33.252152    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-4da07739-openshift-default-6bd77c67d9-7wknn"
Oct 30 22:19:33.684907 ip-10-0-136-210 kubenswrapper[5510]: W1030 22:19:33.684872    5510 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4d4ae01_a1b3_41ea_9e54_88edbe45642c.slice/crio-54e372f78382471142d59580ad3a6d62d5ac5d65df657ec9f9f6dcaa443d8f23 WatchSource:0}: Error finding container 54e372f78382471142d59580ad3a6d62d5ac5d65df657ec9f9f6dcaa443d8f23: Status 404 returned error can't find the container with id 54e372f78382471142d59580ad3a6d62d5ac5d65df657ec9f9f6dcaa443d8f23
Oct 30 22:19:33.688591 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:19:33.688541    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:19:33.688774 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:19:33.688593    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-4da07739-openshift-default-6bd77c67d9-7wknn"]
Oct 30 22:19:33.688774 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:19:33.688627    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:19:33.688774 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:19:33.688689    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:19:33.736326 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:19:33.736298    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-4da07739-openshift-default-6bd77c67d9-7wknn" event={"ID":"f4d4ae01-a1b3-41ea-9e54-88edbe45642c","Type":"ContainerStarted","Data":"54e372f78382471142d59580ad3a6d62d5ac5d65df657ec9f9f6dcaa443d8f23"}
Oct 30 22:19:34.775441 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:19:34.775403    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-4da07739-openshift-default-6bd77c67d9-7wknn" event={"ID":"f4d4ae01-a1b3-41ea-9e54-88edbe45642c","Type":"ContainerStarted","Data":"0266b6992b04d91261c6136773c8e9e08aa1dbfe4c76a25afc7fb6a7a7eb3f49"}
Oct 30 22:19:34.944036 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:19:34.943905    5510 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-4da07739-openshift-default-6bd77c67d9-7wknn" podStartSLOduration=2.9438820039999998 podStartE2EDuration="2.943882004s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:19:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:19:34.943171485 +0000 UTC m=+1755.503981535" watchObservedRunningTime="2026-10-30 22:19:34.943882004 +0000 UTC m=+1755.504692026"
Oct 30 22:19:35.253308 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:19:35.253222    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-4da07739-openshift-default-6bd77c67d9-7wknn"
Oct 30 22:19:35.259745 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:19:35.259712    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-4da07739-openshift-default-6bd77c67d9-7wknn"
Oct 30 22:19:35.707368 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:19:35.707327    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:35.788986 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:19:35.788956    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-4da07739-openshift-default-6bd77c67d9-7wknn"
Oct 30 22:19:35.791280 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:19:35.791254    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-4da07739-openshift-default-6bd77c67d9-7wknn"
Oct 30 22:19:37.990178 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:19:37.990137    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:38.308385 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:19:38.308350    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:38.588533 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:19:38.588455    5510 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6gjbf" containerName="extractor"
Oct 30 22:19:38.671755 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:19:38.671713    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:38.913540 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:19:38.913449    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:39.953682 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:19:39.943010    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:40.210730 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:19:40.210626    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:41.399549 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:19:41.399503    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:41.677948 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:19:41.677855    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:41.757507 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:19:41.757469    5510 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-4da07739-bf6645659-p8vvn"]
Oct 30 22:19:41.797016 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:19:41.796978    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-4da07739-bf6645659-p8vvn"
Oct 30 22:19:41.882698 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:19:41.841338    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-4da07739-bf6645659-p8vvn"]
Oct 30 22:19:41.892366 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:19:41.854980    5510 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-4da07739\""
Oct 30 22:19:41.897736 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:19:41.876206    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7btwt\" (UniqueName: \"kubernetes.io/projected/628be936-435a-4e97-9aaa-98924732fb01-kube-api-access-7btwt\") pod \"payload-pre-processing-e2e-mt-4da07739-bf6645659-p8vvn\" (UID: \"628be936-435a-4e97-9aaa-98924732fb01\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-4da07739-bf6645659-p8vvn"
Oct 30 22:19:41.898353 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:19:41.898326    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/628be936-435a-4e97-9aaa-98924732fb01-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-4da07739-bf6645659-p8vvn\" (UID: \"628be936-435a-4e97-9aaa-98924732fb01\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-4da07739-bf6645659-p8vvn"
Oct 30 22:19:41.904684 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:19:41.898959    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-4da07739-dockercfg-zczwl\""
Oct 30 22:19:42.218759 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:19:42.196687    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/628be936-435a-4e97-9aaa-98924732fb01-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-4da07739-bf6645659-p8vvn\" (UID: \"628be936-435a-4e97-9aaa-98924732fb01\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-4da07739-bf6645659-p8vvn"
Oct 30 22:19:42.221911 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:19:42.221875    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7btwt\" (UniqueName: \"kubernetes.io/projected/628be936-435a-4e97-9aaa-98924732fb01-kube-api-access-7btwt\") pod \"payload-pre-processing-e2e-mt-4da07739-bf6645659-p8vvn\" (UID: \"628be936-435a-4e97-9aaa-98924732fb01\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-4da07739-bf6645659-p8vvn"
Oct 30 22:19:42.235572 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:19:42.220187    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/628be936-435a-4e97-9aaa-98924732fb01-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-4da07739-bf6645659-p8vvn\" (UID: \"628be936-435a-4e97-9aaa-98924732fb01\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-4da07739-bf6645659-p8vvn"
Oct 30 22:19:42.620681 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:19:42.617722    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7btwt\" (UniqueName: \"kubernetes.io/projected/628be936-435a-4e97-9aaa-98924732fb01-kube-api-access-7btwt\") pod \"payload-pre-processing-e2e-mt-4da07739-bf6645659-p8vvn\" (UID: \"628be936-435a-4e97-9aaa-98924732fb01\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-4da07739-bf6645659-p8vvn"
Oct 30 22:19:42.910819 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:19:42.910720    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-4da07739-bf6645659-p8vvn"
Oct 30 22:19:43.414845 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:19:43.414801    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-4da07739-bf6645659-p8vvn"]
Oct 30 22:19:44.498704 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:19:44.498418    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:44.628018 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:19:44.627981    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-4da07739-bf6645659-p8vvn" event={"ID":"628be936-435a-4e97-9aaa-98924732fb01","Type":"ContainerStarted","Data":"282e85b9efc1a787703d885bb0cfb4a5260dff41039625bdf4d03942d02f3597"}
Oct 30 22:19:44.628216 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:19:44.628037    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-4da07739-bf6645659-p8vvn" event={"ID":"628be936-435a-4e97-9aaa-98924732fb01","Type":"ContainerStarted","Data":"0e494a9a0f687fb1fbf5db7e78f086b161f5f67bcb71306913a0a9cf9f0712bc"}
Oct 30 22:19:44.628216 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:19:44.628081    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-4da07739-bf6645659-p8vvn"
Oct 30 22:19:44.728506 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:19:44.728465    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:47.523339 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:19:47.523299    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:48.372138 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:19:48.372097    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:48.690257 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:19:48.659463    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:49.444498 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:19:49.444425    5510 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-4da07739-bf6645659-p8vvn" podStartSLOduration=8.44440413 podStartE2EDuration="8.44440413s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:19:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:19:44.763995037 +0000 UTC m=+1765.324805070" watchObservedRunningTime="2026-10-30 22:19:49.44440413 +0000 UTC m=+1770.005214140"
Oct 30 22:19:49.447538 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:19:49.447507    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-ac5024ad-6685bb6d9f-prktq"]
Oct 30 22:19:49.505030 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:19:49.504997    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-ac5024ad-6bfdb47b7-fxbsd"]
Oct 30 22:19:49.772419 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:19:49.772359    5510 patch_prober.go:29] interesting pod/e2e-shared-a-w0-c42c0a-openshift-default-558b6f6d6f-7rcsl container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.99:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 22:19:49.772921 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:19:49.772441    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w0-c42c0a-openshift-default-558b6f6d6f-7rcsl" podUID="3a76b5b9-88cf-4f7f-9d14-fd725017b62f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.99:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:19:49.794421 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:19:49.794356    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:49.998314 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:19:49.994158    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:51.846069 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:19:51.846037    5510 generic.go:356] "Generic (PLEG): container finished" podID="01891eb3-3336-4e31-a26a-9a1ecb935f9c" containerID="15c70c82c935624bfc0094fd21db370e95de86bdda73f8cb176f92455e7fd968" exitCode=0
Oct 30 22:19:51.846514 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:19:51.846118    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-bee2db53-openshift-default-6b56b8cdb9-jv7mv" event={"ID":"01891eb3-3336-4e31-a26a-9a1ecb935f9c","Type":"ContainerDied","Data":"15c70c82c935624bfc0094fd21db370e95de86bdda73f8cb176f92455e7fd968"}
Oct 30 22:19:51.846651 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:19:51.846633    5510 scope.go:122] "RemoveContainer" containerID="15c70c82c935624bfc0094fd21db370e95de86bdda73f8cb176f92455e7fd968"
Oct 30 22:19:51.848054 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:19:51.848032    5510 generic.go:356] "Generic (PLEG): container finished" podID="3a76b5b9-88cf-4f7f-9d14-fd725017b62f" containerID="f9f8ec7c0b142468c3f7862ce226aa9aa72fbaeb489447aefd7e17bb2921aae2" exitCode=0
Oct 30 22:19:51.848220 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:19:51.848191    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w0-c42c0a-openshift-default-558b6f6d6f-7rcsl" event={"ID":"3a76b5b9-88cf-4f7f-9d14-fd725017b62f","Type":"ContainerDied","Data":"f9f8ec7c0b142468c3f7862ce226aa9aa72fbaeb489447aefd7e17bb2921aae2"}
Oct 30 22:19:51.848759 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:19:51.848740    5510 scope.go:122] "RemoveContainer" containerID="f9f8ec7c0b142468c3f7862ce226aa9aa72fbaeb489447aefd7e17bb2921aae2"
Oct 30 22:19:51.852984 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:19:51.852958    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:19:51.853084 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:19:51.853012    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:19:51.853140 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:19:51.853097    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:19:51.853140 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:19:51.853025    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:19:51.853210 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:19:51.853144    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:19:51.853250 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:19:51.853143    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:19:52.898550 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:19:52.898509    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-bee2db53-openshift-default-6b56b8cdb9-jv7mv" event={"ID":"01891eb3-3336-4e31-a26a-9a1ecb935f9c","Type":"ContainerStarted","Data":"bc8c3d1c2035a325ec675005f3202f183bae9d45f8881edd84ec3f62791803df"}
Oct 30 22:19:52.900892 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:19:52.900868    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w0-c42c0a-openshift-default-558b6f6d6f-7rcsl" event={"ID":"3a76b5b9-88cf-4f7f-9d14-fd725017b62f","Type":"ContainerStarted","Data":"a0c3f8fb005cc45c5fc150779ea05a86b4117cb7ce1c3d7425174d06a8652a66"}
Oct 30 22:19:53.280913 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:19:53.280869    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:53.444679 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:19:53.443462    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w0-c42c0a-openshift-default-558b6f6d6f-7rcsl"
Oct 30 22:19:53.477686 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:19:53.476766    5510 patch_prober.go:29] interesting pod/e2e-shared-a-w0-c42c0a-openshift-default-558b6f6d6f-7rcsl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.99:15021/healthz/ready\": dial tcp 10.133.0.99:15021: connect: connection refused" start-of-body=
Oct 30 22:19:53.477686 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:19:53.476848    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w0-c42c0a-openshift-default-558b6f6d6f-7rcsl" podUID="3a76b5b9-88cf-4f7f-9d14-fd725017b62f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.99:15021/healthz/ready\": dial tcp 10.133.0.99:15021: connect: connection refused"
Oct 30 22:19:53.719944 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:19:53.719875    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-bee2db53-openshift-default-6b56b8cdb9-jv7mv"
Oct 30 22:19:53.721156 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:19:53.721138    5510 patch_prober.go:29] interesting pod/e2e-derive-bee2db53-openshift-default-6b56b8cdb9-jv7mv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.100:15021/healthz/ready\": dial tcp 10.133.0.100:15021: connect: connection refused" start-of-body=
Oct 30 22:19:53.721223 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:19:53.721186    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-bee2db53-openshift-default-6b56b8cdb9-jv7mv" podUID="01891eb3-3336-4e31-a26a-9a1ecb935f9c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.100:15021/healthz/ready\": dial tcp 10.133.0.100:15021: connect: connection refused"
Oct 30 22:19:54.438413 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:19:54.438376    5510 patch_prober.go:29] interesting pod/e2e-shared-a-w0-c42c0a-openshift-default-558b6f6d6f-7rcsl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.99:15021/healthz/ready\": dial tcp 10.133.0.99:15021: connect: connection refused" start-of-body=
Oct 30 22:19:54.438922 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:19:54.438428    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w0-c42c0a-openshift-default-558b6f6d6f-7rcsl" podUID="3a76b5b9-88cf-4f7f-9d14-fd725017b62f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.99:15021/healthz/ready\": dial tcp 10.133.0.99:15021: connect: connection refused"
Oct 30 22:19:54.470681 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:19:54.464775    5510 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-ac5024ad-6685bb6d9f-prktq" podUID="d9e4f1df-768a-4b77-aa2f-0fa8f8d2818f" containerName="payload-processing" containerID="cri-o://20745cc50eabc223ae637cbf5076590e1064bca5f33fd8ece001ae21b83a81b6" gracePeriod=25
Oct 30 22:19:54.505553 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:19:54.505508    5510 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-ac5024ad-6bfdb47b7-fxbsd" podUID="a08bf006-75ff-4597-8ec3-b6d8da49d808" containerName="payload-pre-processing" containerID="cri-o://f5c9187b5e793a4f18a050a02b0fec68abc6f42ccf50ade2705392380d872267" gracePeriod=25
Oct 30 22:19:54.720277 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:19:54.720192    5510 patch_prober.go:29] interesting pod/e2e-derive-bee2db53-openshift-default-6b56b8cdb9-jv7mv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.100:15021/healthz/ready\": dial tcp 10.133.0.100:15021: connect: connection refused" start-of-body=
Oct 30 22:19:54.720277 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:19:54.720256    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-bee2db53-openshift-default-6b56b8cdb9-jv7mv" podUID="01891eb3-3336-4e31-a26a-9a1ecb935f9c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.100:15021/healthz/ready\": dial tcp 10.133.0.100:15021: connect: connection refused"
Oct 30 22:19:54.929041 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:19:54.929001    5510 generic.go:356] "Generic (PLEG): container finished" podID="a08bf006-75ff-4597-8ec3-b6d8da49d808" containerID="f5c9187b5e793a4f18a050a02b0fec68abc6f42ccf50ade2705392380d872267" exitCode=0
Oct 30 22:19:54.929236 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:19:54.929056    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-ac5024ad-6bfdb47b7-fxbsd" event={"ID":"a08bf006-75ff-4597-8ec3-b6d8da49d808","Type":"ContainerDied","Data":"f5c9187b5e793a4f18a050a02b0fec68abc6f42ccf50ade2705392380d872267"}
Oct 30 22:19:54.931404 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:19:54.931377    5510 generic.go:356] "Generic (PLEG): container finished" podID="d9e4f1df-768a-4b77-aa2f-0fa8f8d2818f" containerID="20745cc50eabc223ae637cbf5076590e1064bca5f33fd8ece001ae21b83a81b6" exitCode=0
Oct 30 22:19:54.931516 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:19:54.931459    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-ac5024ad-6685bb6d9f-prktq" event={"ID":"d9e4f1df-768a-4b77-aa2f-0fa8f8d2818f","Type":"ContainerDied","Data":"20745cc50eabc223ae637cbf5076590e1064bca5f33fd8ece001ae21b83a81b6"}
Oct 30 22:19:55.018296 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:19:55.017801    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-ac5024ad-6685bb6d9f-prktq"
Oct 30 22:19:55.041285 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:19:55.041260    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d9e4f1df-768a-4b77-aa2f-0fa8f8d2818f-kube-api-access-g75ph\" (UniqueName: \"kubernetes.io/projected/d9e4f1df-768a-4b77-aa2f-0fa8f8d2818f-kube-api-access-g75ph\") pod \"d9e4f1df-768a-4b77-aa2f-0fa8f8d2818f\" (UID: \"d9e4f1df-768a-4b77-aa2f-0fa8f8d2818f\") "
Oct 30 22:19:55.041848 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:19:55.041826    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d9e4f1df-768a-4b77-aa2f-0fa8f8d2818f-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d9e4f1df-768a-4b77-aa2f-0fa8f8d2818f-plugins-config-volume\") pod \"d9e4f1df-768a-4b77-aa2f-0fa8f8d2818f\" (UID: \"d9e4f1df-768a-4b77-aa2f-0fa8f8d2818f\") "
Oct 30 22:19:55.042235 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:19:55.042207    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d9e4f1df-768a-4b77-aa2f-0fa8f8d2818f-plugins-config-volume" pod "d9e4f1df-768a-4b77-aa2f-0fa8f8d2818f" (UID: "d9e4f1df-768a-4b77-aa2f-0fa8f8d2818f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:19:55.042869 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:19:55.042847    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-ac5024ad-6bfdb47b7-fxbsd"
Oct 30 22:19:55.043075 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:19:55.043055    5510 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d9e4f1df-768a-4b77-aa2f-0fa8f8d2818f-plugins-config-volume\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:19:55.043439 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:19:55.043421    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d9e4f1df-768a-4b77-aa2f-0fa8f8d2818f-kube-api-access-g75ph" pod "d9e4f1df-768a-4b77-aa2f-0fa8f8d2818f" (UID: "d9e4f1df-768a-4b77-aa2f-0fa8f8d2818f"). InnerVolumeSpecName "kube-api-access-g75ph". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:19:55.203433 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:19:55.199353    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a08bf006-75ff-4597-8ec3-b6d8da49d808-kube-api-access-jlzgt\" (UniqueName: \"kubernetes.io/projected/a08bf006-75ff-4597-8ec3-b6d8da49d808-kube-api-access-jlzgt\") pod \"a08bf006-75ff-4597-8ec3-b6d8da49d808\" (UID: \"a08bf006-75ff-4597-8ec3-b6d8da49d808\") "
Oct 30 22:19:55.203433 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:19:55.199689    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a08bf006-75ff-4597-8ec3-b6d8da49d808-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a08bf006-75ff-4597-8ec3-b6d8da49d808-plugins-config-volume\") pod \"a08bf006-75ff-4597-8ec3-b6d8da49d808\" (UID: \"a08bf006-75ff-4597-8ec3-b6d8da49d808\") "
Oct 30 22:19:55.203433 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:19:55.200777    5510 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g75ph\" (UniqueName: \"kubernetes.io/projected/d9e4f1df-768a-4b77-aa2f-0fa8f8d2818f-kube-api-access-g75ph\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:19:55.203433 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:19:55.201208    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a08bf006-75ff-4597-8ec3-b6d8da49d808-plugins-config-volume" pod "a08bf006-75ff-4597-8ec3-b6d8da49d808" (UID: "a08bf006-75ff-4597-8ec3-b6d8da49d808"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:19:55.204560 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:19:55.204535    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a08bf006-75ff-4597-8ec3-b6d8da49d808-kube-api-access-jlzgt" pod "a08bf006-75ff-4597-8ec3-b6d8da49d808" (UID: "a08bf006-75ff-4597-8ec3-b6d8da49d808"). InnerVolumeSpecName "kube-api-access-jlzgt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:19:55.306525 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:19:55.305876    5510 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a08bf006-75ff-4597-8ec3-b6d8da49d808-plugins-config-volume\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:19:55.306525 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:19:55.305912    5510 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jlzgt\" (UniqueName: \"kubernetes.io/projected/a08bf006-75ff-4597-8ec3-b6d8da49d808-kube-api-access-jlzgt\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:19:55.439052 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:19:55.439019    5510 patch_prober.go:29] interesting pod/e2e-shared-a-w0-c42c0a-openshift-default-558b6f6d6f-7rcsl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.99:15021/healthz/ready\": dial tcp 10.133.0.99:15021: connect: connection refused" start-of-body=
Oct 30 22:19:55.439519 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:19:55.439074    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w0-c42c0a-openshift-default-558b6f6d6f-7rcsl" podUID="3a76b5b9-88cf-4f7f-9d14-fd725017b62f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.99:15021/healthz/ready\": dial tcp 10.133.0.99:15021: connect: connection refused"
Oct 30 22:19:55.640980 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:19:55.640951    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-4da07739-bf6645659-p8vvn"
Oct 30 22:19:55.720912 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:19:55.720879    5510 patch_prober.go:29] interesting pod/e2e-derive-bee2db53-openshift-default-6b56b8cdb9-jv7mv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.100:15021/healthz/ready\": dial tcp 10.133.0.100:15021: connect: connection refused" start-of-body=
Oct 30 22:19:55.721091 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:19:55.720931    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-bee2db53-openshift-default-6b56b8cdb9-jv7mv" podUID="01891eb3-3336-4e31-a26a-9a1ecb935f9c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.100:15021/healthz/ready\": dial tcp 10.133.0.100:15021: connect: connection refused"
Oct 30 22:19:55.941504 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:19:55.941419    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-ac5024ad-6bfdb47b7-fxbsd"
Oct 30 22:19:55.941504 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:19:55.941435    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-ac5024ad-6bfdb47b7-fxbsd" event={"ID":"a08bf006-75ff-4597-8ec3-b6d8da49d808","Type":"ContainerDied","Data":"71ca3d9fd1e755ed1138ac1606f10bc229bbbd4712d09c2ec5b4f50f61450dc1"}
Oct 30 22:19:55.941504 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:19:55.941503    5510 scope.go:122] "RemoveContainer" containerID="f5c9187b5e793a4f18a050a02b0fec68abc6f42ccf50ade2705392380d872267"
Oct 30 22:19:55.943083 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:19:55.943018    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-ac5024ad-6685bb6d9f-prktq"
Oct 30 22:19:55.943160 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:19:55.943082    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-ac5024ad-6685bb6d9f-prktq" event={"ID":"d9e4f1df-768a-4b77-aa2f-0fa8f8d2818f","Type":"ContainerDied","Data":"3710688cb4212dd120a49b9a0edbe1f5c47c7c1e38d748ba235271fa10199b08"}
Oct 30 22:19:56.003862 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:19:56.003837    5510 scope.go:122] "RemoveContainer" containerID="20745cc50eabc223ae637cbf5076590e1064bca5f33fd8ece001ae21b83a81b6"
Oct 30 22:19:56.132879 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:19:56.132839    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-ac5024ad-6685bb6d9f-prktq"]
Oct 30 22:19:56.137188 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:19:56.137160    5510 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-ac5024ad-6685bb6d9f-prktq"]
Oct 30 22:19:56.301712 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:19:56.301633    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-ac5024ad-6bfdb47b7-fxbsd"]
Oct 30 22:19:56.307272 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:19:56.307222    5510 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-ac5024ad-6bfdb47b7-fxbsd"]
Oct 30 22:19:56.438196 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:19:56.437776    5510 patch_prober.go:29] interesting pod/e2e-shared-a-w0-c42c0a-openshift-default-558b6f6d6f-7rcsl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.99:15021/healthz/ready\": dial tcp 10.133.0.99:15021: connect: connection refused" start-of-body=
Oct 30 22:19:56.438196 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:19:56.437837    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w0-c42c0a-openshift-default-558b6f6d6f-7rcsl" podUID="3a76b5b9-88cf-4f7f-9d14-fd725017b62f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.99:15021/healthz/ready\": dial tcp 10.133.0.99:15021: connect: connection refused"
Oct 30 22:19:56.720560 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:19:56.720467    5510 patch_prober.go:29] interesting pod/e2e-derive-bee2db53-openshift-default-6b56b8cdb9-jv7mv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.100:15021/healthz/ready\": dial tcp 10.133.0.100:15021: connect: connection refused" start-of-body=
Oct 30 22:19:56.720560 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:19:56.720529    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-bee2db53-openshift-default-6b56b8cdb9-jv7mv" podUID="01891eb3-3336-4e31-a26a-9a1ecb935f9c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.100:15021/healthz/ready\": dial tcp 10.133.0.100:15021: connect: connection refused"
Oct 30 22:19:57.333798 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:19:57.333749    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-bee2db53-openshift-default-6b56b8cdb9-jv7mv"
Oct 30 22:19:57.438591 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:19:57.438559    5510 patch_prober.go:29] interesting pod/e2e-shared-a-w0-c42c0a-openshift-default-558b6f6d6f-7rcsl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.99:15021/healthz/ready\": dial tcp 10.133.0.99:15021: connect: connection refused" start-of-body=
Oct 30 22:19:57.438773 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:19:57.438612    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w0-c42c0a-openshift-default-558b6f6d6f-7rcsl" podUID="3a76b5b9-88cf-4f7f-9d14-fd725017b62f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.99:15021/healthz/ready\": dial tcp 10.133.0.99:15021: connect: connection refused"
Oct 30 22:19:57.599747 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:19:57.599624    5510 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a08bf006-75ff-4597-8ec3-b6d8da49d808" path="/var/lib/kubelet/pods/a08bf006-75ff-4597-8ec3-b6d8da49d808/volumes"
Oct 30 22:19:57.600209 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:19:57.600191    5510 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d9e4f1df-768a-4b77-aa2f-0fa8f8d2818f" path="/var/lib/kubelet/pods/d9e4f1df-768a-4b77-aa2f-0fa8f8d2818f/volumes"
Oct 30 22:19:57.720552 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:19:57.720517    5510 patch_prober.go:29] interesting pod/e2e-derive-bee2db53-openshift-default-6b56b8cdb9-jv7mv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.100:15021/healthz/ready\": dial tcp 10.133.0.100:15021: connect: connection refused" start-of-body=
Oct 30 22:19:57.720765 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:19:57.720570    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-bee2db53-openshift-default-6b56b8cdb9-jv7mv" podUID="01891eb3-3336-4e31-a26a-9a1ecb935f9c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.100:15021/healthz/ready\": dial tcp 10.133.0.100:15021: connect: connection refused"
Oct 30 22:19:58.350784 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:19:58.345457    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:58.476843 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:19:58.469960    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w0-c42c0a-openshift-default-558b6f6d6f-7rcsl"
Oct 30 22:19:58.476843 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:19:58.471780    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w0-c42c0a-openshift-default-558b6f6d6f-7rcsl"
Oct 30 22:19:58.480865 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:19:58.479000    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w0-c42c0a-openshift-default-558b6f6d6f-7rcsl"
Oct 30 22:19:58.725188 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:19:58.725107    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-bee2db53-openshift-default-6b56b8cdb9-jv7mv"
Oct 30 22:19:58.726382 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:19:58.726359    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-bee2db53-openshift-default-6b56b8cdb9-jv7mv"
Oct 30 22:20:01.591617 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:01.591436    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-bee2db53-78cf85d8ff-w2jnq"]
Oct 30 22:20:02.108692 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:02.108615    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:20:02.425735 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:02.425320    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:20:02.941681 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:02.940959    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:20:03.195731 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:03.195375    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:20:03.661678 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:03.661197    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:20:06.492511 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:06.492477    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:20:06.592130 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:06.592085    5510 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-derive-bee2db53-78cf85d8ff-w2jnq" podUID="236afc02-33f3-40f9-b5ee-16816eac2442" containerName="payload-processing" containerID="cri-o://236ed3024d7f71f5032ffbb6b27644990cbf44b44d8a7c685b566516957f9e0b" gracePeriod=25
Oct 30 22:20:06.895181 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:06.895155    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-bee2db53-78cf85d8ff-w2jnq"
Oct 30 22:20:06.914979 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:06.914955    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/236afc02-33f3-40f9-b5ee-16816eac2442-kube-api-access-82qmq\" (UniqueName: \"kubernetes.io/projected/236afc02-33f3-40f9-b5ee-16816eac2442-kube-api-access-82qmq\") pod \"236afc02-33f3-40f9-b5ee-16816eac2442\" (UID: \"236afc02-33f3-40f9-b5ee-16816eac2442\") "
Oct 30 22:20:06.915131 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:06.915072    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/236afc02-33f3-40f9-b5ee-16816eac2442-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/236afc02-33f3-40f9-b5ee-16816eac2442-plugins-config-volume\") pod \"236afc02-33f3-40f9-b5ee-16816eac2442\" (UID: \"236afc02-33f3-40f9-b5ee-16816eac2442\") "
Oct 30 22:20:06.915447 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:06.915422    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/236afc02-33f3-40f9-b5ee-16816eac2442-plugins-config-volume" pod "236afc02-33f3-40f9-b5ee-16816eac2442" (UID: "236afc02-33f3-40f9-b5ee-16816eac2442"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:20:06.915637 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:06.915619    5510 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/236afc02-33f3-40f9-b5ee-16816eac2442-plugins-config-volume\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:20:06.917032 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:06.917010    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/236afc02-33f3-40f9-b5ee-16816eac2442-kube-api-access-82qmq" pod "236afc02-33f3-40f9-b5ee-16816eac2442" (UID: "236afc02-33f3-40f9-b5ee-16816eac2442"). InnerVolumeSpecName "kube-api-access-82qmq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:20:07.016930 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:07.016904    5510 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-82qmq\" (UniqueName: \"kubernetes.io/projected/236afc02-33f3-40f9-b5ee-16816eac2442-kube-api-access-82qmq\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:20:07.160451 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:07.160410    5510 generic.go:356] "Generic (PLEG): container finished" podID="236afc02-33f3-40f9-b5ee-16816eac2442" containerID="236ed3024d7f71f5032ffbb6b27644990cbf44b44d8a7c685b566516957f9e0b" exitCode=0
Oct 30 22:20:07.160451 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:07.160460    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-bee2db53-78cf85d8ff-w2jnq"
Oct 30 22:20:07.160745 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:07.160490    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-bee2db53-78cf85d8ff-w2jnq" event={"ID":"236afc02-33f3-40f9-b5ee-16816eac2442","Type":"ContainerDied","Data":"236ed3024d7f71f5032ffbb6b27644990cbf44b44d8a7c685b566516957f9e0b"}
Oct 30 22:20:07.160745 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:07.160554    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-bee2db53-78cf85d8ff-w2jnq" event={"ID":"236afc02-33f3-40f9-b5ee-16816eac2442","Type":"ContainerDied","Data":"d612df8495b9cae815b6ee3e4f63950f21035eb6c5cb4fc0e5b8b9b93992ee01"}
Oct 30 22:20:07.160745 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:07.160583    5510 scope.go:122] "RemoveContainer" containerID="236ed3024d7f71f5032ffbb6b27644990cbf44b44d8a7c685b566516957f9e0b"
Oct 30 22:20:07.171786 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:07.171766    5510 scope.go:122] "RemoveContainer" containerID="236ed3024d7f71f5032ffbb6b27644990cbf44b44d8a7c685b566516957f9e0b"
Oct 30 22:20:07.172061 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:20:07.172042    5510 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"236ed3024d7f71f5032ffbb6b27644990cbf44b44d8a7c685b566516957f9e0b\": container with ID starting with 236ed3024d7f71f5032ffbb6b27644990cbf44b44d8a7c685b566516957f9e0b not found: ID does not exist" containerID="236ed3024d7f71f5032ffbb6b27644990cbf44b44d8a7c685b566516957f9e0b"
Oct 30 22:20:07.172112 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:07.172071    5510 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"236ed3024d7f71f5032ffbb6b27644990cbf44b44d8a7c685b566516957f9e0b"} err="failed to get container status \"236ed3024d7f71f5032ffbb6b27644990cbf44b44d8a7c685b566516957f9e0b\": rpc error: code = NotFound desc = could not find container \"236ed3024d7f71f5032ffbb6b27644990cbf44b44d8a7c685b566516957f9e0b\": container with ID starting with 236ed3024d7f71f5032ffbb6b27644990cbf44b44d8a7c685b566516957f9e0b not found: ID does not exist"
Oct 30 22:20:07.319521 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:07.319438    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-bee2db53-78cf85d8ff-w2jnq"]
Oct 30 22:20:07.323759 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:07.323733    5510 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-bee2db53-78cf85d8ff-w2jnq"]
Oct 30 22:20:07.554995 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:07.554957    5510 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="236afc02-33f3-40f9-b5ee-16816eac2442" path="/var/lib/kubelet/pods/236afc02-33f3-40f9-b5ee-16816eac2442/volumes"
Oct 30 22:20:07.994606 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:07.994569    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:20:08.297691 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:08.288193    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:20:08.564434 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:20:08.564332    5510 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6gjbf" containerName="extractor"
Oct 30 22:20:11.179983 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:11.179943    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:20:11.424122 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:11.423857    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:20:11.626681 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:11.621183    5510 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w5-3cc1b3-58b54f64ff-q8k4w"]
Oct 30 22:20:11.636681 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:11.628610    5510 cpu_manager.go:414] "containerMap: removing container" podUID="236afc02-33f3-40f9-b5ee-16816eac2442" containerName="payload-processing"
Oct 30 22:20:11.636681 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:11.628644    5510 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="236afc02-33f3-40f9-b5ee-16816eac2442" containerName="payload-processing"
Oct 30 22:20:11.636681 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:11.628695    5510 cpu_manager.go:414] "containerMap: removing container" podUID="a08bf006-75ff-4597-8ec3-b6d8da49d808" containerName="payload-pre-processing"
Oct 30 22:20:11.636681 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:11.628705    5510 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a08bf006-75ff-4597-8ec3-b6d8da49d808" containerName="payload-pre-processing"
Oct 30 22:20:11.636681 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:11.628734    5510 cpu_manager.go:414] "containerMap: removing container" podUID="d9e4f1df-768a-4b77-aa2f-0fa8f8d2818f" containerName="payload-processing"
Oct 30 22:20:11.636681 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:11.628743    5510 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d9e4f1df-768a-4b77-aa2f-0fa8f8d2818f" containerName="payload-processing"
Oct 30 22:20:11.636681 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:11.629480    5510 memory_manager.go:365] "RemoveStaleState removing state" podUID="236afc02-33f3-40f9-b5ee-16816eac2442" containerName="payload-processing"
Oct 30 22:20:11.636681 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:11.629505    5510 memory_manager.go:365] "RemoveStaleState removing state" podUID="a08bf006-75ff-4597-8ec3-b6d8da49d808" containerName="payload-pre-processing"
Oct 30 22:20:11.636681 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:11.629516    5510 memory_manager.go:365] "RemoveStaleState removing state" podUID="d9e4f1df-768a-4b77-aa2f-0fa8f8d2818f" containerName="payload-processing"
Oct 30 22:20:11.650002 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:11.649966    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w5-3cc1b3-58b54f64ff-q8k4w"
Oct 30 22:20:11.664461 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:11.659923    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w5-3cc1b3-58b54f64ff-q8k4w"]
Oct 30 22:20:11.700907 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:11.700871    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d5825ebb-76c4-4a3a-aa17-2bbedab22772-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w5-3cc1b3-58b54f64ff-q8k4w\" (UID: \"d5825ebb-76c4-4a3a-aa17-2bbedab22772\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w5-3cc1b3-58b54f64ff-q8k4w"
Oct 30 22:20:11.701541 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:11.701513    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rsvdp\" (UniqueName: \"kubernetes.io/projected/d5825ebb-76c4-4a3a-aa17-2bbedab22772-kube-api-access-rsvdp\") pod \"payload-pre-processing-e2e-shared-a-w5-3cc1b3-58b54f64ff-q8k4w\" (UID: \"d5825ebb-76c4-4a3a-aa17-2bbedab22772\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w5-3cc1b3-58b54f64ff-q8k4w"
Oct 30 22:20:11.752677 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:11.729530    5510 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w5-3cc1b3-f5bd94dd5-pkc6c"]
Oct 30 22:20:11.757979 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:11.757944    5510 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w5-3cc1b3\""
Oct 30 22:20:11.758382 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:11.758362    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w5-3cc1b3-dockercfg-9sbk2\""
Oct 30 22:20:11.783722 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:11.777940    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w5-3cc1b3-f5bd94dd5-pkc6c"
Oct 30 22:20:11.820855 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:11.817173    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rsvdp\" (UniqueName: \"kubernetes.io/projected/d5825ebb-76c4-4a3a-aa17-2bbedab22772-kube-api-access-rsvdp\") pod \"payload-pre-processing-e2e-shared-a-w5-3cc1b3-58b54f64ff-q8k4w\" (UID: \"d5825ebb-76c4-4a3a-aa17-2bbedab22772\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w5-3cc1b3-58b54f64ff-q8k4w"
Oct 30 22:20:11.820855 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:11.817466    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d5825ebb-76c4-4a3a-aa17-2bbedab22772-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w5-3cc1b3-58b54f64ff-q8k4w\" (UID: \"d5825ebb-76c4-4a3a-aa17-2bbedab22772\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w5-3cc1b3-58b54f64ff-q8k4w"
Oct 30 22:20:11.890733 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:11.890012    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d5825ebb-76c4-4a3a-aa17-2bbedab22772-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w5-3cc1b3-58b54f64ff-q8k4w\" (UID: \"d5825ebb-76c4-4a3a-aa17-2bbedab22772\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w5-3cc1b3-58b54f64ff-q8k4w"
Oct 30 22:20:11.940688 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:11.940426    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w5-3cc1b3-f5bd94dd5-pkc6c"]
Oct 30 22:20:11.990735 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:11.981223    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rsvdp\" (UniqueName: \"kubernetes.io/projected/d5825ebb-76c4-4a3a-aa17-2bbedab22772-kube-api-access-rsvdp\") pod \"payload-pre-processing-e2e-shared-a-w5-3cc1b3-58b54f64ff-q8k4w\" (UID: \"d5825ebb-76c4-4a3a-aa17-2bbedab22772\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w5-3cc1b3-58b54f64ff-q8k4w"
Oct 30 22:20:11.991585 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:11.991212    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w5-3cc1b3-58b54f64ff-q8k4w"
Oct 30 22:20:12.074725 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:12.041416    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7f89f8ac-ad11-4aac-9f27-95d05190fd97-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w5-3cc1b3-f5bd94dd5-pkc6c\" (UID: \"7f89f8ac-ad11-4aac-9f27-95d05190fd97\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w5-3cc1b3-f5bd94dd5-pkc6c"
Oct 30 22:20:12.074725 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:12.041609    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t7wqz\" (UniqueName: \"kubernetes.io/projected/7f89f8ac-ad11-4aac-9f27-95d05190fd97-kube-api-access-t7wqz\") pod \"payload-processing-e2e-shared-a-w5-3cc1b3-f5bd94dd5-pkc6c\" (UID: \"7f89f8ac-ad11-4aac-9f27-95d05190fd97\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w5-3cc1b3-f5bd94dd5-pkc6c"
Oct 30 22:20:12.157992 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:12.157953    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7f89f8ac-ad11-4aac-9f27-95d05190fd97-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w5-3cc1b3-f5bd94dd5-pkc6c\" (UID: \"7f89f8ac-ad11-4aac-9f27-95d05190fd97\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w5-3cc1b3-f5bd94dd5-pkc6c"
Oct 30 22:20:12.158381 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:12.158350    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t7wqz\" (UniqueName: \"kubernetes.io/projected/7f89f8ac-ad11-4aac-9f27-95d05190fd97-kube-api-access-t7wqz\") pod \"payload-processing-e2e-shared-a-w5-3cc1b3-f5bd94dd5-pkc6c\" (UID: \"7f89f8ac-ad11-4aac-9f27-95d05190fd97\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w5-3cc1b3-f5bd94dd5-pkc6c"
Oct 30 22:20:12.176892 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:12.176842    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7f89f8ac-ad11-4aac-9f27-95d05190fd97-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w5-3cc1b3-f5bd94dd5-pkc6c\" (UID: \"7f89f8ac-ad11-4aac-9f27-95d05190fd97\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w5-3cc1b3-f5bd94dd5-pkc6c"
Oct 30 22:20:12.245678 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:12.241496    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t7wqz\" (UniqueName: \"kubernetes.io/projected/7f89f8ac-ad11-4aac-9f27-95d05190fd97-kube-api-access-t7wqz\") pod \"payload-processing-e2e-shared-a-w5-3cc1b3-f5bd94dd5-pkc6c\" (UID: \"7f89f8ac-ad11-4aac-9f27-95d05190fd97\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w5-3cc1b3-f5bd94dd5-pkc6c"
Oct 30 22:20:12.295148 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:12.294936    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w5-3cc1b3-f5bd94dd5-pkc6c"
Oct 30 22:20:12.632431 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:12.632394    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w5-3cc1b3-58b54f64ff-q8k4w"]
Oct 30 22:20:12.633721 ip-10-0-136-210 kubenswrapper[5510]: W1030 22:20:12.633693    5510 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5825ebb_76c4_4a3a_aa17_2bbedab22772.slice/crio-8a3e44f4b1de967cdf3173434fb40111e7d8b0ddc4c5aa2f64c4645c4be7eccf WatchSource:0}: Error finding container 8a3e44f4b1de967cdf3173434fb40111e7d8b0ddc4c5aa2f64c4645c4be7eccf: Status 404 returned error can't find the container with id 8a3e44f4b1de967cdf3173434fb40111e7d8b0ddc4c5aa2f64c4645c4be7eccf
Oct 30 22:20:12.688967 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:12.688933    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w5-3cc1b3-f5bd94dd5-pkc6c"]
Oct 30 22:20:13.469285 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:13.469240    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:20:13.632683 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:13.627034    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w5-3cc1b3-58b54f64ff-q8k4w" event={"ID":"d5825ebb-76c4-4a3a-aa17-2bbedab22772","Type":"ContainerStarted","Data":"72897713dc02410a1922c9888fd4a294b0f318d98846c5030ebb92eb08c0f5ea"}
Oct 30 22:20:13.632683 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:13.627158    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w5-3cc1b3-58b54f64ff-q8k4w" event={"ID":"d5825ebb-76c4-4a3a-aa17-2bbedab22772","Type":"ContainerStarted","Data":"8a3e44f4b1de967cdf3173434fb40111e7d8b0ddc4c5aa2f64c4645c4be7eccf"}
Oct 30 22:20:13.632683 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:13.628275    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w5-3cc1b3-58b54f64ff-q8k4w"
Oct 30 22:20:13.639452 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:13.639414    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w5-3cc1b3-f5bd94dd5-pkc6c" event={"ID":"7f89f8ac-ad11-4aac-9f27-95d05190fd97","Type":"ContainerStarted","Data":"3298ca3870d7ec5968d8437f46b95557e3053a6c4643e5cb8c41735dd0e2abeb"}
Oct 30 22:20:13.639638 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:13.639477    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w5-3cc1b3-f5bd94dd5-pkc6c" event={"ID":"7f89f8ac-ad11-4aac-9f27-95d05190fd97","Type":"ContainerStarted","Data":"c39236d9de7e4efe987a9876c15f31dd9347fdb7844b20e06e7cedcfa2edaf01"}
Oct 30 22:20:13.640419 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:13.640393    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w5-3cc1b3-f5bd94dd5-pkc6c"
Oct 30 22:20:13.675950 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:13.675914    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:20:13.744229 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:13.744134    5510 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w5-3cc1b3-58b54f64ff-q8k4w" podStartSLOduration=2.744120919 podStartE2EDuration="2.744120919s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:20:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:20:13.740788422 +0000 UTC m=+1794.301598427" watchObservedRunningTime="2026-10-30 22:20:13.744120919 +0000 UTC m=+1794.304930925"
Oct 30 22:20:13.853130 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:13.853070    5510 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w5-3cc1b3-f5bd94dd5-pkc6c" podStartSLOduration=2.853052027 podStartE2EDuration="2.853052027s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:20:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:20:13.850466789 +0000 UTC m=+1794.411276814" watchObservedRunningTime="2026-10-30 22:20:13.853052027 +0000 UTC m=+1794.413862073"
Oct 30 22:20:18.309667 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:18.308796    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:20:21.385281 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:21.385242    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-q6kh7"]
Oct 30 22:20:21.385834 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:21.385448    5510 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-q6kh7" podUID="3c82a381-05ad-436f-89dd-0985c3dbad33" containerName="istio-proxy" containerID="cri-o://958d8521ac74ff55fa78bffb7e2167e3fe59f664dd1035b5ed908a7a39c28c13" gracePeriod=30
Oct 30 22:20:21.431205 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:21.431165    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-m2c8l"]
Oct 30 22:20:21.431445 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:21.431420    5510 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-m2c8l" podUID="4798995a-2305-4dae-adf7-45aa66e0cb18" containerName="istio-proxy" containerID="cri-o://23d5aae41a6a6b3345ee01975c2b297980a9c3b6129f27ad2156d594c524e4cd" gracePeriod=30
Oct 30 22:20:21.435927 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:21.435892    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-k5dpm"]
Oct 30 22:20:21.436119 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:21.436078    5510 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-k5dpm" podUID="8724086f-a5f7-4c63-87da-cb9228b03a0c" containerName="istio-proxy" containerID="cri-o://c8299b28bfa098ae0dbbb915292d8ab452f1ad709248e8fc8ea6090ab39f5a99" gracePeriod=30
Oct 30 22:20:22.147581 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:22.147549    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-ac5024ad-6dcc474c55-dh4j9"]
Oct 30 22:20:22.147807 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:22.147779    5510 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ac5024ad-6dcc474c55-dh4j9" podUID="4e58f98c-97c4-4bfa-b934-56414dfd53c7" containerName="maas-api" containerID="cri-o://b5efb250cbd4d90d3459ae1661b81d10407931f0de37454fe8b43d0c5abe6a72" gracePeriod=30
Oct 30 22:20:22.295271 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:22.295233    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:20:22.771796 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:22.769161    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ac5024ad-6dcc474c55-dh4j9"
Oct 30 22:20:22.778698 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:22.777080    5510 generic.go:356] "Generic (PLEG): container finished" podID="4e58f98c-97c4-4bfa-b934-56414dfd53c7" containerID="b5efb250cbd4d90d3459ae1661b81d10407931f0de37454fe8b43d0c5abe6a72" exitCode=0
Oct 30 22:20:22.778698 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:22.777423    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ac5024ad-6dcc474c55-dh4j9" event={"ID":"4e58f98c-97c4-4bfa-b934-56414dfd53c7","Type":"ContainerDied","Data":"b5efb250cbd4d90d3459ae1661b81d10407931f0de37454fe8b43d0c5abe6a72"}
Oct 30 22:20:22.778698 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:22.777495    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ac5024ad-6dcc474c55-dh4j9" event={"ID":"4e58f98c-97c4-4bfa-b934-56414dfd53c7","Type":"ContainerDied","Data":"dd6548dabee491d368d1b7ec7f598869d65b1c5db5a64f9547a4950c7fd9cd3a"}
Oct 30 22:20:22.778698 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:22.777545    5510 scope.go:122] "RemoveContainer" containerID="b5efb250cbd4d90d3459ae1661b81d10407931f0de37454fe8b43d0c5abe6a72"
Oct 30 22:20:22.880580 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:22.878217    5510 scope.go:122] "RemoveContainer" containerID="b5efb250cbd4d90d3459ae1661b81d10407931f0de37454fe8b43d0c5abe6a72"
Oct 30 22:20:22.880580 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:20:22.878841    5510 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b5efb250cbd4d90d3459ae1661b81d10407931f0de37454fe8b43d0c5abe6a72\": container with ID starting with b5efb250cbd4d90d3459ae1661b81d10407931f0de37454fe8b43d0c5abe6a72 not found: ID does not exist" containerID="b5efb250cbd4d90d3459ae1661b81d10407931f0de37454fe8b43d0c5abe6a72"
Oct 30 22:20:22.880580 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:22.878879    5510 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b5efb250cbd4d90d3459ae1661b81d10407931f0de37454fe8b43d0c5abe6a72"} err="failed to get container status \"b5efb250cbd4d90d3459ae1661b81d10407931f0de37454fe8b43d0c5abe6a72\": rpc error: code = NotFound desc = could not find container \"b5efb250cbd4d90d3459ae1661b81d10407931f0de37454fe8b43d0c5abe6a72\": container with ID starting with b5efb250cbd4d90d3459ae1661b81d10407931f0de37454fe8b43d0c5abe6a72 not found: ID does not exist"
Oct 30 22:20:22.901778 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:22.901718    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4e58f98c-97c4-4bfa-b934-56414dfd53c7-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e58f98c-97c4-4bfa-b934-56414dfd53c7-openshift-service-ca\") pod \"4e58f98c-97c4-4bfa-b934-56414dfd53c7\" (UID: \"4e58f98c-97c4-4bfa-b934-56414dfd53c7\") "
Oct 30 22:20:22.902076 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:22.902048    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4e58f98c-97c4-4bfa-b934-56414dfd53c7-openshift-service-ca" pod "4e58f98c-97c4-4bfa-b934-56414dfd53c7" (UID: "4e58f98c-97c4-4bfa-b934-56414dfd53c7"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:20:22.902428 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:22.902402    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4e58f98c-97c4-4bfa-b934-56414dfd53c7-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4e58f98c-97c4-4bfa-b934-56414dfd53c7-maas-api-tls\") pod \"4e58f98c-97c4-4bfa-b934-56414dfd53c7\" (UID: \"4e58f98c-97c4-4bfa-b934-56414dfd53c7\") "
Oct 30 22:20:22.902524 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:22.902515    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4e58f98c-97c4-4bfa-b934-56414dfd53c7-metrics-certs\" (UniqueName: \"kubernetes.io/secret/4e58f98c-97c4-4bfa-b934-56414dfd53c7-metrics-certs\") pod \"4e58f98c-97c4-4bfa-b934-56414dfd53c7\" (UID: \"4e58f98c-97c4-4bfa-b934-56414dfd53c7\") "
Oct 30 22:20:22.902778 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:22.902756    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4e58f98c-97c4-4bfa-b934-56414dfd53c7-kube-api-access-nc2kh\" (UniqueName: \"kubernetes.io/projected/4e58f98c-97c4-4bfa-b934-56414dfd53c7-kube-api-access-nc2kh\") pod \"4e58f98c-97c4-4bfa-b934-56414dfd53c7\" (UID: \"4e58f98c-97c4-4bfa-b934-56414dfd53c7\") "
Oct 30 22:20:22.903484 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:22.903464    5510 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e58f98c-97c4-4bfa-b934-56414dfd53c7-openshift-service-ca\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:20:22.904706 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:22.904643    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4e58f98c-97c4-4bfa-b934-56414dfd53c7-metrics-certs" pod "4e58f98c-97c4-4bfa-b934-56414dfd53c7" (UID: "4e58f98c-97c4-4bfa-b934-56414dfd53c7"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:20:22.904843 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:22.904820    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4e58f98c-97c4-4bfa-b934-56414dfd53c7-maas-api-tls" pod "4e58f98c-97c4-4bfa-b934-56414dfd53c7" (UID: "4e58f98c-97c4-4bfa-b934-56414dfd53c7"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:20:22.905421 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:22.905398    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4e58f98c-97c4-4bfa-b934-56414dfd53c7-kube-api-access-nc2kh" pod "4e58f98c-97c4-4bfa-b934-56414dfd53c7" (UID: "4e58f98c-97c4-4bfa-b934-56414dfd53c7"). InnerVolumeSpecName "kube-api-access-nc2kh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:20:23.005572 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:23.005534    5510 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nc2kh\" (UniqueName: \"kubernetes.io/projected/4e58f98c-97c4-4bfa-b934-56414dfd53c7-kube-api-access-nc2kh\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:20:23.005572 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:23.005573    5510 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4e58f98c-97c4-4bfa-b934-56414dfd53c7-maas-api-tls\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:20:23.005855 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:23.005590    5510 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4e58f98c-97c4-4bfa-b934-56414dfd53c7-metrics-certs\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:20:23.785218 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:23.785187    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ac5024ad-6dcc474c55-dh4j9"
Oct 30 22:20:23.860504 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:23.860413    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:20:23.984992 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:23.984959    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-ac5024ad-6dcc474c55-dh4j9"]
Oct 30 22:20:24.031993 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:24.031949    5510 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-ac5024ad-6dcc474c55-dh4j9"]
Oct 30 22:20:24.137937 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:24.137841    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:20:24.465249 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:24.465162    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:20:24.650155 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:24.650124    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w5-3cc1b3-58b54f64ff-q8k4w"
Oct 30 22:20:25.555306 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:25.555274    5510 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4e58f98c-97c4-4bfa-b934-56414dfd53c7" path="/var/lib/kubelet/pods/4e58f98c-97c4-4bfa-b934-56414dfd53c7/volumes"
Oct 30 22:20:25.656546 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:25.656520    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w5-3cc1b3-f5bd94dd5-pkc6c"
Oct 30 22:20:25.944774 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:25.944692    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:20:26.179679 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:26.178808    5510 patch_prober.go:29] interesting pod/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-m2c8l container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 22:20:26.179679 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:26.178883    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-m2c8l" podUID="4798995a-2305-4dae-adf7-45aa66e0cb18" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 22:20:26.746789 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:26.746758    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-q6kh7"
Oct 30 22:20:26.774611 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:26.774575    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:20:26.795256 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:26.795236    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-k5dpm"
Oct 30 22:20:26.801047 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:26.801026    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-m2c8l"
Oct 30 22:20:26.812012 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:26.811988    5510 generic.go:356] "Generic (PLEG): container finished" podID="8724086f-a5f7-4c63-87da-cb9228b03a0c" containerID="c8299b28bfa098ae0dbbb915292d8ab452f1ad709248e8fc8ea6090ab39f5a99" exitCode=0
Oct 30 22:20:26.812117 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:26.812043    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-k5dpm"
Oct 30 22:20:26.812117 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:26.812069    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-k5dpm" event={"ID":"8724086f-a5f7-4c63-87da-cb9228b03a0c","Type":"ContainerDied","Data":"c8299b28bfa098ae0dbbb915292d8ab452f1ad709248e8fc8ea6090ab39f5a99"}
Oct 30 22:20:26.812227 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:26.812119    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-k5dpm" event={"ID":"8724086f-a5f7-4c63-87da-cb9228b03a0c","Type":"ContainerDied","Data":"2052ea746aa6d4c54ce3e90f3f2b33773210d397996be5f9410ef944da3de8b1"}
Oct 30 22:20:26.812227 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:26.812147    5510 scope.go:122] "RemoveContainer" containerID="c8299b28bfa098ae0dbbb915292d8ab452f1ad709248e8fc8ea6090ab39f5a99"
Oct 30 22:20:26.813556 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:26.813535    5510 generic.go:356] "Generic (PLEG): container finished" podID="4798995a-2305-4dae-adf7-45aa66e0cb18" containerID="23d5aae41a6a6b3345ee01975c2b297980a9c3b6129f27ad2156d594c524e4cd" exitCode=0
Oct 30 22:20:26.813636 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:26.813578    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-m2c8l"
Oct 30 22:20:26.813636 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:26.813619    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-m2c8l" event={"ID":"4798995a-2305-4dae-adf7-45aa66e0cb18","Type":"ContainerDied","Data":"23d5aae41a6a6b3345ee01975c2b297980a9c3b6129f27ad2156d594c524e4cd"}
Oct 30 22:20:26.813768 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:26.813675    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-m2c8l" event={"ID":"4798995a-2305-4dae-adf7-45aa66e0cb18","Type":"ContainerDied","Data":"dd4108f2006d0df98cdd4f005e4ac484549b45a0e4a191de3895412e1047e59e"}
Oct 30 22:20:26.815103 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:26.815084    5510 generic.go:356] "Generic (PLEG): container finished" podID="3c82a381-05ad-436f-89dd-0985c3dbad33" containerID="958d8521ac74ff55fa78bffb7e2167e3fe59f664dd1035b5ed908a7a39c28c13" exitCode=0
Oct 30 22:20:26.815180 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:26.815126    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-q6kh7"
Oct 30 22:20:26.815180 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:26.815147    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-q6kh7" event={"ID":"3c82a381-05ad-436f-89dd-0985c3dbad33","Type":"ContainerDied","Data":"958d8521ac74ff55fa78bffb7e2167e3fe59f664dd1035b5ed908a7a39c28c13"}
Oct 30 22:20:26.815265 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:26.815183    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-q6kh7" event={"ID":"3c82a381-05ad-436f-89dd-0985c3dbad33","Type":"ContainerDied","Data":"8d87a7d2790df605e880d86f67b79d041e72c178bc2fbf2ce2eb7ba50b57e0aa"}
Oct 30 22:20:26.822532 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:26.822509    5510 scope.go:122] "RemoveContainer" containerID="c8299b28bfa098ae0dbbb915292d8ab452f1ad709248e8fc8ea6090ab39f5a99"
Oct 30 22:20:26.822869 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:20:26.822835    5510 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c8299b28bfa098ae0dbbb915292d8ab452f1ad709248e8fc8ea6090ab39f5a99\": container with ID starting with c8299b28bfa098ae0dbbb915292d8ab452f1ad709248e8fc8ea6090ab39f5a99 not found: ID does not exist" containerID="c8299b28bfa098ae0dbbb915292d8ab452f1ad709248e8fc8ea6090ab39f5a99"
Oct 30 22:20:26.822935 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:26.822875    5510 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c8299b28bfa098ae0dbbb915292d8ab452f1ad709248e8fc8ea6090ab39f5a99"} err="failed to get container status \"c8299b28bfa098ae0dbbb915292d8ab452f1ad709248e8fc8ea6090ab39f5a99\": rpc error: code = NotFound desc = could not find container \"c8299b28bfa098ae0dbbb915292d8ab452f1ad709248e8fc8ea6090ab39f5a99\": container with ID starting with c8299b28bfa098ae0dbbb915292d8ab452f1ad709248e8fc8ea6090ab39f5a99 not found: ID does not exist"
Oct 30 22:20:26.822935 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:26.822892    5510 scope.go:122] "RemoveContainer" containerID="23d5aae41a6a6b3345ee01975c2b297980a9c3b6129f27ad2156d594c524e4cd"
Oct 30 22:20:26.832780 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:26.832764    5510 scope.go:122] "RemoveContainer" containerID="23d5aae41a6a6b3345ee01975c2b297980a9c3b6129f27ad2156d594c524e4cd"
Oct 30 22:20:26.833050 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:20:26.833027    5510 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"23d5aae41a6a6b3345ee01975c2b297980a9c3b6129f27ad2156d594c524e4cd\": container with ID starting with 23d5aae41a6a6b3345ee01975c2b297980a9c3b6129f27ad2156d594c524e4cd not found: ID does not exist" containerID="23d5aae41a6a6b3345ee01975c2b297980a9c3b6129f27ad2156d594c524e4cd"
Oct 30 22:20:26.833124 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:26.833062    5510 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"23d5aae41a6a6b3345ee01975c2b297980a9c3b6129f27ad2156d594c524e4cd"} err="failed to get container status \"23d5aae41a6a6b3345ee01975c2b297980a9c3b6129f27ad2156d594c524e4cd\": rpc error: code = NotFound desc = could not find container \"23d5aae41a6a6b3345ee01975c2b297980a9c3b6129f27ad2156d594c524e4cd\": container with ID starting with 23d5aae41a6a6b3345ee01975c2b297980a9c3b6129f27ad2156d594c524e4cd not found: ID does not exist"
Oct 30 22:20:26.833124 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:26.833084    5510 scope.go:122] "RemoveContainer" containerID="958d8521ac74ff55fa78bffb7e2167e3fe59f664dd1035b5ed908a7a39c28c13"
Oct 30 22:20:26.842527 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:26.842503    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3c82a381-05ad-436f-89dd-0985c3dbad33-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3c82a381-05ad-436f-89dd-0985c3dbad33-istio-envoy\") pod \"3c82a381-05ad-436f-89dd-0985c3dbad33\" (UID: \"3c82a381-05ad-436f-89dd-0985c3dbad33\") "
Oct 30 22:20:26.842616 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:26.842531    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3c82a381-05ad-436f-89dd-0985c3dbad33-kube-api-access-zl2qk\" (UniqueName: \"kubernetes.io/projected/3c82a381-05ad-436f-89dd-0985c3dbad33-kube-api-access-zl2qk\") pod \"3c82a381-05ad-436f-89dd-0985c3dbad33\" (UID: \"3c82a381-05ad-436f-89dd-0985c3dbad33\") "
Oct 30 22:20:26.842616 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:26.842595    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3c82a381-05ad-436f-89dd-0985c3dbad33-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3c82a381-05ad-436f-89dd-0985c3dbad33-workload-certs\") pod \"3c82a381-05ad-436f-89dd-0985c3dbad33\" (UID: \"3c82a381-05ad-436f-89dd-0985c3dbad33\") "
Oct 30 22:20:26.842921 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:26.842632    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3c82a381-05ad-436f-89dd-0985c3dbad33-istio-token\" (UniqueName: \"kubernetes.io/projected/3c82a381-05ad-436f-89dd-0985c3dbad33-istio-token\") pod \"3c82a381-05ad-436f-89dd-0985c3dbad33\" (UID: \"3c82a381-05ad-436f-89dd-0985c3dbad33\") "
Oct 30 22:20:26.842921 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:26.842651    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3c82a381-05ad-436f-89dd-0985c3dbad33-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3c82a381-05ad-436f-89dd-0985c3dbad33-istiod-ca-cert\") pod \"3c82a381-05ad-436f-89dd-0985c3dbad33\" (UID: \"3c82a381-05ad-436f-89dd-0985c3dbad33\") "
Oct 30 22:20:26.842921 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:26.842801    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3c82a381-05ad-436f-89dd-0985c3dbad33-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3c82a381-05ad-436f-89dd-0985c3dbad33-istio-data\") pod \"3c82a381-05ad-436f-89dd-0985c3dbad33\" (UID: \"3c82a381-05ad-436f-89dd-0985c3dbad33\") "
Oct 30 22:20:26.842921 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:26.842806    5510 scope.go:122] "RemoveContainer" containerID="958d8521ac74ff55fa78bffb7e2167e3fe59f664dd1035b5ed908a7a39c28c13"
Oct 30 22:20:26.842921 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:26.842841    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3c82a381-05ad-436f-89dd-0985c3dbad33-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c82a381-05ad-436f-89dd-0985c3dbad33-workload-socket\") pod \"3c82a381-05ad-436f-89dd-0985c3dbad33\" (UID: \"3c82a381-05ad-436f-89dd-0985c3dbad33\") "
Oct 30 22:20:26.843183 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:26.842928    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3c82a381-05ad-436f-89dd-0985c3dbad33-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c82a381-05ad-436f-89dd-0985c3dbad33-credential-socket\") pod \"3c82a381-05ad-436f-89dd-0985c3dbad33\" (UID: \"3c82a381-05ad-436f-89dd-0985c3dbad33\") "
Oct 30 22:20:26.843183 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:26.842959    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c82a381-05ad-436f-89dd-0985c3dbad33-workload-certs" pod "3c82a381-05ad-436f-89dd-0985c3dbad33" (UID: "3c82a381-05ad-436f-89dd-0985c3dbad33"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:20:26.843183 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:26.843061    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/3c82a381-05ad-436f-89dd-0985c3dbad33-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3c82a381-05ad-436f-89dd-0985c3dbad33-istio-podinfo\") pod \"3c82a381-05ad-436f-89dd-0985c3dbad33\" (UID: \"3c82a381-05ad-436f-89dd-0985c3dbad33\") "
Oct 30 22:20:26.843354 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:26.843201    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c82a381-05ad-436f-89dd-0985c3dbad33-workload-socket" pod "3c82a381-05ad-436f-89dd-0985c3dbad33" (UID: "3c82a381-05ad-436f-89dd-0985c3dbad33"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:20:26.843354 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:26.843284    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3c82a381-05ad-436f-89dd-0985c3dbad33-istiod-ca-cert" pod "3c82a381-05ad-436f-89dd-0985c3dbad33" (UID: "3c82a381-05ad-436f-89dd-0985c3dbad33"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:20:26.843465 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:26.843409    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c82a381-05ad-436f-89dd-0985c3dbad33-credential-socket" pod "3c82a381-05ad-436f-89dd-0985c3dbad33" (UID: "3c82a381-05ad-436f-89dd-0985c3dbad33"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:20:26.843591 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:20:26.843563    5510 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"958d8521ac74ff55fa78bffb7e2167e3fe59f664dd1035b5ed908a7a39c28c13\": container with ID starting with 958d8521ac74ff55fa78bffb7e2167e3fe59f664dd1035b5ed908a7a39c28c13 not found: ID does not exist" containerID="958d8521ac74ff55fa78bffb7e2167e3fe59f664dd1035b5ed908a7a39c28c13"
Oct 30 22:20:26.843739 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:26.843598    5510 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"958d8521ac74ff55fa78bffb7e2167e3fe59f664dd1035b5ed908a7a39c28c13"} err="failed to get container status \"958d8521ac74ff55fa78bffb7e2167e3fe59f664dd1035b5ed908a7a39c28c13\": rpc error: code = NotFound desc = could not find container \"958d8521ac74ff55fa78bffb7e2167e3fe59f664dd1035b5ed908a7a39c28c13\": container with ID starting with 958d8521ac74ff55fa78bffb7e2167e3fe59f664dd1035b5ed908a7a39c28c13 not found: ID does not exist"
Oct 30 22:20:26.843739 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:26.843641    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c82a381-05ad-436f-89dd-0985c3dbad33-istio-data" pod "3c82a381-05ad-436f-89dd-0985c3dbad33" (UID: "3c82a381-05ad-436f-89dd-0985c3dbad33"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:20:26.844074 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:26.844053    5510 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3c82a381-05ad-436f-89dd-0985c3dbad33-istio-data\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:20:26.844171 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:26.844080    5510 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c82a381-05ad-436f-89dd-0985c3dbad33-workload-socket\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:20:26.844171 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:26.844096    5510 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c82a381-05ad-436f-89dd-0985c3dbad33-credential-socket\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:20:26.844171 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:26.844111    5510 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3c82a381-05ad-436f-89dd-0985c3dbad33-workload-certs\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:20:26.844171 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:26.844125    5510 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3c82a381-05ad-436f-89dd-0985c3dbad33-istiod-ca-cert\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:20:26.844903 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:26.844878    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c82a381-05ad-436f-89dd-0985c3dbad33-istio-envoy" pod "3c82a381-05ad-436f-89dd-0985c3dbad33" (UID: "3c82a381-05ad-436f-89dd-0985c3dbad33"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:20:26.844998 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:26.844911    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3c82a381-05ad-436f-89dd-0985c3dbad33-kube-api-access-zl2qk" pod "3c82a381-05ad-436f-89dd-0985c3dbad33" (UID: "3c82a381-05ad-436f-89dd-0985c3dbad33"). InnerVolumeSpecName "kube-api-access-zl2qk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:20:26.845264 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:26.845247    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3c82a381-05ad-436f-89dd-0985c3dbad33-istio-token" pod "3c82a381-05ad-436f-89dd-0985c3dbad33" (UID: "3c82a381-05ad-436f-89dd-0985c3dbad33"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:20:26.845547 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:26.845530    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3c82a381-05ad-436f-89dd-0985c3dbad33-istio-podinfo" pod "3c82a381-05ad-436f-89dd-0985c3dbad33" (UID: "3c82a381-05ad-436f-89dd-0985c3dbad33"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:20:26.944847 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:26.944760    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8724086f-a5f7-4c63-87da-cb9228b03a0c-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8724086f-a5f7-4c63-87da-cb9228b03a0c-istio-envoy\") pod \"8724086f-a5f7-4c63-87da-cb9228b03a0c\" (UID: \"8724086f-a5f7-4c63-87da-cb9228b03a0c\") "
Oct 30 22:20:26.944847 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:26.944815    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4798995a-2305-4dae-adf7-45aa66e0cb18-istio-token\" (UniqueName: \"kubernetes.io/projected/4798995a-2305-4dae-adf7-45aa66e0cb18-istio-token\") pod \"4798995a-2305-4dae-adf7-45aa66e0cb18\" (UID: \"4798995a-2305-4dae-adf7-45aa66e0cb18\") "
Oct 30 22:20:26.944847 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:26.944837    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4798995a-2305-4dae-adf7-45aa66e0cb18-kube-api-access-g5np9\" (UniqueName: \"kubernetes.io/projected/4798995a-2305-4dae-adf7-45aa66e0cb18-kube-api-access-g5np9\") pod \"4798995a-2305-4dae-adf7-45aa66e0cb18\" (UID: \"4798995a-2305-4dae-adf7-45aa66e0cb18\") "
Oct 30 22:20:26.945139 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:26.944999    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8724086f-a5f7-4c63-87da-cb9228b03a0c-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8724086f-a5f7-4c63-87da-cb9228b03a0c-credential-socket\") pod \"8724086f-a5f7-4c63-87da-cb9228b03a0c\" (UID: \"8724086f-a5f7-4c63-87da-cb9228b03a0c\") "
Oct 30 22:20:26.945139 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:26.945058    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4798995a-2305-4dae-adf7-45aa66e0cb18-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4798995a-2305-4dae-adf7-45aa66e0cb18-istio-envoy\") pod \"4798995a-2305-4dae-adf7-45aa66e0cb18\" (UID: \"4798995a-2305-4dae-adf7-45aa66e0cb18\") "
Oct 30 22:20:26.945139 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:26.945100    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/8724086f-a5f7-4c63-87da-cb9228b03a0c-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8724086f-a5f7-4c63-87da-cb9228b03a0c-istio-podinfo\") pod \"8724086f-a5f7-4c63-87da-cb9228b03a0c\" (UID: \"8724086f-a5f7-4c63-87da-cb9228b03a0c\") "
Oct 30 22:20:26.945139 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:26.945129    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4798995a-2305-4dae-adf7-45aa66e0cb18-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4798995a-2305-4dae-adf7-45aa66e0cb18-credential-socket\") pod \"4798995a-2305-4dae-adf7-45aa66e0cb18\" (UID: \"4798995a-2305-4dae-adf7-45aa66e0cb18\") "
Oct 30 22:20:26.945411 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:26.945245    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8724086f-a5f7-4c63-87da-cb9228b03a0c-kube-api-access-jz85b\" (UniqueName: \"kubernetes.io/projected/8724086f-a5f7-4c63-87da-cb9228b03a0c-kube-api-access-jz85b\") pod \"8724086f-a5f7-4c63-87da-cb9228b03a0c\" (UID: \"8724086f-a5f7-4c63-87da-cb9228b03a0c\") "
Oct 30 22:20:26.945411 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:26.945292    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8724086f-a5f7-4c63-87da-cb9228b03a0c-credential-socket" pod "8724086f-a5f7-4c63-87da-cb9228b03a0c" (UID: "8724086f-a5f7-4c63-87da-cb9228b03a0c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:20:26.945411 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:26.945337    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8724086f-a5f7-4c63-87da-cb9228b03a0c-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8724086f-a5f7-4c63-87da-cb9228b03a0c-workload-socket\") pod \"8724086f-a5f7-4c63-87da-cb9228b03a0c\" (UID: \"8724086f-a5f7-4c63-87da-cb9228b03a0c\") "
Oct 30 22:20:26.945411 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:26.945383    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8724086f-a5f7-4c63-87da-cb9228b03a0c-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8724086f-a5f7-4c63-87da-cb9228b03a0c-istio-data\") pod \"8724086f-a5f7-4c63-87da-cb9228b03a0c\" (UID: \"8724086f-a5f7-4c63-87da-cb9228b03a0c\") "
Oct 30 22:20:26.945620 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:26.945431    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4798995a-2305-4dae-adf7-45aa66e0cb18-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4798995a-2305-4dae-adf7-45aa66e0cb18-workload-certs\") pod \"4798995a-2305-4dae-adf7-45aa66e0cb18\" (UID: \"4798995a-2305-4dae-adf7-45aa66e0cb18\") "
Oct 30 22:20:26.945620 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:26.945475    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8724086f-a5f7-4c63-87da-cb9228b03a0c-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8724086f-a5f7-4c63-87da-cb9228b03a0c-workload-certs\") pod \"8724086f-a5f7-4c63-87da-cb9228b03a0c\" (UID: \"8724086f-a5f7-4c63-87da-cb9228b03a0c\") "
Oct 30 22:20:26.945620 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:26.945519    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4798995a-2305-4dae-adf7-45aa66e0cb18-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4798995a-2305-4dae-adf7-45aa66e0cb18-istio-data\") pod \"4798995a-2305-4dae-adf7-45aa66e0cb18\" (UID: \"4798995a-2305-4dae-adf7-45aa66e0cb18\") "
Oct 30 22:20:26.945620 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:26.945573    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4798995a-2305-4dae-adf7-45aa66e0cb18-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4798995a-2305-4dae-adf7-45aa66e0cb18-workload-socket\") pod \"4798995a-2305-4dae-adf7-45aa66e0cb18\" (UID: \"4798995a-2305-4dae-adf7-45aa66e0cb18\") "
Oct 30 22:20:26.945880 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:26.945720    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8724086f-a5f7-4c63-87da-cb9228b03a0c-istio-token\" (UniqueName: \"kubernetes.io/projected/8724086f-a5f7-4c63-87da-cb9228b03a0c-istio-token\") pod \"8724086f-a5f7-4c63-87da-cb9228b03a0c\" (UID: \"8724086f-a5f7-4c63-87da-cb9228b03a0c\") "
Oct 30 22:20:26.945880 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:26.945733    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4798995a-2305-4dae-adf7-45aa66e0cb18-workload-certs" pod "4798995a-2305-4dae-adf7-45aa66e0cb18" (UID: "4798995a-2305-4dae-adf7-45aa66e0cb18"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:20:26.945880 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:26.945788    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4798995a-2305-4dae-adf7-45aa66e0cb18-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4798995a-2305-4dae-adf7-45aa66e0cb18-istiod-ca-cert\") pod \"4798995a-2305-4dae-adf7-45aa66e0cb18\" (UID: \"4798995a-2305-4dae-adf7-45aa66e0cb18\") "
Oct 30 22:20:26.945880 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:26.945824    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8724086f-a5f7-4c63-87da-cb9228b03a0c-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8724086f-a5f7-4c63-87da-cb9228b03a0c-istiod-ca-cert\") pod \"8724086f-a5f7-4c63-87da-cb9228b03a0c\" (UID: \"8724086f-a5f7-4c63-87da-cb9228b03a0c\") "
Oct 30 22:20:26.946094 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:26.945904    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/4798995a-2305-4dae-adf7-45aa66e0cb18-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4798995a-2305-4dae-adf7-45aa66e0cb18-istio-podinfo\") pod \"4798995a-2305-4dae-adf7-45aa66e0cb18\" (UID: \"4798995a-2305-4dae-adf7-45aa66e0cb18\") "
Oct 30 22:20:26.946094 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:26.946048    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4798995a-2305-4dae-adf7-45aa66e0cb18-workload-socket" pod "4798995a-2305-4dae-adf7-45aa66e0cb18" (UID: "4798995a-2305-4dae-adf7-45aa66e0cb18"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:20:26.946212 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:26.946141    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4798995a-2305-4dae-adf7-45aa66e0cb18-credential-socket" pod "4798995a-2305-4dae-adf7-45aa66e0cb18" (UID: "4798995a-2305-4dae-adf7-45aa66e0cb18"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:20:26.946535 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:26.946380    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4798995a-2305-4dae-adf7-45aa66e0cb18-istiod-ca-cert" pod "4798995a-2305-4dae-adf7-45aa66e0cb18" (UID: "4798995a-2305-4dae-adf7-45aa66e0cb18"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:20:26.946535 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:26.946481    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4798995a-2305-4dae-adf7-45aa66e0cb18-istio-data" pod "4798995a-2305-4dae-adf7-45aa66e0cb18" (UID: "4798995a-2305-4dae-adf7-45aa66e0cb18"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:20:26.946763 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:26.946738    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8724086f-a5f7-4c63-87da-cb9228b03a0c-workload-certs" pod "8724086f-a5f7-4c63-87da-cb9228b03a0c" (UID: "8724086f-a5f7-4c63-87da-cb9228b03a0c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:20:26.946951 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:26.945639    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8724086f-a5f7-4c63-87da-cb9228b03a0c-workload-socket" pod "8724086f-a5f7-4c63-87da-cb9228b03a0c" (UID: "8724086f-a5f7-4c63-87da-cb9228b03a0c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:20:26.947032 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:26.946977    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8724086f-a5f7-4c63-87da-cb9228b03a0c-istio-data" pod "8724086f-a5f7-4c63-87da-cb9228b03a0c" (UID: "8724086f-a5f7-4c63-87da-cb9228b03a0c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:20:26.947092 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:26.947045    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8724086f-a5f7-4c63-87da-cb9228b03a0c-istiod-ca-cert" pod "8724086f-a5f7-4c63-87da-cb9228b03a0c" (UID: "8724086f-a5f7-4c63-87da-cb9228b03a0c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:20:26.947154 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:26.947124    5510 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8724086f-a5f7-4c63-87da-cb9228b03a0c-credential-socket\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:20:26.947154 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:26.947146    5510 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4798995a-2305-4dae-adf7-45aa66e0cb18-credential-socket\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:20:26.947268 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:26.947162    5510 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3c82a381-05ad-436f-89dd-0985c3dbad33-istio-podinfo\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:20:26.947268 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:26.947176    5510 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4798995a-2305-4dae-adf7-45aa66e0cb18-workload-certs\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:20:26.947268 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:26.947189    5510 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8724086f-a5f7-4c63-87da-cb9228b03a0c-workload-certs\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:20:26.947268 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:26.947203    5510 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4798995a-2305-4dae-adf7-45aa66e0cb18-istio-data\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:20:26.947268 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:26.947218    5510 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4798995a-2305-4dae-adf7-45aa66e0cb18-workload-socket\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:20:26.947268 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:26.947231    5510 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3c82a381-05ad-436f-89dd-0985c3dbad33-istio-envoy\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:20:26.947268 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:26.947245    5510 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zl2qk\" (UniqueName: \"kubernetes.io/projected/3c82a381-05ad-436f-89dd-0985c3dbad33-kube-api-access-zl2qk\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:20:26.947268 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:26.947260    5510 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4798995a-2305-4dae-adf7-45aa66e0cb18-istiod-ca-cert\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:20:26.947780 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:26.947275    5510 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3c82a381-05ad-436f-89dd-0985c3dbad33-istio-token\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:20:26.947865 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:26.947823    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8724086f-a5f7-4c63-87da-cb9228b03a0c-kube-api-access-jz85b" pod "8724086f-a5f7-4c63-87da-cb9228b03a0c" (UID: "8724086f-a5f7-4c63-87da-cb9228b03a0c"). InnerVolumeSpecName "kube-api-access-jz85b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:20:26.947972 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:26.947950    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8724086f-a5f7-4c63-87da-cb9228b03a0c-istio-envoy" pod "8724086f-a5f7-4c63-87da-cb9228b03a0c" (UID: "8724086f-a5f7-4c63-87da-cb9228b03a0c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:20:26.948086 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:26.948063    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4798995a-2305-4dae-adf7-45aa66e0cb18-kube-api-access-g5np9" pod "4798995a-2305-4dae-adf7-45aa66e0cb18" (UID: "4798995a-2305-4dae-adf7-45aa66e0cb18"). InnerVolumeSpecName "kube-api-access-g5np9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:20:26.948329 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:26.948305    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8724086f-a5f7-4c63-87da-cb9228b03a0c-istio-podinfo" pod "8724086f-a5f7-4c63-87da-cb9228b03a0c" (UID: "8724086f-a5f7-4c63-87da-cb9228b03a0c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:20:26.948626 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:26.948594    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4798995a-2305-4dae-adf7-45aa66e0cb18-istio-token" pod "4798995a-2305-4dae-adf7-45aa66e0cb18" (UID: "4798995a-2305-4dae-adf7-45aa66e0cb18"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:20:26.948756 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:26.948627    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4798995a-2305-4dae-adf7-45aa66e0cb18-istio-envoy" pod "4798995a-2305-4dae-adf7-45aa66e0cb18" (UID: "4798995a-2305-4dae-adf7-45aa66e0cb18"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:20:26.949127 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:26.949110    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8724086f-a5f7-4c63-87da-cb9228b03a0c-istio-token" pod "8724086f-a5f7-4c63-87da-cb9228b03a0c" (UID: "8724086f-a5f7-4c63-87da-cb9228b03a0c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:20:26.949462 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:26.949449    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4798995a-2305-4dae-adf7-45aa66e0cb18-istio-podinfo" pod "4798995a-2305-4dae-adf7-45aa66e0cb18" (UID: "4798995a-2305-4dae-adf7-45aa66e0cb18"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:20:27.049835 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:27.049803    5510 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8724086f-a5f7-4c63-87da-cb9228b03a0c-istio-token\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:20:27.049835 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:27.049835    5510 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8724086f-a5f7-4c63-87da-cb9228b03a0c-istiod-ca-cert\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:20:27.050009 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:27.049850    5510 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4798995a-2305-4dae-adf7-45aa66e0cb18-istio-podinfo\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:20:27.050009 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:27.049866    5510 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8724086f-a5f7-4c63-87da-cb9228b03a0c-istio-envoy\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:20:27.050009 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:27.049880    5510 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4798995a-2305-4dae-adf7-45aa66e0cb18-istio-token\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:20:27.050009 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:27.049896    5510 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g5np9\" (UniqueName: \"kubernetes.io/projected/4798995a-2305-4dae-adf7-45aa66e0cb18-kube-api-access-g5np9\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:20:27.050009 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:27.049910    5510 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4798995a-2305-4dae-adf7-45aa66e0cb18-istio-envoy\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:20:27.050009 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:27.049923    5510 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8724086f-a5f7-4c63-87da-cb9228b03a0c-istio-podinfo\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:20:27.050009 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:27.049937    5510 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jz85b\" (UniqueName: \"kubernetes.io/projected/8724086f-a5f7-4c63-87da-cb9228b03a0c-kube-api-access-jz85b\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:20:27.050009 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:27.049952    5510 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8724086f-a5f7-4c63-87da-cb9228b03a0c-workload-socket\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:20:27.050009 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:27.049964    5510 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8724086f-a5f7-4c63-87da-cb9228b03a0c-istio-data\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:20:27.328679 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:27.322956    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-q6kh7"]
Oct 30 22:20:27.373681 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:27.368206    5510 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-q6kh7"]
Oct 30 22:20:27.425418 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:27.425354    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-k5dpm"]
Oct 30 22:20:27.461361 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:27.461323    5510 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-k5dpm"]
Oct 30 22:20:27.596140 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:27.596045    5510 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3c82a381-05ad-436f-89dd-0985c3dbad33" path="/var/lib/kubelet/pods/3c82a381-05ad-436f-89dd-0985c3dbad33/volumes"
Oct 30 22:20:27.597181 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:27.597158    5510 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8724086f-a5f7-4c63-87da-cb9228b03a0c" path="/var/lib/kubelet/pods/8724086f-a5f7-4c63-87da-cb9228b03a0c/volumes"
Oct 30 22:20:27.597749 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:27.597730    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-m2c8l"]
Oct 30 22:20:27.613779 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:27.613737    5510 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-m2c8l"]
Oct 30 22:20:27.623070 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:27.621214    5510 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w1-a6b6a5-openshift-default-f7cd49795-zlkqg"]
Oct 30 22:20:27.623070 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:27.622393    5510 cpu_manager.go:414] "containerMap: removing container" podUID="3c82a381-05ad-436f-89dd-0985c3dbad33" containerName="istio-proxy"
Oct 30 22:20:27.623070 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:27.622417    5510 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3c82a381-05ad-436f-89dd-0985c3dbad33" containerName="istio-proxy"
Oct 30 22:20:27.623070 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:27.622430    5510 cpu_manager.go:414] "containerMap: removing container" podUID="4e58f98c-97c4-4bfa-b934-56414dfd53c7" containerName="maas-api"
Oct 30 22:20:27.623070 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:27.622438    5510 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4e58f98c-97c4-4bfa-b934-56414dfd53c7" containerName="maas-api"
Oct 30 22:20:27.623070 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:27.622491    5510 cpu_manager.go:414] "containerMap: removing container" podUID="8724086f-a5f7-4c63-87da-cb9228b03a0c" containerName="istio-proxy"
Oct 30 22:20:27.623070 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:27.622499    5510 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8724086f-a5f7-4c63-87da-cb9228b03a0c" containerName="istio-proxy"
Oct 30 22:20:27.623070 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:27.622524    5510 cpu_manager.go:414] "containerMap: removing container" podUID="4798995a-2305-4dae-adf7-45aa66e0cb18" containerName="istio-proxy"
Oct 30 22:20:27.623070 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:27.622531    5510 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4798995a-2305-4dae-adf7-45aa66e0cb18" containerName="istio-proxy"
Oct 30 22:20:27.623070 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:27.622710    5510 memory_manager.go:365] "RemoveStaleState removing state" podUID="4798995a-2305-4dae-adf7-45aa66e0cb18" containerName="istio-proxy"
Oct 30 22:20:27.623070 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:27.622738    5510 memory_manager.go:365] "RemoveStaleState removing state" podUID="3c82a381-05ad-436f-89dd-0985c3dbad33" containerName="istio-proxy"
Oct 30 22:20:27.623070 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:27.622747    5510 memory_manager.go:365] "RemoveStaleState removing state" podUID="4e58f98c-97c4-4bfa-b934-56414dfd53c7" containerName="maas-api"
Oct 30 22:20:27.623070 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:27.622764    5510 memory_manager.go:365] "RemoveStaleState removing state" podUID="8724086f-a5f7-4c63-87da-cb9228b03a0c" containerName="istio-proxy"
Oct 30 22:20:27.637605 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:27.634878    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w1-a6b6a5-openshift-default-f7cd49795-zlkqg"]
Oct 30 22:20:27.637605 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:27.635005    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w1-a6b6a5-openshift-default-f7cd49795-zlkqg"
Oct 30 22:20:27.685164 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:27.685132    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w1-a6b6a5-openshift-default-dockercfg-msb98\""
Oct 30 22:20:27.703100 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:27.703064    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/61fdf6bd-8922-4db6-a4cf-4fa9ea967b20-workload-socket\") pod \"e2e-shared-a-w1-a6b6a5-openshift-default-f7cd49795-zlkqg\" (UID: \"61fdf6bd-8922-4db6-a4cf-4fa9ea967b20\") " pod="openshift-ingress/e2e-shared-a-w1-a6b6a5-openshift-default-f7cd49795-zlkqg"
Oct 30 22:20:27.703290 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:27.703264    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/61fdf6bd-8922-4db6-a4cf-4fa9ea967b20-istiod-ca-cert\") pod \"e2e-shared-a-w1-a6b6a5-openshift-default-f7cd49795-zlkqg\" (UID: \"61fdf6bd-8922-4db6-a4cf-4fa9ea967b20\") " pod="openshift-ingress/e2e-shared-a-w1-a6b6a5-openshift-default-f7cd49795-zlkqg"
Oct 30 22:20:27.703402 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:27.703377    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/61fdf6bd-8922-4db6-a4cf-4fa9ea967b20-istio-podinfo\") pod \"e2e-shared-a-w1-a6b6a5-openshift-default-f7cd49795-zlkqg\" (UID: \"61fdf6bd-8922-4db6-a4cf-4fa9ea967b20\") " pod="openshift-ingress/e2e-shared-a-w1-a6b6a5-openshift-default-f7cd49795-zlkqg"
Oct 30 22:20:27.703532 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:27.703436    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/61fdf6bd-8922-4db6-a4cf-4fa9ea967b20-istio-data\") pod \"e2e-shared-a-w1-a6b6a5-openshift-default-f7cd49795-zlkqg\" (UID: \"61fdf6bd-8922-4db6-a4cf-4fa9ea967b20\") " pod="openshift-ingress/e2e-shared-a-w1-a6b6a5-openshift-default-f7cd49795-zlkqg"
Oct 30 22:20:27.703598 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:27.703537    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/61fdf6bd-8922-4db6-a4cf-4fa9ea967b20-istio-token\") pod \"e2e-shared-a-w1-a6b6a5-openshift-default-f7cd49795-zlkqg\" (UID: \"61fdf6bd-8922-4db6-a4cf-4fa9ea967b20\") " pod="openshift-ingress/e2e-shared-a-w1-a6b6a5-openshift-default-f7cd49795-zlkqg"
Oct 30 22:20:27.703598 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:27.703589    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/61fdf6bd-8922-4db6-a4cf-4fa9ea967b20-credential-socket\") pod \"e2e-shared-a-w1-a6b6a5-openshift-default-f7cd49795-zlkqg\" (UID: \"61fdf6bd-8922-4db6-a4cf-4fa9ea967b20\") " pod="openshift-ingress/e2e-shared-a-w1-a6b6a5-openshift-default-f7cd49795-zlkqg"
Oct 30 22:20:27.703748 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:27.703612    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/61fdf6bd-8922-4db6-a4cf-4fa9ea967b20-istio-envoy\") pod \"e2e-shared-a-w1-a6b6a5-openshift-default-f7cd49795-zlkqg\" (UID: \"61fdf6bd-8922-4db6-a4cf-4fa9ea967b20\") " pod="openshift-ingress/e2e-shared-a-w1-a6b6a5-openshift-default-f7cd49795-zlkqg"
Oct 30 22:20:27.703748 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:27.703721    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6d2xc\" (UniqueName: \"kubernetes.io/projected/61fdf6bd-8922-4db6-a4cf-4fa9ea967b20-kube-api-access-6d2xc\") pod \"e2e-shared-a-w1-a6b6a5-openshift-default-f7cd49795-zlkqg\" (UID: \"61fdf6bd-8922-4db6-a4cf-4fa9ea967b20\") " pod="openshift-ingress/e2e-shared-a-w1-a6b6a5-openshift-default-f7cd49795-zlkqg"
Oct 30 22:20:27.703889 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:27.703872    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/61fdf6bd-8922-4db6-a4cf-4fa9ea967b20-workload-certs\") pod \"e2e-shared-a-w1-a6b6a5-openshift-default-f7cd49795-zlkqg\" (UID: \"61fdf6bd-8922-4db6-a4cf-4fa9ea967b20\") " pod="openshift-ingress/e2e-shared-a-w1-a6b6a5-openshift-default-f7cd49795-zlkqg"
Oct 30 22:20:27.804360 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:27.804321    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:20:27.805044 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:27.805023    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/61fdf6bd-8922-4db6-a4cf-4fa9ea967b20-workload-certs\") pod \"e2e-shared-a-w1-a6b6a5-openshift-default-f7cd49795-zlkqg\" (UID: \"61fdf6bd-8922-4db6-a4cf-4fa9ea967b20\") " pod="openshift-ingress/e2e-shared-a-w1-a6b6a5-openshift-default-f7cd49795-zlkqg"
Oct 30 22:20:27.805112 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:27.805100    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/61fdf6bd-8922-4db6-a4cf-4fa9ea967b20-workload-socket\") pod \"e2e-shared-a-w1-a6b6a5-openshift-default-f7cd49795-zlkqg\" (UID: \"61fdf6bd-8922-4db6-a4cf-4fa9ea967b20\") " pod="openshift-ingress/e2e-shared-a-w1-a6b6a5-openshift-default-f7cd49795-zlkqg"
Oct 30 22:20:27.805189 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:27.805178    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/61fdf6bd-8922-4db6-a4cf-4fa9ea967b20-istiod-ca-cert\") pod \"e2e-shared-a-w1-a6b6a5-openshift-default-f7cd49795-zlkqg\" (UID: \"61fdf6bd-8922-4db6-a4cf-4fa9ea967b20\") " pod="openshift-ingress/e2e-shared-a-w1-a6b6a5-openshift-default-f7cd49795-zlkqg"
Oct 30 22:20:27.805239 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:27.805227    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/61fdf6bd-8922-4db6-a4cf-4fa9ea967b20-istio-podinfo\") pod \"e2e-shared-a-w1-a6b6a5-openshift-default-f7cd49795-zlkqg\" (UID: \"61fdf6bd-8922-4db6-a4cf-4fa9ea967b20\") " pod="openshift-ingress/e2e-shared-a-w1-a6b6a5-openshift-default-f7cd49795-zlkqg"
Oct 30 22:20:27.805289 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:27.805250    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/61fdf6bd-8922-4db6-a4cf-4fa9ea967b20-istio-data\") pod \"e2e-shared-a-w1-a6b6a5-openshift-default-f7cd49795-zlkqg\" (UID: \"61fdf6bd-8922-4db6-a4cf-4fa9ea967b20\") " pod="openshift-ingress/e2e-shared-a-w1-a6b6a5-openshift-default-f7cd49795-zlkqg"
Oct 30 22:20:27.805364 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:27.805347    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/61fdf6bd-8922-4db6-a4cf-4fa9ea967b20-istio-token\") pod \"e2e-shared-a-w1-a6b6a5-openshift-default-f7cd49795-zlkqg\" (UID: \"61fdf6bd-8922-4db6-a4cf-4fa9ea967b20\") " pod="openshift-ingress/e2e-shared-a-w1-a6b6a5-openshift-default-f7cd49795-zlkqg"
Oct 30 22:20:27.805430 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:27.805399    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/61fdf6bd-8922-4db6-a4cf-4fa9ea967b20-credential-socket\") pod \"e2e-shared-a-w1-a6b6a5-openshift-default-f7cd49795-zlkqg\" (UID: \"61fdf6bd-8922-4db6-a4cf-4fa9ea967b20\") " pod="openshift-ingress/e2e-shared-a-w1-a6b6a5-openshift-default-f7cd49795-zlkqg"
Oct 30 22:20:27.805485 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:27.805435    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/61fdf6bd-8922-4db6-a4cf-4fa9ea967b20-istio-envoy\") pod \"e2e-shared-a-w1-a6b6a5-openshift-default-f7cd49795-zlkqg\" (UID: \"61fdf6bd-8922-4db6-a4cf-4fa9ea967b20\") " pod="openshift-ingress/e2e-shared-a-w1-a6b6a5-openshift-default-f7cd49795-zlkqg"
Oct 30 22:20:27.805485 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:27.805471    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/61fdf6bd-8922-4db6-a4cf-4fa9ea967b20-workload-certs\") pod \"e2e-shared-a-w1-a6b6a5-openshift-default-f7cd49795-zlkqg\" (UID: \"61fdf6bd-8922-4db6-a4cf-4fa9ea967b20\") " pod="openshift-ingress/e2e-shared-a-w1-a6b6a5-openshift-default-f7cd49795-zlkqg"
Oct 30 22:20:27.805590 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:27.805494    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6d2xc\" (UniqueName: \"kubernetes.io/projected/61fdf6bd-8922-4db6-a4cf-4fa9ea967b20-kube-api-access-6d2xc\") pod \"e2e-shared-a-w1-a6b6a5-openshift-default-f7cd49795-zlkqg\" (UID: \"61fdf6bd-8922-4db6-a4cf-4fa9ea967b20\") " pod="openshift-ingress/e2e-shared-a-w1-a6b6a5-openshift-default-f7cd49795-zlkqg"
Oct 30 22:20:27.805590 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:27.805542    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/61fdf6bd-8922-4db6-a4cf-4fa9ea967b20-workload-socket\") pod \"e2e-shared-a-w1-a6b6a5-openshift-default-f7cd49795-zlkqg\" (UID: \"61fdf6bd-8922-4db6-a4cf-4fa9ea967b20\") " pod="openshift-ingress/e2e-shared-a-w1-a6b6a5-openshift-default-f7cd49795-zlkqg"
Oct 30 22:20:27.805739 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:27.805674    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/61fdf6bd-8922-4db6-a4cf-4fa9ea967b20-istio-data\") pod \"e2e-shared-a-w1-a6b6a5-openshift-default-f7cd49795-zlkqg\" (UID: \"61fdf6bd-8922-4db6-a4cf-4fa9ea967b20\") " pod="openshift-ingress/e2e-shared-a-w1-a6b6a5-openshift-default-f7cd49795-zlkqg"
Oct 30 22:20:27.805813 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:27.805763    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/61fdf6bd-8922-4db6-a4cf-4fa9ea967b20-credential-socket\") pod \"e2e-shared-a-w1-a6b6a5-openshift-default-f7cd49795-zlkqg\" (UID: \"61fdf6bd-8922-4db6-a4cf-4fa9ea967b20\") " pod="openshift-ingress/e2e-shared-a-w1-a6b6a5-openshift-default-f7cd49795-zlkqg"
Oct 30 22:20:27.805958 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:27.805927    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/61fdf6bd-8922-4db6-a4cf-4fa9ea967b20-istiod-ca-cert\") pod \"e2e-shared-a-w1-a6b6a5-openshift-default-f7cd49795-zlkqg\" (UID: \"61fdf6bd-8922-4db6-a4cf-4fa9ea967b20\") " pod="openshift-ingress/e2e-shared-a-w1-a6b6a5-openshift-default-f7cd49795-zlkqg"
Oct 30 22:20:27.807800 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:27.807775    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/61fdf6bd-8922-4db6-a4cf-4fa9ea967b20-istio-envoy\") pod \"e2e-shared-a-w1-a6b6a5-openshift-default-f7cd49795-zlkqg\" (UID: \"61fdf6bd-8922-4db6-a4cf-4fa9ea967b20\") " pod="openshift-ingress/e2e-shared-a-w1-a6b6a5-openshift-default-f7cd49795-zlkqg"
Oct 30 22:20:27.807916 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:27.807884    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/61fdf6bd-8922-4db6-a4cf-4fa9ea967b20-istio-podinfo\") pod \"e2e-shared-a-w1-a6b6a5-openshift-default-f7cd49795-zlkqg\" (UID: \"61fdf6bd-8922-4db6-a4cf-4fa9ea967b20\") " pod="openshift-ingress/e2e-shared-a-w1-a6b6a5-openshift-default-f7cd49795-zlkqg"
Oct 30 22:20:27.873377 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:27.873294    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/61fdf6bd-8922-4db6-a4cf-4fa9ea967b20-istio-token\") pod \"e2e-shared-a-w1-a6b6a5-openshift-default-f7cd49795-zlkqg\" (UID: \"61fdf6bd-8922-4db6-a4cf-4fa9ea967b20\") " pod="openshift-ingress/e2e-shared-a-w1-a6b6a5-openshift-default-f7cd49795-zlkqg"
Oct 30 22:20:27.873809 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:27.873784    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6d2xc\" (UniqueName: \"kubernetes.io/projected/61fdf6bd-8922-4db6-a4cf-4fa9ea967b20-kube-api-access-6d2xc\") pod \"e2e-shared-a-w1-a6b6a5-openshift-default-f7cd49795-zlkqg\" (UID: \"61fdf6bd-8922-4db6-a4cf-4fa9ea967b20\") " pod="openshift-ingress/e2e-shared-a-w1-a6b6a5-openshift-default-f7cd49795-zlkqg"
Oct 30 22:20:27.964680 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:27.960000    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w1-a6b6a5-openshift-default-f7cd49795-zlkqg"
Oct 30 22:20:28.027089 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:28.027054    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:20:28.465474 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:28.465439    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w1-a6b6a5-openshift-default-f7cd49795-zlkqg"]
Oct 30 22:20:28.466130 ip-10-0-136-210 kubenswrapper[5510]: W1030 22:20:28.466103    5510 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod61fdf6bd_8922_4db6_a4cf_4fa9ea967b20.slice/crio-618e747c0d0ca490e2801e15d62e5547e180323deedafed4276ca20cfba1046f WatchSource:0}: Error finding container 618e747c0d0ca490e2801e15d62e5547e180323deedafed4276ca20cfba1046f: Status 404 returned error can't find the container with id 618e747c0d0ca490e2801e15d62e5547e180323deedafed4276ca20cfba1046f
Oct 30 22:20:28.469340 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:28.469301    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:20:28.469428 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:28.469390    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:20:28.469481 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:28.469433    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:20:28.518072 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:28.518039    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:20:28.838740 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:28.838699    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w1-a6b6a5-openshift-default-f7cd49795-zlkqg" event={"ID":"61fdf6bd-8922-4db6-a4cf-4fa9ea967b20","Type":"ContainerStarted","Data":"244da90203c70fbbaf03359fbaf0ae00b6517b196db93176f7390a74b61d45e6"}
Oct 30 22:20:28.839139 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:28.838767    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w1-a6b6a5-openshift-default-f7cd49795-zlkqg" event={"ID":"61fdf6bd-8922-4db6-a4cf-4fa9ea967b20","Type":"ContainerStarted","Data":"618e747c0d0ca490e2801e15d62e5547e180323deedafed4276ca20cfba1046f"}
Oct 30 22:20:28.954080 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:28.954023    5510 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w1-a6b6a5-openshift-default-f7cd49795-zlkqg" podStartSLOduration=1.95400851 podStartE2EDuration="1.95400851s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:20:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:20:28.952352894 +0000 UTC m=+1809.513162920" watchObservedRunningTime="2026-10-30 22:20:28.95400851 +0000 UTC m=+1809.514818516"
Oct 30 22:20:28.961598 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:28.961550    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w1-a6b6a5-openshift-default-f7cd49795-zlkqg"
Oct 30 22:20:29.442680 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:29.435001    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:20:29.568504 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:29.568470    5510 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4798995a-2305-4dae-adf7-45aa66e0cb18" path="/var/lib/kubelet/pods/4798995a-2305-4dae-adf7-45aa66e0cb18/volumes"
Oct 30 22:20:29.967517 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:29.967446    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w1-a6b6a5-openshift-default-f7cd49795-zlkqg"
Oct 30 22:20:30.853294 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:30.853260    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w1-a6b6a5-openshift-default-f7cd49795-zlkqg"
Oct 30 22:20:30.854286 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:30.854263    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w1-a6b6a5-openshift-default-f7cd49795-zlkqg"
Oct 30 22:20:31.805935 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:31.805895    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:20:32.062536 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:32.062441    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:20:33.999064 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:33.999028    5510 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-4da07739-5dcb995df-m8gbm"]
Oct 30 22:20:34.005122 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:34.005101    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4da07739-5dcb995df-m8gbm"
Oct 30 22:20:34.056734 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:34.056698    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-4da07739-serving-cert\""
Oct 30 22:20:34.061867 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:34.061832    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-4da07739-5dcb995df-m8gbm"]
Oct 30 22:20:34.114328 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:34.114292    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2f58edd6-b3f3-4074-af7f-04970eddc81e-openshift-service-ca\") pod \"maas-api-e2e-mt-4da07739-5dcb995df-m8gbm\" (UID: \"2f58edd6-b3f3-4074-af7f-04970eddc81e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4da07739-5dcb995df-m8gbm"
Oct 30 22:20:34.114545 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:34.114366    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2f58edd6-b3f3-4074-af7f-04970eddc81e-maas-api-tls\") pod \"maas-api-e2e-mt-4da07739-5dcb995df-m8gbm\" (UID: \"2f58edd6-b3f3-4074-af7f-04970eddc81e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4da07739-5dcb995df-m8gbm"
Oct 30 22:20:34.114618 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:34.114572    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2f58edd6-b3f3-4074-af7f-04970eddc81e-metrics-certs\") pod \"maas-api-e2e-mt-4da07739-5dcb995df-m8gbm\" (UID: \"2f58edd6-b3f3-4074-af7f-04970eddc81e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4da07739-5dcb995df-m8gbm"
Oct 30 22:20:34.114852 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:34.114826    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8qgf7\" (UniqueName: \"kubernetes.io/projected/2f58edd6-b3f3-4074-af7f-04970eddc81e-kube-api-access-8qgf7\") pod \"maas-api-e2e-mt-4da07739-5dcb995df-m8gbm\" (UID: \"2f58edd6-b3f3-4074-af7f-04970eddc81e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4da07739-5dcb995df-m8gbm"
Oct 30 22:20:34.216833 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:34.216792    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2f58edd6-b3f3-4074-af7f-04970eddc81e-maas-api-tls\") pod \"maas-api-e2e-mt-4da07739-5dcb995df-m8gbm\" (UID: \"2f58edd6-b3f3-4074-af7f-04970eddc81e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4da07739-5dcb995df-m8gbm"
Oct 30 22:20:34.217020 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:34.216921    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2f58edd6-b3f3-4074-af7f-04970eddc81e-metrics-certs\") pod \"maas-api-e2e-mt-4da07739-5dcb995df-m8gbm\" (UID: \"2f58edd6-b3f3-4074-af7f-04970eddc81e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4da07739-5dcb995df-m8gbm"
Oct 30 22:20:34.217020 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:20:34.216934    5510 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-4da07739-serving-cert: secret "maas-api-e2e-mt-4da07739-serving-cert" not found
Oct 30 22:20:34.217020 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:20:34.217001    5510 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2f58edd6-b3f3-4074-af7f-04970eddc81e-maas-api-tls podName:2f58edd6-b3f3-4074-af7f-04970eddc81e nodeName:}" failed. No retries permitted until 2026-10-30 22:20:34.716982072 +0000 UTC m=+1815.277792056 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/2f58edd6-b3f3-4074-af7f-04970eddc81e-maas-api-tls") pod "maas-api-e2e-mt-4da07739-5dcb995df-m8gbm" (UID: "2f58edd6-b3f3-4074-af7f-04970eddc81e") : secret "maas-api-e2e-mt-4da07739-serving-cert" not found
Oct 30 22:20:34.217151 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:34.217123    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8qgf7\" (UniqueName: \"kubernetes.io/projected/2f58edd6-b3f3-4074-af7f-04970eddc81e-kube-api-access-8qgf7\") pod \"maas-api-e2e-mt-4da07739-5dcb995df-m8gbm\" (UID: \"2f58edd6-b3f3-4074-af7f-04970eddc81e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4da07739-5dcb995df-m8gbm"
Oct 30 22:20:34.217252 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:34.217239    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2f58edd6-b3f3-4074-af7f-04970eddc81e-openshift-service-ca\") pod \"maas-api-e2e-mt-4da07739-5dcb995df-m8gbm\" (UID: \"2f58edd6-b3f3-4074-af7f-04970eddc81e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4da07739-5dcb995df-m8gbm"
Oct 30 22:20:34.217938 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:34.217916    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2f58edd6-b3f3-4074-af7f-04970eddc81e-openshift-service-ca\") pod \"maas-api-e2e-mt-4da07739-5dcb995df-m8gbm\" (UID: \"2f58edd6-b3f3-4074-af7f-04970eddc81e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4da07739-5dcb995df-m8gbm"
Oct 30 22:20:34.219586 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:34.219560    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2f58edd6-b3f3-4074-af7f-04970eddc81e-metrics-certs\") pod \"maas-api-e2e-mt-4da07739-5dcb995df-m8gbm\" (UID: \"2f58edd6-b3f3-4074-af7f-04970eddc81e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4da07739-5dcb995df-m8gbm"
Oct 30 22:20:34.274748 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:34.271483    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:20:34.300678 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:34.297873    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8qgf7\" (UniqueName: \"kubernetes.io/projected/2f58edd6-b3f3-4074-af7f-04970eddc81e-kube-api-access-8qgf7\") pod \"maas-api-e2e-mt-4da07739-5dcb995df-m8gbm\" (UID: \"2f58edd6-b3f3-4074-af7f-04970eddc81e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4da07739-5dcb995df-m8gbm"
Oct 30 22:20:34.485922 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:34.485882    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:20:34.731550 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:34.731451    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2f58edd6-b3f3-4074-af7f-04970eddc81e-maas-api-tls\") pod \"maas-api-e2e-mt-4da07739-5dcb995df-m8gbm\" (UID: \"2f58edd6-b3f3-4074-af7f-04970eddc81e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4da07739-5dcb995df-m8gbm"
Oct 30 22:20:34.734074 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:34.734048    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2f58edd6-b3f3-4074-af7f-04970eddc81e-maas-api-tls\") pod \"maas-api-e2e-mt-4da07739-5dcb995df-m8gbm\" (UID: \"2f58edd6-b3f3-4074-af7f-04970eddc81e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4da07739-5dcb995df-m8gbm"
Oct 30 22:20:34.920507 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:34.920471    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4da07739-5dcb995df-m8gbm"
Oct 30 22:20:34.939919 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:34.939890    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:20:35.295999 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:35.264907    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:20:35.907727 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:35.881147    5510 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w1-a6b6a5-85cc5f8687-hdbfk"]
Oct 30 22:20:35.987266 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:35.977795    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w1-a6b6a5-85cc5f8687-hdbfk"]
Oct 30 22:20:35.987266 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:35.977960    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-a6b6a5-85cc5f8687-hdbfk"
Oct 30 22:20:35.997335 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:35.993591    5510 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w1-a6b6a5-7976bff669-4lf96"]
Oct 30 22:20:36.002483 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:36.002133    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w1-a6b6a5-7976bff669-4lf96"]
Oct 30 22:20:36.002483 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:36.002248    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-a6b6a5-7976bff669-4lf96"
Oct 30 22:20:36.028230 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:36.028202    5510 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w1-a6b6a5\""
Oct 30 22:20:36.074812 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:36.074772    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w1-a6b6a5-dockercfg-stf4t\""
Oct 30 22:20:36.083910 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:36.083878    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rpwpb\" (UniqueName: \"kubernetes.io/projected/7554b329-1324-4e03-96ac-f8545413fbe3-kube-api-access-rpwpb\") pod \"payload-pre-processing-e2e-shared-a-w1-a6b6a5-85cc5f8687-hdbfk\" (UID: \"7554b329-1324-4e03-96ac-f8545413fbe3\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-a6b6a5-85cc5f8687-hdbfk"
Oct 30 22:20:36.084069 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:36.083941    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/19d32fc2-60a2-4040-813c-8b2127047510-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w1-a6b6a5-7976bff669-4lf96\" (UID: \"19d32fc2-60a2-4040-813c-8b2127047510\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w1-a6b6a5-7976bff669-4lf96"
Oct 30 22:20:36.084126 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:36.084107    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7554b329-1324-4e03-96ac-f8545413fbe3-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w1-a6b6a5-85cc5f8687-hdbfk\" (UID: \"7554b329-1324-4e03-96ac-f8545413fbe3\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-a6b6a5-85cc5f8687-hdbfk"
Oct 30 22:20:36.084277 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:36.084257    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4567t\" (UniqueName: \"kubernetes.io/projected/19d32fc2-60a2-4040-813c-8b2127047510-kube-api-access-4567t\") pod \"payload-processing-e2e-shared-a-w1-a6b6a5-7976bff669-4lf96\" (UID: \"19d32fc2-60a2-4040-813c-8b2127047510\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w1-a6b6a5-7976bff669-4lf96"
Oct 30 22:20:36.097401 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:36.097374    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-4da07739-5dcb995df-m8gbm"]
Oct 30 22:20:36.098214 ip-10-0-136-210 kubenswrapper[5510]: W1030 22:20:36.098186    5510 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f58edd6_b3f3_4074_af7f_04970eddc81e.slice/crio-026f761b5cb5681be3cd6b471cacec5cdc5f94d8ef3a278fc9a6e17898871ce9 WatchSource:0}: Error finding container 026f761b5cb5681be3cd6b471cacec5cdc5f94d8ef3a278fc9a6e17898871ce9: Status 404 returned error can't find the container with id 026f761b5cb5681be3cd6b471cacec5cdc5f94d8ef3a278fc9a6e17898871ce9
Oct 30 22:20:36.099649 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:36.099631    5510 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 22:20:36.201875 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:36.201764    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rpwpb\" (UniqueName: \"kubernetes.io/projected/7554b329-1324-4e03-96ac-f8545413fbe3-kube-api-access-rpwpb\") pod \"payload-pre-processing-e2e-shared-a-w1-a6b6a5-85cc5f8687-hdbfk\" (UID: \"7554b329-1324-4e03-96ac-f8545413fbe3\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-a6b6a5-85cc5f8687-hdbfk"
Oct 30 22:20:36.202235 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:36.202192    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/19d32fc2-60a2-4040-813c-8b2127047510-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w1-a6b6a5-7976bff669-4lf96\" (UID: \"19d32fc2-60a2-4040-813c-8b2127047510\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w1-a6b6a5-7976bff669-4lf96"
Oct 30 22:20:36.202591 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:36.202565    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7554b329-1324-4e03-96ac-f8545413fbe3-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w1-a6b6a5-85cc5f8687-hdbfk\" (UID: \"7554b329-1324-4e03-96ac-f8545413fbe3\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-a6b6a5-85cc5f8687-hdbfk"
Oct 30 22:20:36.202898 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:36.202876    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4567t\" (UniqueName: \"kubernetes.io/projected/19d32fc2-60a2-4040-813c-8b2127047510-kube-api-access-4567t\") pod \"payload-processing-e2e-shared-a-w1-a6b6a5-7976bff669-4lf96\" (UID: \"19d32fc2-60a2-4040-813c-8b2127047510\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w1-a6b6a5-7976bff669-4lf96"
Oct 30 22:20:36.204026 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:36.203862    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7554b329-1324-4e03-96ac-f8545413fbe3-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w1-a6b6a5-85cc5f8687-hdbfk\" (UID: \"7554b329-1324-4e03-96ac-f8545413fbe3\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-a6b6a5-85cc5f8687-hdbfk"
Oct 30 22:20:36.204221 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:36.204197    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/19d32fc2-60a2-4040-813c-8b2127047510-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w1-a6b6a5-7976bff669-4lf96\" (UID: \"19d32fc2-60a2-4040-813c-8b2127047510\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w1-a6b6a5-7976bff669-4lf96"
Oct 30 22:20:36.283733 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:36.282579    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rpwpb\" (UniqueName: \"kubernetes.io/projected/7554b329-1324-4e03-96ac-f8545413fbe3-kube-api-access-rpwpb\") pod \"payload-pre-processing-e2e-shared-a-w1-a6b6a5-85cc5f8687-hdbfk\" (UID: \"7554b329-1324-4e03-96ac-f8545413fbe3\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-a6b6a5-85cc5f8687-hdbfk"
Oct 30 22:20:36.283733 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:36.283222    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4567t\" (UniqueName: \"kubernetes.io/projected/19d32fc2-60a2-4040-813c-8b2127047510-kube-api-access-4567t\") pod \"payload-processing-e2e-shared-a-w1-a6b6a5-7976bff669-4lf96\" (UID: \"19d32fc2-60a2-4040-813c-8b2127047510\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w1-a6b6a5-7976bff669-4lf96"
Oct 30 22:20:36.324777 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:36.324749    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-a6b6a5-85cc5f8687-hdbfk"
Oct 30 22:20:36.333467 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:36.333318    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-a6b6a5-7976bff669-4lf96"
Oct 30 22:20:36.588836 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:36.588799    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6fvq6"]
Oct 30 22:20:36.589083 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:36.589056    5510 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6fvq6" podUID="5ff054ea-b27c-4c64-be2d-c645c0ca3d24" containerName="istio-proxy" containerID="cri-o://32d2dbdde860ac1cc4f5ac866be960f485032f283e065535f10556e82ee3d659" gracePeriod=30
Oct 30 22:20:36.593162 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:36.593128    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-48cnj"]
Oct 30 22:20:36.593334 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:36.593310    5510 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-48cnj" podUID="6aa08ff6-6f78-4a6e-83bd-07cf5d43b7b0" containerName="istio-proxy" containerID="cri-o://457dc830f7bf2bcddeeb03600608586d37a9d994491941e3ba957f653556657b" gracePeriod=30
Oct 30 22:20:36.635936 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:36.635904    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6mvrk"]
Oct 30 22:20:36.636108 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:36.636088    5510 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6mvrk" podUID="6cbfd1c6-ca09-4bee-a6d3-6461eac2c2d5" containerName="istio-proxy" containerID="cri-o://d55780c704afcc7e3d56c850cc0477a6aba36d95498a131e84b5479e7674d50d" gracePeriod=30
Oct 30 22:20:36.674981 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:36.674948    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-w8zj5"]
Oct 30 22:20:36.675182 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:36.675160    5510 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-w8zj5" podUID="07ed7cb3-6111-4809-8b1f-27a41ff6cafb" containerName="istio-proxy" containerID="cri-o://b03b2355a03dd6578e26e989fa3ba975166ed2d768c8f1ca4112d6a484404acd" gracePeriod=30
Oct 30 22:20:36.678962 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:36.678933    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-v4cxn"]
Oct 30 22:20:36.679173 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:36.679146    5510 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-v4cxn" podUID="5542b62f-bc73-47d7-8615-180f43324911" containerName="istio-proxy" containerID="cri-o://c6b85d95983fbf83ac908236d7f59308ddfbd8918e9314137276f3ecd11bfce6" gracePeriod=30
Oct 30 22:20:36.683564 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:36.683537    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-f969x"]
Oct 30 22:20:36.683783 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:36.683760    5510 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-f969x" podUID="b0d37b4a-b986-438a-a811-9037505f0e6a" containerName="istio-proxy" containerID="cri-o://ec2bae482f04cc37a8b77e7640794dd7b79c3183380a1e47d15be53d8bf7a675" gracePeriod=30
Oct 30 22:20:36.725346 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:36.725322    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w1-a6b6a5-85cc5f8687-hdbfk"]
Oct 30 22:20:36.727999 ip-10-0-136-210 kubenswrapper[5510]: W1030 22:20:36.727970    5510 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7554b329_1324_4e03_96ac_f8545413fbe3.slice/crio-c2c534e4e5e78552824c093cb7971e959f06c8057b99d1da6c56efee427e235f WatchSource:0}: Error finding container c2c534e4e5e78552824c093cb7971e959f06c8057b99d1da6c56efee427e235f: Status 404 returned error can't find the container with id c2c534e4e5e78552824c093cb7971e959f06c8057b99d1da6c56efee427e235f
Oct 30 22:20:36.759739 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:36.759712    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w1-a6b6a5-7976bff669-4lf96"]
Oct 30 22:20:36.765215 ip-10-0-136-210 kubenswrapper[5510]: W1030 22:20:36.765190    5510 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19d32fc2_60a2_4040_813c_8b2127047510.slice/crio-901dfa620e2980c52fee51e7391af162a90b597a29c66ccd28ad5a7b1a896404 WatchSource:0}: Error finding container 901dfa620e2980c52fee51e7391af162a90b597a29c66ccd28ad5a7b1a896404: Status 404 returned error can't find the container with id 901dfa620e2980c52fee51e7391af162a90b597a29c66ccd28ad5a7b1a896404
Oct 30 22:20:37.023829 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:37.023788    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-a6b6a5-7976bff669-4lf96" event={"ID":"19d32fc2-60a2-4040-813c-8b2127047510","Type":"ContainerStarted","Data":"3408840d0822d9773658d8401b20199ad8c7ac7017a38b2af2dc94ef2ec0b9c9"}
Oct 30 22:20:37.024002 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:37.023890    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-a6b6a5-7976bff669-4lf96"
Oct 30 22:20:37.024002 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:37.023916    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-a6b6a5-7976bff669-4lf96" event={"ID":"19d32fc2-60a2-4040-813c-8b2127047510","Type":"ContainerStarted","Data":"901dfa620e2980c52fee51e7391af162a90b597a29c66ccd28ad5a7b1a896404"}
Oct 30 22:20:37.025754 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:37.025723    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-a6b6a5-85cc5f8687-hdbfk" event={"ID":"7554b329-1324-4e03-96ac-f8545413fbe3","Type":"ContainerStarted","Data":"e88750cbf071a7b3e3d980b3ecb989642c87dfc4460813b30c3eb1f3cfb7aa7f"}
Oct 30 22:20:37.025861 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:37.025785    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-a6b6a5-85cc5f8687-hdbfk" event={"ID":"7554b329-1324-4e03-96ac-f8545413fbe3","Type":"ContainerStarted","Data":"c2c534e4e5e78552824c093cb7971e959f06c8057b99d1da6c56efee427e235f"}
Oct 30 22:20:37.025861 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:37.025819    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-a6b6a5-85cc5f8687-hdbfk"
Oct 30 22:20:37.027525 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:37.027493    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4da07739-5dcb995df-m8gbm" event={"ID":"2f58edd6-b3f3-4074-af7f-04970eddc81e","Type":"ContainerStarted","Data":"026f761b5cb5681be3cd6b471cacec5cdc5f94d8ef3a278fc9a6e17898871ce9"}
Oct 30 22:20:37.436598 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:37.436539    5510 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-a6b6a5-85cc5f8687-hdbfk" podStartSLOduration=2.436523032 podStartE2EDuration="2.436523032s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:20:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:20:37.43358308 +0000 UTC m=+1817.994393107" watchObservedRunningTime="2026-10-30 22:20:37.436523032 +0000 UTC m=+1817.997333036"
Oct 30 22:20:37.640637 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:37.640568    5510 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-a6b6a5-7976bff669-4lf96" podStartSLOduration=2.640544761 podStartE2EDuration="2.640544761s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:20:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:20:37.589918747 +0000 UTC m=+1818.150728752" watchObservedRunningTime="2026-10-30 22:20:37.640544761 +0000 UTC m=+1818.201354767"
Oct 30 22:20:38.036749 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:38.036712    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4da07739-5dcb995df-m8gbm" event={"ID":"2f58edd6-b3f3-4074-af7f-04970eddc81e","Type":"ContainerStarted","Data":"28c5136904803d3c2f3bd41644b3c04948751011dd8446dc8f706f95b48d570d"}
Oct 30 22:20:38.036928 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:38.036810    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4da07739-5dcb995df-m8gbm"
Oct 30 22:20:38.154259 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:38.153775    5510 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4da07739-5dcb995df-m8gbm" podStartSLOduration=4.261386058 podStartE2EDuration="5.153757741s" totalImagesPullingTime="892.371683ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:20:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:20:38.152563149 +0000 UTC m=+1818.713373170" watchObservedRunningTime="2026-10-30 22:20:38.153757741 +0000 UTC m=+1818.714567752"
Oct 30 22:20:38.440060 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:20:38.439973    5510 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6gjbf" containerName="extractor"
Oct 30 22:20:38.559785 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:38.559750    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-bee2db53-7ffd6dbfcb-s7jf4"]
Oct 30 22:20:38.559967 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:38.559948    5510 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-bee2db53-7ffd6dbfcb-s7jf4" podUID="9678c16a-f6f3-4055-af08-f3d0adaab85f" containerName="maas-api" containerID="cri-o://f9038a0a29975a8ce832dc81d33872d5fe7c68a69c0e4fe2e2158ca83aea36e7" gracePeriod=30
Oct 30 22:20:38.707577 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:38.707483    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:20:39.027003 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:39.026972    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-bee2db53-7ffd6dbfcb-s7jf4"
Oct 30 22:20:39.129681 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:39.122627    5510 generic.go:356] "Generic (PLEG): container finished" podID="9678c16a-f6f3-4055-af08-f3d0adaab85f" containerID="f9038a0a29975a8ce832dc81d33872d5fe7c68a69c0e4fe2e2158ca83aea36e7" exitCode=0
Oct 30 22:20:39.129681 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:39.123768    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-bee2db53-7ffd6dbfcb-s7jf4"
Oct 30 22:20:39.129681 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:39.123924    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-bee2db53-7ffd6dbfcb-s7jf4" event={"ID":"9678c16a-f6f3-4055-af08-f3d0adaab85f","Type":"ContainerDied","Data":"f9038a0a29975a8ce832dc81d33872d5fe7c68a69c0e4fe2e2158ca83aea36e7"}
Oct 30 22:20:39.129681 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:39.123966    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-bee2db53-7ffd6dbfcb-s7jf4" event={"ID":"9678c16a-f6f3-4055-af08-f3d0adaab85f","Type":"ContainerDied","Data":"a19176bf510f9f98a5ce90d8e9ef81717831e6a8e2f25e87e766afaf087ee2ef"}
Oct 30 22:20:39.129681 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:39.123999    5510 scope.go:122] "RemoveContainer" containerID="f9038a0a29975a8ce832dc81d33872d5fe7c68a69c0e4fe2e2158ca83aea36e7"
Oct 30 22:20:39.129681 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:39.129260    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9678c16a-f6f3-4055-af08-f3d0adaab85f-metrics-certs\" (UniqueName: \"kubernetes.io/secret/9678c16a-f6f3-4055-af08-f3d0adaab85f-metrics-certs\") pod \"9678c16a-f6f3-4055-af08-f3d0adaab85f\" (UID: \"9678c16a-f6f3-4055-af08-f3d0adaab85f\") "
Oct 30 22:20:39.129681 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:39.129331    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9678c16a-f6f3-4055-af08-f3d0adaab85f-kube-api-access-p6mbm\" (UniqueName: \"kubernetes.io/projected/9678c16a-f6f3-4055-af08-f3d0adaab85f-kube-api-access-p6mbm\") pod \"9678c16a-f6f3-4055-af08-f3d0adaab85f\" (UID: \"9678c16a-f6f3-4055-af08-f3d0adaab85f\") "
Oct 30 22:20:39.129681 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:39.129388    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9678c16a-f6f3-4055-af08-f3d0adaab85f-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9678c16a-f6f3-4055-af08-f3d0adaab85f-openshift-service-ca\") pod \"9678c16a-f6f3-4055-af08-f3d0adaab85f\" (UID: \"9678c16a-f6f3-4055-af08-f3d0adaab85f\") "
Oct 30 22:20:39.129681 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:39.129680    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9678c16a-f6f3-4055-af08-f3d0adaab85f-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9678c16a-f6f3-4055-af08-f3d0adaab85f-maas-api-tls\") pod \"9678c16a-f6f3-4055-af08-f3d0adaab85f\" (UID: \"9678c16a-f6f3-4055-af08-f3d0adaab85f\") "
Oct 30 22:20:39.183987 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:39.183713    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9678c16a-f6f3-4055-af08-f3d0adaab85f-kube-api-access-p6mbm" pod "9678c16a-f6f3-4055-af08-f3d0adaab85f" (UID: "9678c16a-f6f3-4055-af08-f3d0adaab85f"). InnerVolumeSpecName "kube-api-access-p6mbm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:20:39.187108 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:39.186091    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9678c16a-f6f3-4055-af08-f3d0adaab85f-metrics-certs" pod "9678c16a-f6f3-4055-af08-f3d0adaab85f" (UID: "9678c16a-f6f3-4055-af08-f3d0adaab85f"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:20:39.195505 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:39.195466    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9678c16a-f6f3-4055-af08-f3d0adaab85f-maas-api-tls" pod "9678c16a-f6f3-4055-af08-f3d0adaab85f" (UID: "9678c16a-f6f3-4055-af08-f3d0adaab85f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:20:39.210839 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:39.207844    5510 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9678c16a-f6f3-4055-af08-f3d0adaab85f-maas-api-tls\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:20:39.210839 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:39.207876    5510 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9678c16a-f6f3-4055-af08-f3d0adaab85f-metrics-certs\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:20:39.210839 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:39.207893    5510 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p6mbm\" (UniqueName: \"kubernetes.io/projected/9678c16a-f6f3-4055-af08-f3d0adaab85f-kube-api-access-p6mbm\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:20:39.223822 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:39.223339    5510 scope.go:122] "RemoveContainer" containerID="f9038a0a29975a8ce832dc81d33872d5fe7c68a69c0e4fe2e2158ca83aea36e7"
Oct 30 22:20:39.238741 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:20:39.224341    5510 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f9038a0a29975a8ce832dc81d33872d5fe7c68a69c0e4fe2e2158ca83aea36e7\": container with ID starting with f9038a0a29975a8ce832dc81d33872d5fe7c68a69c0e4fe2e2158ca83aea36e7 not found: ID does not exist" containerID="f9038a0a29975a8ce832dc81d33872d5fe7c68a69c0e4fe2e2158ca83aea36e7"
Oct 30 22:20:39.238741 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:39.224384    5510 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f9038a0a29975a8ce832dc81d33872d5fe7c68a69c0e4fe2e2158ca83aea36e7"} err="failed to get container status \"f9038a0a29975a8ce832dc81d33872d5fe7c68a69c0e4fe2e2158ca83aea36e7\": rpc error: code = NotFound desc = could not find container \"f9038a0a29975a8ce832dc81d33872d5fe7c68a69c0e4fe2e2158ca83aea36e7\": container with ID starting with f9038a0a29975a8ce832dc81d33872d5fe7c68a69c0e4fe2e2158ca83aea36e7 not found: ID does not exist"
Oct 30 22:20:39.238741 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:39.236330    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9678c16a-f6f3-4055-af08-f3d0adaab85f-openshift-service-ca" pod "9678c16a-f6f3-4055-af08-f3d0adaab85f" (UID: "9678c16a-f6f3-4055-af08-f3d0adaab85f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:20:39.310210 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:39.310170    5510 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9678c16a-f6f3-4055-af08-f3d0adaab85f-openshift-service-ca\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:20:39.607805 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:39.598733    5510 patch_prober.go:29] interesting pod/e2e-worker-w2-bc714d-openshift-default-67558b95cf-w8zj5 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 22:20:39.607805 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:39.598796    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-w8zj5" podUID="07ed7cb3-6111-4809-8b1f-27a41ff6cafb" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 22:20:39.853092 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:39.853051    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-bee2db53-7ffd6dbfcb-s7jf4"]
Oct 30 22:20:39.853273 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:39.853115    5510 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-bee2db53-7ffd6dbfcb-s7jf4"]
Oct 30 22:20:40.163911 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:40.163877    5510 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-ed44da-9b6f6d88c-jz6p9"]
Oct 30 22:20:40.164666 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:40.164640    5510 cpu_manager.go:414] "containerMap: removing container" podUID="9678c16a-f6f3-4055-af08-f3d0adaab85f" containerName="maas-api"
Oct 30 22:20:40.164737 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:40.164671    5510 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9678c16a-f6f3-4055-af08-f3d0adaab85f" containerName="maas-api"
Oct 30 22:20:40.164863 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:40.164851    5510 memory_manager.go:365] "RemoveStaleState removing state" podUID="9678c16a-f6f3-4055-af08-f3d0adaab85f" containerName="maas-api"
Oct 30 22:20:40.170181 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:40.170161    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-ed44da-9b6f6d88c-jz6p9"
Oct 30 22:20:40.220628 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:40.220596    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w0-ed44da-serving-cert\""
Oct 30 22:20:40.231701 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:40.226240    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-ed44da-9b6f6d88c-jz6p9"]
Oct 30 22:20:40.307809 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:40.307769    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f8bd673e-4938-421a-bb40-5eb3e7f7802f-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w0-ed44da-9b6f6d88c-jz6p9\" (UID: \"f8bd673e-4938-421a-bb40-5eb3e7f7802f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-ed44da-9b6f6d88c-jz6p9"
Oct 30 22:20:40.308013 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:40.307927    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f8bd673e-4938-421a-bb40-5eb3e7f7802f-maas-api-tls\") pod \"maas-api-e2e-shared-b-w0-ed44da-9b6f6d88c-jz6p9\" (UID: \"f8bd673e-4938-421a-bb40-5eb3e7f7802f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-ed44da-9b6f6d88c-jz6p9"
Oct 30 22:20:40.308072 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:40.308024    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f8bd673e-4938-421a-bb40-5eb3e7f7802f-metrics-certs\") pod \"maas-api-e2e-shared-b-w0-ed44da-9b6f6d88c-jz6p9\" (UID: \"f8bd673e-4938-421a-bb40-5eb3e7f7802f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-ed44da-9b6f6d88c-jz6p9"
Oct 30 22:20:40.308126 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:40.308076    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8r65x\" (UniqueName: \"kubernetes.io/projected/f8bd673e-4938-421a-bb40-5eb3e7f7802f-kube-api-access-8r65x\") pod \"maas-api-e2e-shared-b-w0-ed44da-9b6f6d88c-jz6p9\" (UID: \"f8bd673e-4938-421a-bb40-5eb3e7f7802f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-ed44da-9b6f6d88c-jz6p9"
Oct 30 22:20:40.314634 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:40.314591    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:20:40.424789 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:40.415120    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f8bd673e-4938-421a-bb40-5eb3e7f7802f-maas-api-tls\") pod \"maas-api-e2e-shared-b-w0-ed44da-9b6f6d88c-jz6p9\" (UID: \"f8bd673e-4938-421a-bb40-5eb3e7f7802f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-ed44da-9b6f6d88c-jz6p9"
Oct 30 22:20:40.424789 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:40.415250    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f8bd673e-4938-421a-bb40-5eb3e7f7802f-metrics-certs\") pod \"maas-api-e2e-shared-b-w0-ed44da-9b6f6d88c-jz6p9\" (UID: \"f8bd673e-4938-421a-bb40-5eb3e7f7802f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-ed44da-9b6f6d88c-jz6p9"
Oct 30 22:20:40.424789 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:40.415312    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8r65x\" (UniqueName: \"kubernetes.io/projected/f8bd673e-4938-421a-bb40-5eb3e7f7802f-kube-api-access-8r65x\") pod \"maas-api-e2e-shared-b-w0-ed44da-9b6f6d88c-jz6p9\" (UID: \"f8bd673e-4938-421a-bb40-5eb3e7f7802f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-ed44da-9b6f6d88c-jz6p9"
Oct 30 22:20:40.424789 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:40.415627    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f8bd673e-4938-421a-bb40-5eb3e7f7802f-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w0-ed44da-9b6f6d88c-jz6p9\" (UID: \"f8bd673e-4938-421a-bb40-5eb3e7f7802f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-ed44da-9b6f6d88c-jz6p9"
Oct 30 22:20:40.435691 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:20:40.431314    5510 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-ed44da-serving-cert: secret "maas-api-e2e-shared-b-w0-ed44da-serving-cert" not found
Oct 30 22:20:40.457694 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:40.453072    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f8bd673e-4938-421a-bb40-5eb3e7f7802f-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w0-ed44da-9b6f6d88c-jz6p9\" (UID: \"f8bd673e-4938-421a-bb40-5eb3e7f7802f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-ed44da-9b6f6d88c-jz6p9"
Oct 30 22:20:40.493264 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:20:40.493236    5510 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f8bd673e-4938-421a-bb40-5eb3e7f7802f-maas-api-tls podName:f8bd673e-4938-421a-bb40-5eb3e7f7802f nodeName:}" failed. No retries permitted until 2026-10-30 22:20:40.993208275 +0000 UTC m=+1821.554018270 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/f8bd673e-4938-421a-bb40-5eb3e7f7802f-maas-api-tls") pod "maas-api-e2e-shared-b-w0-ed44da-9b6f6d88c-jz6p9" (UID: "f8bd673e-4938-421a-bb40-5eb3e7f7802f") : secret "maas-api-e2e-shared-b-w0-ed44da-serving-cert" not found
Oct 30 22:20:40.531631 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:40.529795    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f8bd673e-4938-421a-bb40-5eb3e7f7802f-metrics-certs\") pod \"maas-api-e2e-shared-b-w0-ed44da-9b6f6d88c-jz6p9\" (UID: \"f8bd673e-4938-421a-bb40-5eb3e7f7802f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-ed44da-9b6f6d88c-jz6p9"
Oct 30 22:20:40.532588 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:40.532557    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8r65x\" (UniqueName: \"kubernetes.io/projected/f8bd673e-4938-421a-bb40-5eb3e7f7802f-kube-api-access-8r65x\") pod \"maas-api-e2e-shared-b-w0-ed44da-9b6f6d88c-jz6p9\" (UID: \"f8bd673e-4938-421a-bb40-5eb3e7f7802f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-ed44da-9b6f6d88c-jz6p9"
Oct 30 22:20:40.560674 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:40.556883    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:20:40.758109 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:40.758024    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:20:40.861391 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:40.861320    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-bee2db53-openshift-default-6b56b8cdb9-jv7mv"]
Oct 30 22:20:40.861628 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:40.861595    5510 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-bee2db53-openshift-default-6b56b8cdb9-jv7mv" podUID="01891eb3-3336-4e31-a26a-9a1ecb935f9c" containerName="istio-proxy" containerID="cri-o://bc8c3d1c2035a325ec675005f3202f183bae9d45f8881edd84ec3f62791803df" gracePeriod=30
Oct 30 22:20:41.015740 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:41.009111    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f8bd673e-4938-421a-bb40-5eb3e7f7802f-maas-api-tls\") pod \"maas-api-e2e-shared-b-w0-ed44da-9b6f6d88c-jz6p9\" (UID: \"f8bd673e-4938-421a-bb40-5eb3e7f7802f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-ed44da-9b6f6d88c-jz6p9"
Oct 30 22:20:41.044715 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:41.044501    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:20:41.066697 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:41.064412    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f8bd673e-4938-421a-bb40-5eb3e7f7802f-maas-api-tls\") pod \"maas-api-e2e-shared-b-w0-ed44da-9b6f6d88c-jz6p9\" (UID: \"f8bd673e-4938-421a-bb40-5eb3e7f7802f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-ed44da-9b6f6d88c-jz6p9"
Oct 30 22:20:41.091678 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:41.088977    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-ed44da-9b6f6d88c-jz6p9"
Oct 30 22:20:41.555476 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:41.555403    5510 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9678c16a-f6f3-4055-af08-f3d0adaab85f" path="/var/lib/kubelet/pods/9678c16a-f6f3-4055-af08-f3d0adaab85f/volumes"
Oct 30 22:20:41.694775 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:41.694746    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-ed44da-9b6f6d88c-jz6p9"]
Oct 30 22:20:41.739833 ip-10-0-136-210 kubenswrapper[5510]: W1030 22:20:41.739795    5510 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8bd673e_4938_421a_bb40_5eb3e7f7802f.slice/crio-42b5d9c2b748bacc5e36c5e1a35f17e39fb8b31d37c93d86bc58b8a7c83baa83 WatchSource:0}: Error finding container 42b5d9c2b748bacc5e36c5e1a35f17e39fb8b31d37c93d86bc58b8a7c83baa83: Status 404 returned error can't find the container with id 42b5d9c2b748bacc5e36c5e1a35f17e39fb8b31d37c93d86bc58b8a7c83baa83
Oct 30 22:20:41.974907 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:41.974878    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6fvq6"
Oct 30 22:20:42.036260 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.036231    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6mvrk"
Oct 30 22:20:42.057107 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.057071    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5ff054ea-b27c-4c64-be2d-c645c0ca3d24-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5ff054ea-b27c-4c64-be2d-c645c0ca3d24-workload-socket\") pod \"5ff054ea-b27c-4c64-be2d-c645c0ca3d24\" (UID: \"5ff054ea-b27c-4c64-be2d-c645c0ca3d24\") "
Oct 30 22:20:42.057268 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.057142    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5ff054ea-b27c-4c64-be2d-c645c0ca3d24-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5ff054ea-b27c-4c64-be2d-c645c0ca3d24-istio-envoy\") pod \"5ff054ea-b27c-4c64-be2d-c645c0ca3d24\" (UID: \"5ff054ea-b27c-4c64-be2d-c645c0ca3d24\") "
Oct 30 22:20:42.057268 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.057162    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5ff054ea-b27c-4c64-be2d-c645c0ca3d24-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5ff054ea-b27c-4c64-be2d-c645c0ca3d24-istio-data\") pod \"5ff054ea-b27c-4c64-be2d-c645c0ca3d24\" (UID: \"5ff054ea-b27c-4c64-be2d-c645c0ca3d24\") "
Oct 30 22:20:42.057268 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.057195    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5ff054ea-b27c-4c64-be2d-c645c0ca3d24-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5ff054ea-b27c-4c64-be2d-c645c0ca3d24-istiod-ca-cert\") pod \"5ff054ea-b27c-4c64-be2d-c645c0ca3d24\" (UID: \"5ff054ea-b27c-4c64-be2d-c645c0ca3d24\") "
Oct 30 22:20:42.057268 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.057239    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/5ff054ea-b27c-4c64-be2d-c645c0ca3d24-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5ff054ea-b27c-4c64-be2d-c645c0ca3d24-istio-podinfo\") pod \"5ff054ea-b27c-4c64-be2d-c645c0ca3d24\" (UID: \"5ff054ea-b27c-4c64-be2d-c645c0ca3d24\") "
Oct 30 22:20:42.057506 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.057269    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5ff054ea-b27c-4c64-be2d-c645c0ca3d24-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5ff054ea-b27c-4c64-be2d-c645c0ca3d24-credential-socket\") pod \"5ff054ea-b27c-4c64-be2d-c645c0ca3d24\" (UID: \"5ff054ea-b27c-4c64-be2d-c645c0ca3d24\") "
Oct 30 22:20:42.057506 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.057419    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5ff054ea-b27c-4c64-be2d-c645c0ca3d24-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5ff054ea-b27c-4c64-be2d-c645c0ca3d24-workload-certs\") pod \"5ff054ea-b27c-4c64-be2d-c645c0ca3d24\" (UID: \"5ff054ea-b27c-4c64-be2d-c645c0ca3d24\") "
Oct 30 22:20:42.057612 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.057576    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5ff054ea-b27c-4c64-be2d-c645c0ca3d24-istio-token\" (UniqueName: \"kubernetes.io/projected/5ff054ea-b27c-4c64-be2d-c645c0ca3d24-istio-token\") pod \"5ff054ea-b27c-4c64-be2d-c645c0ca3d24\" (UID: \"5ff054ea-b27c-4c64-be2d-c645c0ca3d24\") "
Oct 30 22:20:42.058348 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.058248    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5ff054ea-b27c-4c64-be2d-c645c0ca3d24-workload-socket" pod "5ff054ea-b27c-4c64-be2d-c645c0ca3d24" (UID: "5ff054ea-b27c-4c64-be2d-c645c0ca3d24"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:20:42.077085 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.074747    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5ff054ea-b27c-4c64-be2d-c645c0ca3d24-workload-certs" pod "5ff054ea-b27c-4c64-be2d-c645c0ca3d24" (UID: "5ff054ea-b27c-4c64-be2d-c645c0ca3d24"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:20:42.077085 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.075467    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5ff054ea-b27c-4c64-be2d-c645c0ca3d24-istio-data" pod "5ff054ea-b27c-4c64-be2d-c645c0ca3d24" (UID: "5ff054ea-b27c-4c64-be2d-c645c0ca3d24"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:20:42.077085 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.075640    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5ff054ea-b27c-4c64-be2d-c645c0ca3d24-istiod-ca-cert" pod "5ff054ea-b27c-4c64-be2d-c645c0ca3d24" (UID: "5ff054ea-b27c-4c64-be2d-c645c0ca3d24"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:20:42.077085 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.075709    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5ff054ea-b27c-4c64-be2d-c645c0ca3d24-credential-socket" pod "5ff054ea-b27c-4c64-be2d-c645c0ca3d24" (UID: "5ff054ea-b27c-4c64-be2d-c645c0ca3d24"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:20:42.077085 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.075772    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5ff054ea-b27c-4c64-be2d-c645c0ca3d24-kube-api-access-nx7zv\" (UniqueName: \"kubernetes.io/projected/5ff054ea-b27c-4c64-be2d-c645c0ca3d24-kube-api-access-nx7zv\") pod \"5ff054ea-b27c-4c64-be2d-c645c0ca3d24\" (UID: \"5ff054ea-b27c-4c64-be2d-c645c0ca3d24\") "
Oct 30 22:20:42.077085 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.076734    5510 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5ff054ea-b27c-4c64-be2d-c645c0ca3d24-workload-socket\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:20:42.077085 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.076757    5510 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5ff054ea-b27c-4c64-be2d-c645c0ca3d24-istio-data\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:20:42.077085 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.076769    5510 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5ff054ea-b27c-4c64-be2d-c645c0ca3d24-istiod-ca-cert\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:20:42.077085 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.076784    5510 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5ff054ea-b27c-4c64-be2d-c645c0ca3d24-credential-socket\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:20:42.077085 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.076796    5510 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5ff054ea-b27c-4c64-be2d-c645c0ca3d24-workload-certs\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:20:42.113527 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.113484    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5ff054ea-b27c-4c64-be2d-c645c0ca3d24-kube-api-access-nx7zv" pod "5ff054ea-b27c-4c64-be2d-c645c0ca3d24" (UID: "5ff054ea-b27c-4c64-be2d-c645c0ca3d24"). InnerVolumeSpecName "kube-api-access-nx7zv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:20:42.114489 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.114461    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5ff054ea-b27c-4c64-be2d-c645c0ca3d24-istio-podinfo" pod "5ff054ea-b27c-4c64-be2d-c645c0ca3d24" (UID: "5ff054ea-b27c-4c64-be2d-c645c0ca3d24"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:20:42.114624 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.114609    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5ff054ea-b27c-4c64-be2d-c645c0ca3d24-istio-token" pod "5ff054ea-b27c-4c64-be2d-c645c0ca3d24" (UID: "5ff054ea-b27c-4c64-be2d-c645c0ca3d24"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:20:42.141182 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.141144    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5ff054ea-b27c-4c64-be2d-c645c0ca3d24-istio-envoy" pod "5ff054ea-b27c-4c64-be2d-c645c0ca3d24" (UID: "5ff054ea-b27c-4c64-be2d-c645c0ca3d24"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:20:42.177729 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.177699    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6cbfd1c6-ca09-4bee-a6d3-6461eac2c2d5-istio-token\" (UniqueName: \"kubernetes.io/projected/6cbfd1c6-ca09-4bee-a6d3-6461eac2c2d5-istio-token\") pod \"6cbfd1c6-ca09-4bee-a6d3-6461eac2c2d5\" (UID: \"6cbfd1c6-ca09-4bee-a6d3-6461eac2c2d5\") "
Oct 30 22:20:42.177862 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.177765    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6cbfd1c6-ca09-4bee-a6d3-6461eac2c2d5-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6cbfd1c6-ca09-4bee-a6d3-6461eac2c2d5-workload-socket\") pod \"6cbfd1c6-ca09-4bee-a6d3-6461eac2c2d5\" (UID: \"6cbfd1c6-ca09-4bee-a6d3-6461eac2c2d5\") "
Oct 30 22:20:42.177926 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.177914    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/6cbfd1c6-ca09-4bee-a6d3-6461eac2c2d5-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6cbfd1c6-ca09-4bee-a6d3-6461eac2c2d5-istio-podinfo\") pod \"6cbfd1c6-ca09-4bee-a6d3-6461eac2c2d5\" (UID: \"6cbfd1c6-ca09-4bee-a6d3-6461eac2c2d5\") "
Oct 30 22:20:42.178024 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.178012    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6cbfd1c6-ca09-4bee-a6d3-6461eac2c2d5-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6cbfd1c6-ca09-4bee-a6d3-6461eac2c2d5-workload-certs\") pod \"6cbfd1c6-ca09-4bee-a6d3-6461eac2c2d5\" (UID: \"6cbfd1c6-ca09-4bee-a6d3-6461eac2c2d5\") "
Oct 30 22:20:42.178112 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.178099    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6cbfd1c6-ca09-4bee-a6d3-6461eac2c2d5-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6cbfd1c6-ca09-4bee-a6d3-6461eac2c2d5-istio-envoy\") pod \"6cbfd1c6-ca09-4bee-a6d3-6461eac2c2d5\" (UID: \"6cbfd1c6-ca09-4bee-a6d3-6461eac2c2d5\") "
Oct 30 22:20:42.178164 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.178154    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6cbfd1c6-ca09-4bee-a6d3-6461eac2c2d5-kube-api-access-v6nv7\" (UniqueName: \"kubernetes.io/projected/6cbfd1c6-ca09-4bee-a6d3-6461eac2c2d5-kube-api-access-v6nv7\") pod \"6cbfd1c6-ca09-4bee-a6d3-6461eac2c2d5\" (UID: \"6cbfd1c6-ca09-4bee-a6d3-6461eac2c2d5\") "
Oct 30 22:20:42.178218 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.178181    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6cbfd1c6-ca09-4bee-a6d3-6461eac2c2d5-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6cbfd1c6-ca09-4bee-a6d3-6461eac2c2d5-credential-socket\") pod \"6cbfd1c6-ca09-4bee-a6d3-6461eac2c2d5\" (UID: \"6cbfd1c6-ca09-4bee-a6d3-6461eac2c2d5\") "
Oct 30 22:20:42.178273 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.178232    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6cbfd1c6-ca09-4bee-a6d3-6461eac2c2d5-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6cbfd1c6-ca09-4bee-a6d3-6461eac2c2d5-istiod-ca-cert\") pod \"6cbfd1c6-ca09-4bee-a6d3-6461eac2c2d5\" (UID: \"6cbfd1c6-ca09-4bee-a6d3-6461eac2c2d5\") "
Oct 30 22:20:42.178320 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.178281    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6cbfd1c6-ca09-4bee-a6d3-6461eac2c2d5-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6cbfd1c6-ca09-4bee-a6d3-6461eac2c2d5-istio-data\") pod \"6cbfd1c6-ca09-4bee-a6d3-6461eac2c2d5\" (UID: \"6cbfd1c6-ca09-4bee-a6d3-6461eac2c2d5\") "
Oct 30 22:20:42.178966 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.178948    5510 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5ff054ea-b27c-4c64-be2d-c645c0ca3d24-istio-podinfo\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:20:42.179032 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.178969    5510 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5ff054ea-b27c-4c64-be2d-c645c0ca3d24-istio-token\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:20:42.179032 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.178980    5510 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nx7zv\" (UniqueName: \"kubernetes.io/projected/5ff054ea-b27c-4c64-be2d-c645c0ca3d24-kube-api-access-nx7zv\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:20:42.179032 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.178989    5510 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5ff054ea-b27c-4c64-be2d-c645c0ca3d24-istio-envoy\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:20:42.179719 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.179352    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6cbfd1c6-ca09-4bee-a6d3-6461eac2c2d5-workload-certs" pod "6cbfd1c6-ca09-4bee-a6d3-6461eac2c2d5" (UID: "6cbfd1c6-ca09-4bee-a6d3-6461eac2c2d5"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:20:42.179719 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.179474    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6cbfd1c6-ca09-4bee-a6d3-6461eac2c2d5-credential-socket" pod "6cbfd1c6-ca09-4bee-a6d3-6461eac2c2d5" (UID: "6cbfd1c6-ca09-4bee-a6d3-6461eac2c2d5"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:20:42.179719 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.179692    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6cbfd1c6-ca09-4bee-a6d3-6461eac2c2d5-istio-data" pod "6cbfd1c6-ca09-4bee-a6d3-6461eac2c2d5" (UID: "6cbfd1c6-ca09-4bee-a6d3-6461eac2c2d5"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:20:42.179719 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.179709    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6cbfd1c6-ca09-4bee-a6d3-6461eac2c2d5-workload-socket" pod "6cbfd1c6-ca09-4bee-a6d3-6461eac2c2d5" (UID: "6cbfd1c6-ca09-4bee-a6d3-6461eac2c2d5"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:20:42.179938 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.179724    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6cbfd1c6-ca09-4bee-a6d3-6461eac2c2d5-istiod-ca-cert" pod "6cbfd1c6-ca09-4bee-a6d3-6461eac2c2d5" (UID: "6cbfd1c6-ca09-4bee-a6d3-6461eac2c2d5"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:20:42.190423 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.190392    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6cbfd1c6-ca09-4bee-a6d3-6461eac2c2d5-istio-envoy" pod "6cbfd1c6-ca09-4bee-a6d3-6461eac2c2d5" (UID: "6cbfd1c6-ca09-4bee-a6d3-6461eac2c2d5"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:20:42.190537 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.190449    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6cbfd1c6-ca09-4bee-a6d3-6461eac2c2d5-istio-token" pod "6cbfd1c6-ca09-4bee-a6d3-6461eac2c2d5" (UID: "6cbfd1c6-ca09-4bee-a6d3-6461eac2c2d5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:20:42.192430 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.192390    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6cbfd1c6-ca09-4bee-a6d3-6461eac2c2d5-kube-api-access-v6nv7" pod "6cbfd1c6-ca09-4bee-a6d3-6461eac2c2d5" (UID: "6cbfd1c6-ca09-4bee-a6d3-6461eac2c2d5"). InnerVolumeSpecName "kube-api-access-v6nv7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:20:42.193495 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.193466    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6cbfd1c6-ca09-4bee-a6d3-6461eac2c2d5-istio-podinfo" pod "6cbfd1c6-ca09-4bee-a6d3-6461eac2c2d5" (UID: "6cbfd1c6-ca09-4bee-a6d3-6461eac2c2d5"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:20:42.202670 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.202632    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-48cnj"
Oct 30 22:20:42.205257 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.205223    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-f969x"
Oct 30 22:20:42.214487 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.214454    5510 generic.go:356] "Generic (PLEG): container finished" podID="6aa08ff6-6f78-4a6e-83bd-07cf5d43b7b0" containerID="457dc830f7bf2bcddeeb03600608586d37a9d994491941e3ba957f653556657b" exitCode=0
Oct 30 22:20:42.214604 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.214577    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-48cnj" event={"ID":"6aa08ff6-6f78-4a6e-83bd-07cf5d43b7b0","Type":"ContainerDied","Data":"457dc830f7bf2bcddeeb03600608586d37a9d994491941e3ba957f653556657b"}
Oct 30 22:20:42.214716 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.214628    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-48cnj" event={"ID":"6aa08ff6-6f78-4a6e-83bd-07cf5d43b7b0","Type":"ContainerDied","Data":"eb723c1060732baa900e4166970417222a52e782db181f6bbe678c2401900a57"}
Oct 30 22:20:42.214716 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.214701    5510 scope.go:122] "RemoveContainer" containerID="457dc830f7bf2bcddeeb03600608586d37a9d994491941e3ba957f653556657b"
Oct 30 22:20:42.214851 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.214834    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-48cnj"
Oct 30 22:20:42.215313 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.215288    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-w8zj5"
Oct 30 22:20:42.217347 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.217053    5510 generic.go:356] "Generic (PLEG): container finished" podID="07ed7cb3-6111-4809-8b1f-27a41ff6cafb" containerID="b03b2355a03dd6578e26e989fa3ba975166ed2d768c8f1ca4112d6a484404acd" exitCode=0
Oct 30 22:20:42.217347 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.217131    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-w8zj5" event={"ID":"07ed7cb3-6111-4809-8b1f-27a41ff6cafb","Type":"ContainerDied","Data":"b03b2355a03dd6578e26e989fa3ba975166ed2d768c8f1ca4112d6a484404acd"}
Oct 30 22:20:42.217347 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.217171    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-w8zj5" event={"ID":"07ed7cb3-6111-4809-8b1f-27a41ff6cafb","Type":"ContainerDied","Data":"99e97160a33ea2adec740144c2730b43556389a5a6f0c711ad0e0e414d2acf51"}
Oct 30 22:20:42.228266 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.228240    5510 generic.go:356] "Generic (PLEG): container finished" podID="b0d37b4a-b986-438a-a811-9037505f0e6a" containerID="ec2bae482f04cc37a8b77e7640794dd7b79c3183380a1e47d15be53d8bf7a675" exitCode=0
Oct 30 22:20:42.228619 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.228600    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-f969x"
Oct 30 22:20:42.228894 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.228875    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-f969x" event={"ID":"b0d37b4a-b986-438a-a811-9037505f0e6a","Type":"ContainerDied","Data":"ec2bae482f04cc37a8b77e7640794dd7b79c3183380a1e47d15be53d8bf7a675"}
Oct 30 22:20:42.229027 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.229012    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-f969x" event={"ID":"b0d37b4a-b986-438a-a811-9037505f0e6a","Type":"ContainerDied","Data":"3e9e71fdf38e83391ea1c00e9ae7c4f6b50abea630ea214be94b0a710d0f3cd5"}
Oct 30 22:20:42.237213 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.237178    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-v4cxn"
Oct 30 22:20:42.238329 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.238302    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-ed44da-9b6f6d88c-jz6p9" event={"ID":"f8bd673e-4938-421a-bb40-5eb3e7f7802f","Type":"ContainerStarted","Data":"42b5d9c2b748bacc5e36c5e1a35f17e39fb8b31d37c93d86bc58b8a7c83baa83"}
Oct 30 22:20:42.239631 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.239614    5510 scope.go:122] "RemoveContainer" containerID="457dc830f7bf2bcddeeb03600608586d37a9d994491941e3ba957f653556657b"
Oct 30 22:20:42.240209 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:20:42.240187    5510 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"457dc830f7bf2bcddeeb03600608586d37a9d994491941e3ba957f653556657b\": container with ID starting with 457dc830f7bf2bcddeeb03600608586d37a9d994491941e3ba957f653556657b not found: ID does not exist" containerID="457dc830f7bf2bcddeeb03600608586d37a9d994491941e3ba957f653556657b"
Oct 30 22:20:42.240311 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.240216    5510 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"457dc830f7bf2bcddeeb03600608586d37a9d994491941e3ba957f653556657b"} err="failed to get container status \"457dc830f7bf2bcddeeb03600608586d37a9d994491941e3ba957f653556657b\": rpc error: code = NotFound desc = could not find container \"457dc830f7bf2bcddeeb03600608586d37a9d994491941e3ba957f653556657b\": container with ID starting with 457dc830f7bf2bcddeeb03600608586d37a9d994491941e3ba957f653556657b not found: ID does not exist"
Oct 30 22:20:42.240311 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.240237    5510 scope.go:122] "RemoveContainer" containerID="b03b2355a03dd6578e26e989fa3ba975166ed2d768c8f1ca4112d6a484404acd"
Oct 30 22:20:42.241626 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.241606    5510 generic.go:356] "Generic (PLEG): container finished" podID="5542b62f-bc73-47d7-8615-180f43324911" containerID="c6b85d95983fbf83ac908236d7f59308ddfbd8918e9314137276f3ecd11bfce6" exitCode=0
Oct 30 22:20:42.241754 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.241735    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-v4cxn" event={"ID":"5542b62f-bc73-47d7-8615-180f43324911","Type":"ContainerDied","Data":"c6b85d95983fbf83ac908236d7f59308ddfbd8918e9314137276f3ecd11bfce6"}
Oct 30 22:20:42.241821 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.241782    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-v4cxn" event={"ID":"5542b62f-bc73-47d7-8615-180f43324911","Type":"ContainerDied","Data":"941b827793a7cd2a808186af7148024ddd475cd61bde3ed027b95a67412d68e1"}
Oct 30 22:20:42.242036 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.242018    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-v4cxn"
Oct 30 22:20:42.244852 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.244832    5510 generic.go:356] "Generic (PLEG): container finished" podID="5ff054ea-b27c-4c64-be2d-c645c0ca3d24" containerID="32d2dbdde860ac1cc4f5ac866be960f485032f283e065535f10556e82ee3d659" exitCode=0
Oct 30 22:20:42.245032 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.244895    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6fvq6"
Oct 30 22:20:42.245601 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.244927    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6fvq6" event={"ID":"5ff054ea-b27c-4c64-be2d-c645c0ca3d24","Type":"ContainerDied","Data":"32d2dbdde860ac1cc4f5ac866be960f485032f283e065535f10556e82ee3d659"}
Oct 30 22:20:42.245601 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.245456    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6fvq6" event={"ID":"5ff054ea-b27c-4c64-be2d-c645c0ca3d24","Type":"ContainerDied","Data":"2f3048de9b7826761e5cafca5e011ff1506133ecd03d8331b178e8ea9a36e9c8"}
Oct 30 22:20:42.251301 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.251280    5510 generic.go:356] "Generic (PLEG): container finished" podID="6cbfd1c6-ca09-4bee-a6d3-6461eac2c2d5" containerID="d55780c704afcc7e3d56c850cc0477a6aba36d95498a131e84b5479e7674d50d" exitCode=0
Oct 30 22:20:42.251484 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.251451    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6mvrk" event={"ID":"6cbfd1c6-ca09-4bee-a6d3-6461eac2c2d5","Type":"ContainerDied","Data":"d55780c704afcc7e3d56c850cc0477a6aba36d95498a131e84b5479e7674d50d"}
Oct 30 22:20:42.251554 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.251353    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6mvrk"
Oct 30 22:20:42.251554 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.251516    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6mvrk" event={"ID":"6cbfd1c6-ca09-4bee-a6d3-6461eac2c2d5","Type":"ContainerDied","Data":"2ab0440c18ffc1afbbe1fc60dd29a65ab4a1b5c41bb2d12bc1949287a581f8b6"}
Oct 30 22:20:42.254812 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.254783    5510 scope.go:122] "RemoveContainer" containerID="b03b2355a03dd6578e26e989fa3ba975166ed2d768c8f1ca4112d6a484404acd"
Oct 30 22:20:42.255310 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:20:42.255284    5510 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b03b2355a03dd6578e26e989fa3ba975166ed2d768c8f1ca4112d6a484404acd\": container with ID starting with b03b2355a03dd6578e26e989fa3ba975166ed2d768c8f1ca4112d6a484404acd not found: ID does not exist" containerID="b03b2355a03dd6578e26e989fa3ba975166ed2d768c8f1ca4112d6a484404acd"
Oct 30 22:20:42.255392 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.255318    5510 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b03b2355a03dd6578e26e989fa3ba975166ed2d768c8f1ca4112d6a484404acd"} err="failed to get container status \"b03b2355a03dd6578e26e989fa3ba975166ed2d768c8f1ca4112d6a484404acd\": rpc error: code = NotFound desc = could not find container \"b03b2355a03dd6578e26e989fa3ba975166ed2d768c8f1ca4112d6a484404acd\": container with ID starting with b03b2355a03dd6578e26e989fa3ba975166ed2d768c8f1ca4112d6a484404acd not found: ID does not exist"
Oct 30 22:20:42.255392 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.255341    5510 scope.go:122] "RemoveContainer" containerID="ec2bae482f04cc37a8b77e7640794dd7b79c3183380a1e47d15be53d8bf7a675"
Oct 30 22:20:42.280096 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.280063    5510 scope.go:122] "RemoveContainer" containerID="ec2bae482f04cc37a8b77e7640794dd7b79c3183380a1e47d15be53d8bf7a675"
Oct 30 22:20:42.280497 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.280468    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b0d37b4a-b986-438a-a811-9037505f0e6a-kube-api-access-6nvw5\" (UniqueName: \"kubernetes.io/projected/b0d37b4a-b986-438a-a811-9037505f0e6a-kube-api-access-6nvw5\") pod \"b0d37b4a-b986-438a-a811-9037505f0e6a\" (UID: \"b0d37b4a-b986-438a-a811-9037505f0e6a\") "
Oct 30 22:20:42.280597 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.280530    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6aa08ff6-6f78-4a6e-83bd-07cf5d43b7b0-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6aa08ff6-6f78-4a6e-83bd-07cf5d43b7b0-credential-socket\") pod \"6aa08ff6-6f78-4a6e-83bd-07cf5d43b7b0\" (UID: \"6aa08ff6-6f78-4a6e-83bd-07cf5d43b7b0\") "
Oct 30 22:20:42.280597 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.280587    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/07ed7cb3-6111-4809-8b1f-27a41ff6cafb-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/07ed7cb3-6111-4809-8b1f-27a41ff6cafb-workload-certs\") pod \"07ed7cb3-6111-4809-8b1f-27a41ff6cafb\" (UID: \"07ed7cb3-6111-4809-8b1f-27a41ff6cafb\") "
Oct 30 22:20:42.280987 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.280624    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6aa08ff6-6f78-4a6e-83bd-07cf5d43b7b0-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6aa08ff6-6f78-4a6e-83bd-07cf5d43b7b0-workload-certs\") pod \"6aa08ff6-6f78-4a6e-83bd-07cf5d43b7b0\" (UID: \"6aa08ff6-6f78-4a6e-83bd-07cf5d43b7b0\") "
Oct 30 22:20:42.280987 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.280650    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6aa08ff6-6f78-4a6e-83bd-07cf5d43b7b0-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6aa08ff6-6f78-4a6e-83bd-07cf5d43b7b0-istiod-ca-cert\") pod \"6aa08ff6-6f78-4a6e-83bd-07cf5d43b7b0\" (UID: \"6aa08ff6-6f78-4a6e-83bd-07cf5d43b7b0\") "
Oct 30 22:20:42.280987 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.280768    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/07ed7cb3-6111-4809-8b1f-27a41ff6cafb-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/07ed7cb3-6111-4809-8b1f-27a41ff6cafb-istio-podinfo\") pod \"07ed7cb3-6111-4809-8b1f-27a41ff6cafb\" (UID: \"07ed7cb3-6111-4809-8b1f-27a41ff6cafb\") "
Oct 30 22:20:42.280987 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.280803    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b0d37b4a-b986-438a-a811-9037505f0e6a-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b0d37b4a-b986-438a-a811-9037505f0e6a-istio-envoy\") pod \"b0d37b4a-b986-438a-a811-9037505f0e6a\" (UID: \"b0d37b4a-b986-438a-a811-9037505f0e6a\") "
Oct 30 22:20:42.280987 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.280817    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6aa08ff6-6f78-4a6e-83bd-07cf5d43b7b0-credential-socket" pod "6aa08ff6-6f78-4a6e-83bd-07cf5d43b7b0" (UID: "6aa08ff6-6f78-4a6e-83bd-07cf5d43b7b0"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:20:42.280987 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.280851    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/07ed7cb3-6111-4809-8b1f-27a41ff6cafb-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/07ed7cb3-6111-4809-8b1f-27a41ff6cafb-credential-socket\") pod \"07ed7cb3-6111-4809-8b1f-27a41ff6cafb\" (UID: \"07ed7cb3-6111-4809-8b1f-27a41ff6cafb\") "
Oct 30 22:20:42.280987 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.280889    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b0d37b4a-b986-438a-a811-9037505f0e6a-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0d37b4a-b986-438a-a811-9037505f0e6a-credential-socket\") pod \"b0d37b4a-b986-438a-a811-9037505f0e6a\" (UID: \"b0d37b4a-b986-438a-a811-9037505f0e6a\") "
Oct 30 22:20:42.280987 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.280971    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b0d37b4a-b986-438a-a811-9037505f0e6a-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b0d37b4a-b986-438a-a811-9037505f0e6a-istiod-ca-cert\") pod \"b0d37b4a-b986-438a-a811-9037505f0e6a\" (UID: \"b0d37b4a-b986-438a-a811-9037505f0e6a\") "
Oct 30 22:20:42.281335 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.281026    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/07ed7cb3-6111-4809-8b1f-27a41ff6cafb-kube-api-access-wcj85\" (UniqueName: \"kubernetes.io/projected/07ed7cb3-6111-4809-8b1f-27a41ff6cafb-kube-api-access-wcj85\") pod \"07ed7cb3-6111-4809-8b1f-27a41ff6cafb\" (UID: \"07ed7cb3-6111-4809-8b1f-27a41ff6cafb\") "
Oct 30 22:20:42.281335 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.281078    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6aa08ff6-6f78-4a6e-83bd-07cf5d43b7b0-istio-token\" (UniqueName: \"kubernetes.io/projected/6aa08ff6-6f78-4a6e-83bd-07cf5d43b7b0-istio-token\") pod \"6aa08ff6-6f78-4a6e-83bd-07cf5d43b7b0\" (UID: \"6aa08ff6-6f78-4a6e-83bd-07cf5d43b7b0\") "
Oct 30 22:20:42.281335 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.281104    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6aa08ff6-6f78-4a6e-83bd-07cf5d43b7b0-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6aa08ff6-6f78-4a6e-83bd-07cf5d43b7b0-workload-socket\") pod \"6aa08ff6-6f78-4a6e-83bd-07cf5d43b7b0\" (UID: \"6aa08ff6-6f78-4a6e-83bd-07cf5d43b7b0\") "
Oct 30 22:20:42.281335 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.281126    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6aa08ff6-6f78-4a6e-83bd-07cf5d43b7b0-istiod-ca-cert" pod "6aa08ff6-6f78-4a6e-83bd-07cf5d43b7b0" (UID: "6aa08ff6-6f78-4a6e-83bd-07cf5d43b7b0"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:20:42.281335 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.281139    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/07ed7cb3-6111-4809-8b1f-27a41ff6cafb-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/07ed7cb3-6111-4809-8b1f-27a41ff6cafb-workload-socket\") pod \"07ed7cb3-6111-4809-8b1f-27a41ff6cafb\" (UID: \"07ed7cb3-6111-4809-8b1f-27a41ff6cafb\") "
Oct 30 22:20:42.281335 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.281196    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b0d37b4a-b986-438a-a811-9037505f0e6a-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b0d37b4a-b986-438a-a811-9037505f0e6a-workload-certs\") pod \"b0d37b4a-b986-438a-a811-9037505f0e6a\" (UID: \"b0d37b4a-b986-438a-a811-9037505f0e6a\") "
Oct 30 22:20:42.281335 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.281281    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/07ed7cb3-6111-4809-8b1f-27a41ff6cafb-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/07ed7cb3-6111-4809-8b1f-27a41ff6cafb-istiod-ca-cert\") pod \"07ed7cb3-6111-4809-8b1f-27a41ff6cafb\" (UID: \"07ed7cb3-6111-4809-8b1f-27a41ff6cafb\") "
Oct 30 22:20:42.281683 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.281375    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/07ed7cb3-6111-4809-8b1f-27a41ff6cafb-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/07ed7cb3-6111-4809-8b1f-27a41ff6cafb-istio-envoy\") pod \"07ed7cb3-6111-4809-8b1f-27a41ff6cafb\" (UID: \"07ed7cb3-6111-4809-8b1f-27a41ff6cafb\") "
Oct 30 22:20:42.281683 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.281407    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/6aa08ff6-6f78-4a6e-83bd-07cf5d43b7b0-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6aa08ff6-6f78-4a6e-83bd-07cf5d43b7b0-istio-podinfo\") pod \"6aa08ff6-6f78-4a6e-83bd-07cf5d43b7b0\" (UID: \"6aa08ff6-6f78-4a6e-83bd-07cf5d43b7b0\") "
Oct 30 22:20:42.281683 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.281460    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6aa08ff6-6f78-4a6e-83bd-07cf5d43b7b0-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6aa08ff6-6f78-4a6e-83bd-07cf5d43b7b0-istio-envoy\") pod \"6aa08ff6-6f78-4a6e-83bd-07cf5d43b7b0\" (UID: \"6aa08ff6-6f78-4a6e-83bd-07cf5d43b7b0\") "
Oct 30 22:20:42.281683 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.281505    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6aa08ff6-6f78-4a6e-83bd-07cf5d43b7b0-kube-api-access-5jwbx\" (UniqueName: \"kubernetes.io/projected/6aa08ff6-6f78-4a6e-83bd-07cf5d43b7b0-kube-api-access-5jwbx\") pod \"6aa08ff6-6f78-4a6e-83bd-07cf5d43b7b0\" (UID: \"6aa08ff6-6f78-4a6e-83bd-07cf5d43b7b0\") "
Oct 30 22:20:42.281683 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.281536    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/b0d37b4a-b986-438a-a811-9037505f0e6a-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b0d37b4a-b986-438a-a811-9037505f0e6a-istio-podinfo\") pod \"b0d37b4a-b986-438a-a811-9037505f0e6a\" (UID: \"b0d37b4a-b986-438a-a811-9037505f0e6a\") "
Oct 30 22:20:42.281683 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.281554    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b0d37b4a-b986-438a-a811-9037505f0e6a-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0d37b4a-b986-438a-a811-9037505f0e6a-workload-socket\") pod \"b0d37b4a-b986-438a-a811-9037505f0e6a\" (UID: \"b0d37b4a-b986-438a-a811-9037505f0e6a\") "
Oct 30 22:20:42.282143 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.281683    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/07ed7cb3-6111-4809-8b1f-27a41ff6cafb-istio-token\" (UniqueName: \"kubernetes.io/projected/07ed7cb3-6111-4809-8b1f-27a41ff6cafb-istio-token\") pod \"07ed7cb3-6111-4809-8b1f-27a41ff6cafb\" (UID: \"07ed7cb3-6111-4809-8b1f-27a41ff6cafb\") "
Oct 30 22:20:42.282143 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.281714    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b0d37b4a-b986-438a-a811-9037505f0e6a-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b0d37b4a-b986-438a-a811-9037505f0e6a-istio-data\") pod \"b0d37b4a-b986-438a-a811-9037505f0e6a\" (UID: \"b0d37b4a-b986-438a-a811-9037505f0e6a\") "
Oct 30 22:20:42.282143 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.281798    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b0d37b4a-b986-438a-a811-9037505f0e6a-istio-token\" (UniqueName: \"kubernetes.io/projected/b0d37b4a-b986-438a-a811-9037505f0e6a-istio-token\") pod \"b0d37b4a-b986-438a-a811-9037505f0e6a\" (UID: \"b0d37b4a-b986-438a-a811-9037505f0e6a\") "
Oct 30 22:20:42.282143 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.281842    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07ed7cb3-6111-4809-8b1f-27a41ff6cafb-credential-socket" pod "07ed7cb3-6111-4809-8b1f-27a41ff6cafb" (UID: "07ed7cb3-6111-4809-8b1f-27a41ff6cafb"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:20:42.282143 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.281856    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/07ed7cb3-6111-4809-8b1f-27a41ff6cafb-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/07ed7cb3-6111-4809-8b1f-27a41ff6cafb-istio-data\") pod \"07ed7cb3-6111-4809-8b1f-27a41ff6cafb\" (UID: \"07ed7cb3-6111-4809-8b1f-27a41ff6cafb\") "
Oct 30 22:20:42.282143 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.281899    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6aa08ff6-6f78-4a6e-83bd-07cf5d43b7b0-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6aa08ff6-6f78-4a6e-83bd-07cf5d43b7b0-istio-data\") pod \"6aa08ff6-6f78-4a6e-83bd-07cf5d43b7b0\" (UID: \"6aa08ff6-6f78-4a6e-83bd-07cf5d43b7b0\") "
Oct 30 22:20:42.282143 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:20:42.282123    5510 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ec2bae482f04cc37a8b77e7640794dd7b79c3183380a1e47d15be53d8bf7a675\": container with ID starting with ec2bae482f04cc37a8b77e7640794dd7b79c3183380a1e47d15be53d8bf7a675 not found: ID does not exist" containerID="ec2bae482f04cc37a8b77e7640794dd7b79c3183380a1e47d15be53d8bf7a675"
Oct 30 22:20:42.283962 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.282162    5510 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ec2bae482f04cc37a8b77e7640794dd7b79c3183380a1e47d15be53d8bf7a675"} err="failed to get container status \"ec2bae482f04cc37a8b77e7640794dd7b79c3183380a1e47d15be53d8bf7a675\": rpc error: code = NotFound desc = could not find container \"ec2bae482f04cc37a8b77e7640794dd7b79c3183380a1e47d15be53d8bf7a675\": container with ID starting with ec2bae482f04cc37a8b77e7640794dd7b79c3183380a1e47d15be53d8bf7a675 not found: ID does not exist"
Oct 30 22:20:42.283962 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.282206    5510 scope.go:122] "RemoveContainer" containerID="c6b85d95983fbf83ac908236d7f59308ddfbd8918e9314137276f3ecd11bfce6"
Oct 30 22:20:42.283962 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.282843    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6aa08ff6-6f78-4a6e-83bd-07cf5d43b7b0-workload-certs" pod "6aa08ff6-6f78-4a6e-83bd-07cf5d43b7b0" (UID: "6aa08ff6-6f78-4a6e-83bd-07cf5d43b7b0"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:20:42.283962 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.282887    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07ed7cb3-6111-4809-8b1f-27a41ff6cafb-workload-certs" pod "07ed7cb3-6111-4809-8b1f-27a41ff6cafb" (UID: "07ed7cb3-6111-4809-8b1f-27a41ff6cafb"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:20:42.283962 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.282974    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6aa08ff6-6f78-4a6e-83bd-07cf5d43b7b0-workload-socket" pod "6aa08ff6-6f78-4a6e-83bd-07cf5d43b7b0" (UID: "6aa08ff6-6f78-4a6e-83bd-07cf5d43b7b0"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:20:42.283962 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.283067    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6aa08ff6-6f78-4a6e-83bd-07cf5d43b7b0-istio-data" pod "6aa08ff6-6f78-4a6e-83bd-07cf5d43b7b0" (UID: "6aa08ff6-6f78-4a6e-83bd-07cf5d43b7b0"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:20:42.283962 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.283096    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b0d37b4a-b986-438a-a811-9037505f0e6a-istiod-ca-cert" pod "b0d37b4a-b986-438a-a811-9037505f0e6a" (UID: "b0d37b4a-b986-438a-a811-9037505f0e6a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:20:42.283962 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.283252    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b0d37b4a-b986-438a-a811-9037505f0e6a-istio-data" pod "b0d37b4a-b986-438a-a811-9037505f0e6a" (UID: "b0d37b4a-b986-438a-a811-9037505f0e6a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:20:42.283962 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.283273    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b0d37b4a-b986-438a-a811-9037505f0e6a-workload-certs" pod "b0d37b4a-b986-438a-a811-9037505f0e6a" (UID: "b0d37b4a-b986-438a-a811-9037505f0e6a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:20:42.283962 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.283408    5510 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6cbfd1c6-ca09-4bee-a6d3-6461eac2c2d5-workload-socket\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:20:42.283962 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.283425    5510 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6aa08ff6-6f78-4a6e-83bd-07cf5d43b7b0-credential-socket\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:20:42.283962 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.283439    5510 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/07ed7cb3-6111-4809-8b1f-27a41ff6cafb-workload-certs\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:20:42.283962 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.283454    5510 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6aa08ff6-6f78-4a6e-83bd-07cf5d43b7b0-workload-certs\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:20:42.283962 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.283468    5510 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6aa08ff6-6f78-4a6e-83bd-07cf5d43b7b0-istiod-ca-cert\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:20:42.283962 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.283483    5510 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/07ed7cb3-6111-4809-8b1f-27a41ff6cafb-credential-socket\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:20:42.283962 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.283498    5510 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6cbfd1c6-ca09-4bee-a6d3-6461eac2c2d5-istio-podinfo\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:20:42.283962 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.283512    5510 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b0d37b4a-b986-438a-a811-9037505f0e6a-istiod-ca-cert\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:20:42.283962 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.283526    5510 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6aa08ff6-6f78-4a6e-83bd-07cf5d43b7b0-workload-socket\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:20:42.283962 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.283541    5510 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b0d37b4a-b986-438a-a811-9037505f0e6a-workload-certs\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:20:42.285220 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.285133    5510 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6cbfd1c6-ca09-4bee-a6d3-6461eac2c2d5-workload-certs\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:20:42.285220 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.284255    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b0d37b4a-b986-438a-a811-9037505f0e6a-workload-socket" pod "b0d37b4a-b986-438a-a811-9037505f0e6a" (UID: "b0d37b4a-b986-438a-a811-9037505f0e6a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:20:42.285220 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.284315    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/07ed7cb3-6111-4809-8b1f-27a41ff6cafb-istio-podinfo" pod "07ed7cb3-6111-4809-8b1f-27a41ff6cafb" (UID: "07ed7cb3-6111-4809-8b1f-27a41ff6cafb"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:20:42.285220 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.284773    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/07ed7cb3-6111-4809-8b1f-27a41ff6cafb-istiod-ca-cert" pod "07ed7cb3-6111-4809-8b1f-27a41ff6cafb" (UID: "07ed7cb3-6111-4809-8b1f-27a41ff6cafb"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:20:42.285428 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.285094    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07ed7cb3-6111-4809-8b1f-27a41ff6cafb-istio-data" pod "07ed7cb3-6111-4809-8b1f-27a41ff6cafb" (UID: "07ed7cb3-6111-4809-8b1f-27a41ff6cafb"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:20:42.285428 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.285153    5510 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6cbfd1c6-ca09-4bee-a6d3-6461eac2c2d5-istio-envoy\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:20:42.285428 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.285269    5510 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v6nv7\" (UniqueName: \"kubernetes.io/projected/6cbfd1c6-ca09-4bee-a6d3-6461eac2c2d5-kube-api-access-v6nv7\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:20:42.285428 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.285289    5510 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6cbfd1c6-ca09-4bee-a6d3-6461eac2c2d5-credential-socket\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:20:42.285428 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.285303    5510 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b0d37b4a-b986-438a-a811-9037505f0e6a-istio-data\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:20:42.285428 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.285319    5510 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6cbfd1c6-ca09-4bee-a6d3-6461eac2c2d5-istiod-ca-cert\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:20:42.285428 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.285335    5510 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6cbfd1c6-ca09-4bee-a6d3-6461eac2c2d5-istio-data\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:20:42.285428 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.285348    5510 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6aa08ff6-6f78-4a6e-83bd-07cf5d43b7b0-istio-data\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:20:42.285428 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.285363    5510 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6cbfd1c6-ca09-4bee-a6d3-6461eac2c2d5-istio-token\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:20:42.285428 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.285384    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b0d37b4a-b986-438a-a811-9037505f0e6a-credential-socket" pod "b0d37b4a-b986-438a-a811-9037505f0e6a" (UID: "b0d37b4a-b986-438a-a811-9037505f0e6a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:20:42.288011 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.287982    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07ed7cb3-6111-4809-8b1f-27a41ff6cafb-workload-socket" pod "07ed7cb3-6111-4809-8b1f-27a41ff6cafb" (UID: "07ed7cb3-6111-4809-8b1f-27a41ff6cafb"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:20:42.290744 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.290673    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/07ed7cb3-6111-4809-8b1f-27a41ff6cafb-istio-token" pod "07ed7cb3-6111-4809-8b1f-27a41ff6cafb" (UID: "07ed7cb3-6111-4809-8b1f-27a41ff6cafb"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:20:42.290744 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.290731    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6aa08ff6-6f78-4a6e-83bd-07cf5d43b7b0-istio-token" pod "6aa08ff6-6f78-4a6e-83bd-07cf5d43b7b0" (UID: "6aa08ff6-6f78-4a6e-83bd-07cf5d43b7b0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:20:42.290903 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.290793    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b0d37b4a-b986-438a-a811-9037505f0e6a-istio-envoy" pod "b0d37b4a-b986-438a-a811-9037505f0e6a" (UID: "b0d37b4a-b986-438a-a811-9037505f0e6a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:20:42.291102 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.291077    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b0d37b4a-b986-438a-a811-9037505f0e6a-istio-token" pod "b0d37b4a-b986-438a-a811-9037505f0e6a" (UID: "b0d37b4a-b986-438a-a811-9037505f0e6a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:20:42.291102 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.291088    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6aa08ff6-6f78-4a6e-83bd-07cf5d43b7b0-istio-envoy" pod "6aa08ff6-6f78-4a6e-83bd-07cf5d43b7b0" (UID: "6aa08ff6-6f78-4a6e-83bd-07cf5d43b7b0"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:20:42.291259 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.291183    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6aa08ff6-6f78-4a6e-83bd-07cf5d43b7b0-kube-api-access-5jwbx" pod "6aa08ff6-6f78-4a6e-83bd-07cf5d43b7b0" (UID: "6aa08ff6-6f78-4a6e-83bd-07cf5d43b7b0"). InnerVolumeSpecName "kube-api-access-5jwbx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:20:42.291363 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.291341    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b0d37b4a-b986-438a-a811-9037505f0e6a-kube-api-access-6nvw5" pod "b0d37b4a-b986-438a-a811-9037505f0e6a" (UID: "b0d37b4a-b986-438a-a811-9037505f0e6a"). InnerVolumeSpecName "kube-api-access-6nvw5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:20:42.291512 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.291491    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b0d37b4a-b986-438a-a811-9037505f0e6a-istio-podinfo" pod "b0d37b4a-b986-438a-a811-9037505f0e6a" (UID: "b0d37b4a-b986-438a-a811-9037505f0e6a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:20:42.291953 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.291928    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/07ed7cb3-6111-4809-8b1f-27a41ff6cafb-kube-api-access-wcj85" pod "07ed7cb3-6111-4809-8b1f-27a41ff6cafb" (UID: "07ed7cb3-6111-4809-8b1f-27a41ff6cafb"). InnerVolumeSpecName "kube-api-access-wcj85". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:20:42.292342 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.292286    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6aa08ff6-6f78-4a6e-83bd-07cf5d43b7b0-istio-podinfo" pod "6aa08ff6-6f78-4a6e-83bd-07cf5d43b7b0" (UID: "6aa08ff6-6f78-4a6e-83bd-07cf5d43b7b0"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:20:42.292973 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.292949    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07ed7cb3-6111-4809-8b1f-27a41ff6cafb-istio-envoy" pod "07ed7cb3-6111-4809-8b1f-27a41ff6cafb" (UID: "07ed7cb3-6111-4809-8b1f-27a41ff6cafb"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:20:42.299259 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.299242    5510 scope.go:122] "RemoveContainer" containerID="c6b85d95983fbf83ac908236d7f59308ddfbd8918e9314137276f3ecd11bfce6"
Oct 30 22:20:42.299541 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:20:42.299502    5510 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c6b85d95983fbf83ac908236d7f59308ddfbd8918e9314137276f3ecd11bfce6\": container with ID starting with c6b85d95983fbf83ac908236d7f59308ddfbd8918e9314137276f3ecd11bfce6 not found: ID does not exist" containerID="c6b85d95983fbf83ac908236d7f59308ddfbd8918e9314137276f3ecd11bfce6"
Oct 30 22:20:42.299633 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.299541    5510 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c6b85d95983fbf83ac908236d7f59308ddfbd8918e9314137276f3ecd11bfce6"} err="failed to get container status \"c6b85d95983fbf83ac908236d7f59308ddfbd8918e9314137276f3ecd11bfce6\": rpc error: code = NotFound desc = could not find container \"c6b85d95983fbf83ac908236d7f59308ddfbd8918e9314137276f3ecd11bfce6\": container with ID starting with c6b85d95983fbf83ac908236d7f59308ddfbd8918e9314137276f3ecd11bfce6 not found: ID does not exist"
Oct 30 22:20:42.299633 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.299566    5510 scope.go:122] "RemoveContainer" containerID="32d2dbdde860ac1cc4f5ac866be960f485032f283e065535f10556e82ee3d659"
Oct 30 22:20:42.313858 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.313822    5510 scope.go:122] "RemoveContainer" containerID="a24ba72543ab31aeb5b19f93d3fba849190ec169d677ec88fd48df2f4f237693"
Oct 30 22:20:42.323964 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.323941    5510 scope.go:122] "RemoveContainer" containerID="32d2dbdde860ac1cc4f5ac866be960f485032f283e065535f10556e82ee3d659"
Oct 30 22:20:42.324311 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:20:42.324289    5510 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"32d2dbdde860ac1cc4f5ac866be960f485032f283e065535f10556e82ee3d659\": container with ID starting with 32d2dbdde860ac1cc4f5ac866be960f485032f283e065535f10556e82ee3d659 not found: ID does not exist" containerID="32d2dbdde860ac1cc4f5ac866be960f485032f283e065535f10556e82ee3d659"
Oct 30 22:20:42.324390 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.324321    5510 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"32d2dbdde860ac1cc4f5ac866be960f485032f283e065535f10556e82ee3d659"} err="failed to get container status \"32d2dbdde860ac1cc4f5ac866be960f485032f283e065535f10556e82ee3d659\": rpc error: code = NotFound desc = could not find container \"32d2dbdde860ac1cc4f5ac866be960f485032f283e065535f10556e82ee3d659\": container with ID starting with 32d2dbdde860ac1cc4f5ac866be960f485032f283e065535f10556e82ee3d659 not found: ID does not exist"
Oct 30 22:20:42.324390 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.324343    5510 scope.go:122] "RemoveContainer" containerID="a24ba72543ab31aeb5b19f93d3fba849190ec169d677ec88fd48df2f4f237693"
Oct 30 22:20:42.324666 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:20:42.324629    5510 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a24ba72543ab31aeb5b19f93d3fba849190ec169d677ec88fd48df2f4f237693\": container with ID starting with a24ba72543ab31aeb5b19f93d3fba849190ec169d677ec88fd48df2f4f237693 not found: ID does not exist" containerID="a24ba72543ab31aeb5b19f93d3fba849190ec169d677ec88fd48df2f4f237693"
Oct 30 22:20:42.324726 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.324673    5510 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a24ba72543ab31aeb5b19f93d3fba849190ec169d677ec88fd48df2f4f237693"} err="failed to get container status \"a24ba72543ab31aeb5b19f93d3fba849190ec169d677ec88fd48df2f4f237693\": rpc error: code = NotFound desc = could not find container \"a24ba72543ab31aeb5b19f93d3fba849190ec169d677ec88fd48df2f4f237693\": container with ID starting with a24ba72543ab31aeb5b19f93d3fba849190ec169d677ec88fd48df2f4f237693 not found: ID does not exist"
Oct 30 22:20:42.324726 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.324689    5510 scope.go:122] "RemoveContainer" containerID="d55780c704afcc7e3d56c850cc0477a6aba36d95498a131e84b5479e7674d50d"
Oct 30 22:20:42.337030 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.337010    5510 scope.go:122] "RemoveContainer" containerID="d55780c704afcc7e3d56c850cc0477a6aba36d95498a131e84b5479e7674d50d"
Oct 30 22:20:42.337513 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:20:42.337486    5510 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d55780c704afcc7e3d56c850cc0477a6aba36d95498a131e84b5479e7674d50d\": container with ID starting with d55780c704afcc7e3d56c850cc0477a6aba36d95498a131e84b5479e7674d50d not found: ID does not exist" containerID="d55780c704afcc7e3d56c850cc0477a6aba36d95498a131e84b5479e7674d50d"
Oct 30 22:20:42.337592 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.337523    5510 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d55780c704afcc7e3d56c850cc0477a6aba36d95498a131e84b5479e7674d50d"} err="failed to get container status \"d55780c704afcc7e3d56c850cc0477a6aba36d95498a131e84b5479e7674d50d\": rpc error: code = NotFound desc = could not find container \"d55780c704afcc7e3d56c850cc0477a6aba36d95498a131e84b5479e7674d50d\": container with ID starting with d55780c704afcc7e3d56c850cc0477a6aba36d95498a131e84b5479e7674d50d not found: ID does not exist"
Oct 30 22:20:42.385958 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.385871    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/5542b62f-bc73-47d7-8615-180f43324911-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5542b62f-bc73-47d7-8615-180f43324911-istio-podinfo\") pod \"5542b62f-bc73-47d7-8615-180f43324911\" (UID: \"5542b62f-bc73-47d7-8615-180f43324911\") "
Oct 30 22:20:42.385958 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.385907    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5542b62f-bc73-47d7-8615-180f43324911-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5542b62f-bc73-47d7-8615-180f43324911-istio-envoy\") pod \"5542b62f-bc73-47d7-8615-180f43324911\" (UID: \"5542b62f-bc73-47d7-8615-180f43324911\") "
Oct 30 22:20:42.386239 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.386207    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5542b62f-bc73-47d7-8615-180f43324911-istio-token\" (UniqueName: \"kubernetes.io/projected/5542b62f-bc73-47d7-8615-180f43324911-istio-token\") pod \"5542b62f-bc73-47d7-8615-180f43324911\" (UID: \"5542b62f-bc73-47d7-8615-180f43324911\") "
Oct 30 22:20:42.386376 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.386271    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5542b62f-bc73-47d7-8615-180f43324911-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5542b62f-bc73-47d7-8615-180f43324911-istiod-ca-cert\") pod \"5542b62f-bc73-47d7-8615-180f43324911\" (UID: \"5542b62f-bc73-47d7-8615-180f43324911\") "
Oct 30 22:20:42.386376 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.386301    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5542b62f-bc73-47d7-8615-180f43324911-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5542b62f-bc73-47d7-8615-180f43324911-credential-socket\") pod \"5542b62f-bc73-47d7-8615-180f43324911\" (UID: \"5542b62f-bc73-47d7-8615-180f43324911\") "
Oct 30 22:20:42.386497 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.386409    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5542b62f-bc73-47d7-8615-180f43324911-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5542b62f-bc73-47d7-8615-180f43324911-workload-socket\") pod \"5542b62f-bc73-47d7-8615-180f43324911\" (UID: \"5542b62f-bc73-47d7-8615-180f43324911\") "
Oct 30 22:20:42.386497 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.386452    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5542b62f-bc73-47d7-8615-180f43324911-kube-api-access-xhh86\" (UniqueName: \"kubernetes.io/projected/5542b62f-bc73-47d7-8615-180f43324911-kube-api-access-xhh86\") pod \"5542b62f-bc73-47d7-8615-180f43324911\" (UID: \"5542b62f-bc73-47d7-8615-180f43324911\") "
Oct 30 22:20:42.386607 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.386532    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5542b62f-bc73-47d7-8615-180f43324911-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5542b62f-bc73-47d7-8615-180f43324911-istio-data\") pod \"5542b62f-bc73-47d7-8615-180f43324911\" (UID: \"5542b62f-bc73-47d7-8615-180f43324911\") "
Oct 30 22:20:42.386607 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.386560    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5542b62f-bc73-47d7-8615-180f43324911-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5542b62f-bc73-47d7-8615-180f43324911-workload-certs\") pod \"5542b62f-bc73-47d7-8615-180f43324911\" (UID: \"5542b62f-bc73-47d7-8615-180f43324911\") "
Oct 30 22:20:42.386734 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.386627    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5542b62f-bc73-47d7-8615-180f43324911-credential-socket" pod "5542b62f-bc73-47d7-8615-180f43324911" (UID: "5542b62f-bc73-47d7-8615-180f43324911"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:20:42.386800 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.386740    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5542b62f-bc73-47d7-8615-180f43324911-istiod-ca-cert" pod "5542b62f-bc73-47d7-8615-180f43324911" (UID: "5542b62f-bc73-47d7-8615-180f43324911"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:20:42.387861 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.387002    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5542b62f-bc73-47d7-8615-180f43324911-istio-data" pod "5542b62f-bc73-47d7-8615-180f43324911" (UID: "5542b62f-bc73-47d7-8615-180f43324911"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:20:42.387861 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.387078    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5542b62f-bc73-47d7-8615-180f43324911-workload-socket" pod "5542b62f-bc73-47d7-8615-180f43324911" (UID: "5542b62f-bc73-47d7-8615-180f43324911"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:20:42.387861 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.387223    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5542b62f-bc73-47d7-8615-180f43324911-workload-certs" pod "5542b62f-bc73-47d7-8615-180f43324911" (UID: "5542b62f-bc73-47d7-8615-180f43324911"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:20:42.387861 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.387496    5510 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6nvw5\" (UniqueName: \"kubernetes.io/projected/b0d37b4a-b986-438a-a811-9037505f0e6a-kube-api-access-6nvw5\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:20:42.387861 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.387521    5510 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5542b62f-bc73-47d7-8615-180f43324911-istiod-ca-cert\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:20:42.387861 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.387536    5510 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5542b62f-bc73-47d7-8615-180f43324911-credential-socket\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:20:42.387861 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.387550    5510 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5542b62f-bc73-47d7-8615-180f43324911-workload-socket\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:20:42.387861 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.387564    5510 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/07ed7cb3-6111-4809-8b1f-27a41ff6cafb-istio-podinfo\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:20:42.387861 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.387577    5510 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b0d37b4a-b986-438a-a811-9037505f0e6a-istio-envoy\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:20:42.387861 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.387591    5510 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0d37b4a-b986-438a-a811-9037505f0e6a-credential-socket\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:20:42.387861 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.387604    5510 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5542b62f-bc73-47d7-8615-180f43324911-istio-data\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:20:42.387861 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.387618    5510 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5542b62f-bc73-47d7-8615-180f43324911-workload-certs\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:20:42.387861 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.387632    5510 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wcj85\" (UniqueName: \"kubernetes.io/projected/07ed7cb3-6111-4809-8b1f-27a41ff6cafb-kube-api-access-wcj85\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:20:42.387861 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.387646    5510 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6aa08ff6-6f78-4a6e-83bd-07cf5d43b7b0-istio-token\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:20:42.387861 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.387699    5510 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/07ed7cb3-6111-4809-8b1f-27a41ff6cafb-workload-socket\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:20:42.387861 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.387714    5510 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/07ed7cb3-6111-4809-8b1f-27a41ff6cafb-istiod-ca-cert\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:20:42.387861 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.387728    5510 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/07ed7cb3-6111-4809-8b1f-27a41ff6cafb-istio-envoy\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:20:42.387861 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.387741    5510 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6aa08ff6-6f78-4a6e-83bd-07cf5d43b7b0-istio-podinfo\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:20:42.387861 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.387754    5510 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6aa08ff6-6f78-4a6e-83bd-07cf5d43b7b0-istio-envoy\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:20:42.387861 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.387770    5510 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5jwbx\" (UniqueName: \"kubernetes.io/projected/6aa08ff6-6f78-4a6e-83bd-07cf5d43b7b0-kube-api-access-5jwbx\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:20:42.387861 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.387784    5510 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b0d37b4a-b986-438a-a811-9037505f0e6a-istio-podinfo\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:20:42.387861 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.387798    5510 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0d37b4a-b986-438a-a811-9037505f0e6a-workload-socket\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:20:42.387861 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.387812    5510 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/07ed7cb3-6111-4809-8b1f-27a41ff6cafb-istio-token\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:20:42.387861 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.387825    5510 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b0d37b4a-b986-438a-a811-9037505f0e6a-istio-token\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:20:42.387861 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.387838    5510 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/07ed7cb3-6111-4809-8b1f-27a41ff6cafb-istio-data\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:20:42.390039 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.389277    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5542b62f-bc73-47d7-8615-180f43324911-istio-podinfo" pod "5542b62f-bc73-47d7-8615-180f43324911" (UID: "5542b62f-bc73-47d7-8615-180f43324911"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:20:42.390326 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.390298    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5542b62f-bc73-47d7-8615-180f43324911-istio-envoy" pod "5542b62f-bc73-47d7-8615-180f43324911" (UID: "5542b62f-bc73-47d7-8615-180f43324911"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:20:42.390494 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.390473    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5542b62f-bc73-47d7-8615-180f43324911-kube-api-access-xhh86" pod "5542b62f-bc73-47d7-8615-180f43324911" (UID: "5542b62f-bc73-47d7-8615-180f43324911"). InnerVolumeSpecName "kube-api-access-xhh86". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:20:42.390633 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.390601    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5542b62f-bc73-47d7-8615-180f43324911-istio-token" pod "5542b62f-bc73-47d7-8615-180f43324911" (UID: "5542b62f-bc73-47d7-8615-180f43324911"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:20:42.489569 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.489530    5510 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5542b62f-bc73-47d7-8615-180f43324911-istio-token\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:20:42.489569 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.489573    5510 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xhh86\" (UniqueName: \"kubernetes.io/projected/5542b62f-bc73-47d7-8615-180f43324911-kube-api-access-xhh86\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:20:42.489855 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.489589    5510 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5542b62f-bc73-47d7-8615-180f43324911-istio-podinfo\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:20:42.489855 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.489606    5510 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5542b62f-bc73-47d7-8615-180f43324911-istio-envoy\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:20:42.805269 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.805227    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6mvrk"]
Oct 30 22:20:42.853804 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:42.853773    5510 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6mvrk"]
Oct 30 22:20:43.030570 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:43.030529    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-v4cxn"]
Oct 30 22:20:43.047191 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:43.047156    5510 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-v4cxn"]
Oct 30 22:20:43.153507 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:43.153456    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-f969x"]
Oct 30 22:20:43.160576 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:43.160544    5510 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-f969x"]
Oct 30 22:20:43.263766 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:43.263734    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6fvq6"]
Oct 30 22:20:43.264632 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:43.264608    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-w8zj5"
Oct 30 22:20:43.267694 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:43.267632    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-ed44da-9b6f6d88c-jz6p9" event={"ID":"f8bd673e-4938-421a-bb40-5eb3e7f7802f","Type":"ContainerStarted","Data":"0bef431ef8a859e9eb41393ac8a3f93b9b6541c0f05e20fdee02003361424a70"}
Oct 30 22:20:43.267937 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:43.267875    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-ed44da-9b6f6d88c-jz6p9"
Oct 30 22:20:43.274398 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:43.274375    5510 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-6fvq6"]
Oct 30 22:20:43.375813 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:43.375731    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-48cnj"]
Oct 30 22:20:43.384699 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:43.384632    5510 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-48cnj"]
Oct 30 22:20:43.490457 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:43.490422    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-w8zj5"]
Oct 30 22:20:43.495586 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:43.495556    5510 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-w8zj5"]
Oct 30 22:20:43.578689 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:43.560168    5510 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="07ed7cb3-6111-4809-8b1f-27a41ff6cafb" path="/var/lib/kubelet/pods/07ed7cb3-6111-4809-8b1f-27a41ff6cafb/volumes"
Oct 30 22:20:43.578689 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:43.561283    5510 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5542b62f-bc73-47d7-8615-180f43324911" path="/var/lib/kubelet/pods/5542b62f-bc73-47d7-8615-180f43324911/volumes"
Oct 30 22:20:43.578689 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:43.562216    5510 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5ff054ea-b27c-4c64-be2d-c645c0ca3d24" path="/var/lib/kubelet/pods/5ff054ea-b27c-4c64-be2d-c645c0ca3d24/volumes"
Oct 30 22:20:43.578689 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:43.565151    5510 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6aa08ff6-6f78-4a6e-83bd-07cf5d43b7b0" path="/var/lib/kubelet/pods/6aa08ff6-6f78-4a6e-83bd-07cf5d43b7b0/volumes"
Oct 30 22:20:43.578689 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:43.566175    5510 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6cbfd1c6-ca09-4bee-a6d3-6461eac2c2d5" path="/var/lib/kubelet/pods/6cbfd1c6-ca09-4bee-a6d3-6461eac2c2d5/volumes"
Oct 30 22:20:43.578689 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:43.567142    5510 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b0d37b4a-b986-438a-a811-9037505f0e6a" path="/var/lib/kubelet/pods/b0d37b4a-b986-438a-a811-9037505f0e6a/volumes"
Oct 30 22:20:43.731959 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:43.731870    5510 patch_prober.go:29] interesting pod/e2e-derive-bee2db53-openshift-default-6b56b8cdb9-jv7mv container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 22:20:43.731959 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:43.731941    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-derive-bee2db53-openshift-default-6b56b8cdb9-jv7mv" podUID="01891eb3-3336-4e31-a26a-9a1ecb935f9c" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 22:20:44.129274 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:44.129248    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4da07739-5dcb995df-m8gbm"
Oct 30 22:20:44.249357 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:44.249303    5510 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-ed44da-9b6f6d88c-jz6p9" podStartSLOduration=3.129427701 podStartE2EDuration="4.249287687s" totalImagesPullingTime="1.119859986s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:20:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:20:43.630700124 +0000 UTC m=+1824.191510130" watchObservedRunningTime="2026-10-30 22:20:44.249287687 +0000 UTC m=+1824.810097694"
Oct 30 22:20:45.914894 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:45.912873    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:20:46.182787 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:46.182759    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-bee2db53-openshift-default-6b56b8cdb9-jv7mv"
Oct 30 22:20:46.292551 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:46.292515    5510 generic.go:356] "Generic (PLEG): container finished" podID="01891eb3-3336-4e31-a26a-9a1ecb935f9c" containerID="bc8c3d1c2035a325ec675005f3202f183bae9d45f8881edd84ec3f62791803df" exitCode=0
Oct 30 22:20:46.292750 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:46.292561    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-bee2db53-openshift-default-6b56b8cdb9-jv7mv"
Oct 30 22:20:46.292750 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:46.292590    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-bee2db53-openshift-default-6b56b8cdb9-jv7mv" event={"ID":"01891eb3-3336-4e31-a26a-9a1ecb935f9c","Type":"ContainerDied","Data":"bc8c3d1c2035a325ec675005f3202f183bae9d45f8881edd84ec3f62791803df"}
Oct 30 22:20:46.292750 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:46.292633    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-bee2db53-openshift-default-6b56b8cdb9-jv7mv" event={"ID":"01891eb3-3336-4e31-a26a-9a1ecb935f9c","Type":"ContainerDied","Data":"62dd22d27a87229f702e34b1cebbfb63f2f8f55e7a892ae634ac637937a8ec18"}
Oct 30 22:20:46.292750 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:46.292690    5510 scope.go:122] "RemoveContainer" containerID="bc8c3d1c2035a325ec675005f3202f183bae9d45f8881edd84ec3f62791803df"
Oct 30 22:20:46.305887 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:46.305860    5510 scope.go:122] "RemoveContainer" containerID="15c70c82c935624bfc0094fd21db370e95de86bdda73f8cb176f92455e7fd968"
Oct 30 22:20:46.316854 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:46.316830    5510 scope.go:122] "RemoveContainer" containerID="bc8c3d1c2035a325ec675005f3202f183bae9d45f8881edd84ec3f62791803df"
Oct 30 22:20:46.317160 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:20:46.317138    5510 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bc8c3d1c2035a325ec675005f3202f183bae9d45f8881edd84ec3f62791803df\": container with ID starting with bc8c3d1c2035a325ec675005f3202f183bae9d45f8881edd84ec3f62791803df not found: ID does not exist" containerID="bc8c3d1c2035a325ec675005f3202f183bae9d45f8881edd84ec3f62791803df"
Oct 30 22:20:46.317264 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:46.317174    5510 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bc8c3d1c2035a325ec675005f3202f183bae9d45f8881edd84ec3f62791803df"} err="failed to get container status \"bc8c3d1c2035a325ec675005f3202f183bae9d45f8881edd84ec3f62791803df\": rpc error: code = NotFound desc = could not find container \"bc8c3d1c2035a325ec675005f3202f183bae9d45f8881edd84ec3f62791803df\": container with ID starting with bc8c3d1c2035a325ec675005f3202f183bae9d45f8881edd84ec3f62791803df not found: ID does not exist"
Oct 30 22:20:46.317264 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:46.317199    5510 scope.go:122] "RemoveContainer" containerID="15c70c82c935624bfc0094fd21db370e95de86bdda73f8cb176f92455e7fd968"
Oct 30 22:20:46.317536 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:20:46.317514    5510 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"15c70c82c935624bfc0094fd21db370e95de86bdda73f8cb176f92455e7fd968\": container with ID starting with 15c70c82c935624bfc0094fd21db370e95de86bdda73f8cb176f92455e7fd968 not found: ID does not exist" containerID="15c70c82c935624bfc0094fd21db370e95de86bdda73f8cb176f92455e7fd968"
Oct 30 22:20:46.317618 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:46.317544    5510 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"15c70c82c935624bfc0094fd21db370e95de86bdda73f8cb176f92455e7fd968"} err="failed to get container status \"15c70c82c935624bfc0094fd21db370e95de86bdda73f8cb176f92455e7fd968\": rpc error: code = NotFound desc = could not find container \"15c70c82c935624bfc0094fd21db370e95de86bdda73f8cb176f92455e7fd968\": container with ID starting with 15c70c82c935624bfc0094fd21db370e95de86bdda73f8cb176f92455e7fd968 not found: ID does not exist"
Oct 30 22:20:46.355534 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:46.355503    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/01891eb3-3336-4e31-a26a-9a1ecb935f9c-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/01891eb3-3336-4e31-a26a-9a1ecb935f9c-workload-socket\") pod \"01891eb3-3336-4e31-a26a-9a1ecb935f9c\" (UID: \"01891eb3-3336-4e31-a26a-9a1ecb935f9c\") "
Oct 30 22:20:46.355726 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:46.355551    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/01891eb3-3336-4e31-a26a-9a1ecb935f9c-istio-token\" (UniqueName: \"kubernetes.io/projected/01891eb3-3336-4e31-a26a-9a1ecb935f9c-istio-token\") pod \"01891eb3-3336-4e31-a26a-9a1ecb935f9c\" (UID: \"01891eb3-3336-4e31-a26a-9a1ecb935f9c\") "
Oct 30 22:20:46.355726 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:46.355581    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/01891eb3-3336-4e31-a26a-9a1ecb935f9c-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/01891eb3-3336-4e31-a26a-9a1ecb935f9c-istiod-ca-cert\") pod \"01891eb3-3336-4e31-a26a-9a1ecb935f9c\" (UID: \"01891eb3-3336-4e31-a26a-9a1ecb935f9c\") "
Oct 30 22:20:46.355861 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:46.355735    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/01891eb3-3336-4e31-a26a-9a1ecb935f9c-kube-api-access-qmnpj\" (UniqueName: \"kubernetes.io/projected/01891eb3-3336-4e31-a26a-9a1ecb935f9c-kube-api-access-qmnpj\") pod \"01891eb3-3336-4e31-a26a-9a1ecb935f9c\" (UID: \"01891eb3-3336-4e31-a26a-9a1ecb935f9c\") "
Oct 30 22:20:46.355861 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:46.355790    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/01891eb3-3336-4e31-a26a-9a1ecb935f9c-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/01891eb3-3336-4e31-a26a-9a1ecb935f9c-istio-data\") pod \"01891eb3-3336-4e31-a26a-9a1ecb935f9c\" (UID: \"01891eb3-3336-4e31-a26a-9a1ecb935f9c\") "
Oct 30 22:20:46.355861 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:46.355801    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/01891eb3-3336-4e31-a26a-9a1ecb935f9c-workload-socket" pod "01891eb3-3336-4e31-a26a-9a1ecb935f9c" (UID: "01891eb3-3336-4e31-a26a-9a1ecb935f9c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:20:46.356029 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:46.355910    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/01891eb3-3336-4e31-a26a-9a1ecb935f9c-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/01891eb3-3336-4e31-a26a-9a1ecb935f9c-workload-certs\") pod \"01891eb3-3336-4e31-a26a-9a1ecb935f9c\" (UID: \"01891eb3-3336-4e31-a26a-9a1ecb935f9c\") "
Oct 30 22:20:46.356029 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:46.356001    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/01891eb3-3336-4e31-a26a-9a1ecb935f9c-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/01891eb3-3336-4e31-a26a-9a1ecb935f9c-istio-envoy\") pod \"01891eb3-3336-4e31-a26a-9a1ecb935f9c\" (UID: \"01891eb3-3336-4e31-a26a-9a1ecb935f9c\") "
Oct 30 22:20:46.356127 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:46.356022    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/01891eb3-3336-4e31-a26a-9a1ecb935f9c-istiod-ca-cert" pod "01891eb3-3336-4e31-a26a-9a1ecb935f9c" (UID: "01891eb3-3336-4e31-a26a-9a1ecb935f9c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:20:46.356127 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:46.356060    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/01891eb3-3336-4e31-a26a-9a1ecb935f9c-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/01891eb3-3336-4e31-a26a-9a1ecb935f9c-credential-socket\") pod \"01891eb3-3336-4e31-a26a-9a1ecb935f9c\" (UID: \"01891eb3-3336-4e31-a26a-9a1ecb935f9c\") "
Oct 30 22:20:46.356233 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:46.356119    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/01891eb3-3336-4e31-a26a-9a1ecb935f9c-istio-data" pod "01891eb3-3336-4e31-a26a-9a1ecb935f9c" (UID: "01891eb3-3336-4e31-a26a-9a1ecb935f9c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:20:46.356233 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:46.356193    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/01891eb3-3336-4e31-a26a-9a1ecb935f9c-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/01891eb3-3336-4e31-a26a-9a1ecb935f9c-istio-podinfo\") pod \"01891eb3-3336-4e31-a26a-9a1ecb935f9c\" (UID: \"01891eb3-3336-4e31-a26a-9a1ecb935f9c\") "
Oct 30 22:20:46.356340 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:46.356275    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/01891eb3-3336-4e31-a26a-9a1ecb935f9c-workload-certs" pod "01891eb3-3336-4e31-a26a-9a1ecb935f9c" (UID: "01891eb3-3336-4e31-a26a-9a1ecb935f9c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:20:46.356446 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:46.356427    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/01891eb3-3336-4e31-a26a-9a1ecb935f9c-credential-socket" pod "01891eb3-3336-4e31-a26a-9a1ecb935f9c" (UID: "01891eb3-3336-4e31-a26a-9a1ecb935f9c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:20:46.357055 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:46.357027    5510 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/01891eb3-3336-4e31-a26a-9a1ecb935f9c-workload-certs\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:20:46.357055 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:46.357056    5510 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/01891eb3-3336-4e31-a26a-9a1ecb935f9c-credential-socket\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:20:46.357238 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:46.357068    5510 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/01891eb3-3336-4e31-a26a-9a1ecb935f9c-workload-socket\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:20:46.357238 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:46.357083    5510 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/01891eb3-3336-4e31-a26a-9a1ecb935f9c-istiod-ca-cert\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:20:46.357238 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:46.357095    5510 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/01891eb3-3336-4e31-a26a-9a1ecb935f9c-istio-data\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:20:46.358599 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:46.358571    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/01891eb3-3336-4e31-a26a-9a1ecb935f9c-kube-api-access-qmnpj" pod "01891eb3-3336-4e31-a26a-9a1ecb935f9c" (UID: "01891eb3-3336-4e31-a26a-9a1ecb935f9c"). InnerVolumeSpecName "kube-api-access-qmnpj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:20:46.358772 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:46.358617    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/01891eb3-3336-4e31-a26a-9a1ecb935f9c-istio-token" pod "01891eb3-3336-4e31-a26a-9a1ecb935f9c" (UID: "01891eb3-3336-4e31-a26a-9a1ecb935f9c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:20:46.358897 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:46.358868    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/01891eb3-3336-4e31-a26a-9a1ecb935f9c-istio-podinfo" pod "01891eb3-3336-4e31-a26a-9a1ecb935f9c" (UID: "01891eb3-3336-4e31-a26a-9a1ecb935f9c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:20:46.358953 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:46.358903    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/01891eb3-3336-4e31-a26a-9a1ecb935f9c-istio-envoy" pod "01891eb3-3336-4e31-a26a-9a1ecb935f9c" (UID: "01891eb3-3336-4e31-a26a-9a1ecb935f9c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:20:46.458218 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:46.458140    5510 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qmnpj\" (UniqueName: \"kubernetes.io/projected/01891eb3-3336-4e31-a26a-9a1ecb935f9c-kube-api-access-qmnpj\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:20:46.458218 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:46.458170    5510 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/01891eb3-3336-4e31-a26a-9a1ecb935f9c-istio-envoy\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:20:46.458218 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:46.458181    5510 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/01891eb3-3336-4e31-a26a-9a1ecb935f9c-istio-podinfo\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:20:46.458218 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:46.458190    5510 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/01891eb3-3336-4e31-a26a-9a1ecb935f9c-istio-token\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:20:46.710534 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:46.710453    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-bee2db53-openshift-default-6b56b8cdb9-jv7mv"]
Oct 30 22:20:46.715626 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:46.715590    5510 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-bee2db53-openshift-default-6b56b8cdb9-jv7mv"]
Oct 30 22:20:47.554495 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:47.554455    5510 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="01891eb3-3336-4e31-a26a-9a1ecb935f9c" path="/var/lib/kubelet/pods/01891eb3-3336-4e31-a26a-9a1ecb935f9c/volumes"
Oct 30 22:20:48.040247 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:48.040216    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-a6b6a5-7976bff669-4lf96"
Oct 30 22:20:48.040418 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:48.040330    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-a6b6a5-85cc5f8687-hdbfk"
Oct 30 22:20:49.283303 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:49.283273    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-ed44da-9b6f6d88c-jz6p9"
Oct 30 22:20:50.691960 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:50.691918    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:20:55.410678 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:20:55.410531    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:21:00.016730 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:00.016150    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:21:04.212272 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:04.212230    5510 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-3cc1b3-6b779bbcc6-dmbbd"]
Oct 30 22:21:04.212991 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:04.212974    5510 cpu_manager.go:414] "containerMap: removing container" podUID="6cbfd1c6-ca09-4bee-a6d3-6461eac2c2d5" containerName="istio-proxy"
Oct 30 22:21:04.212991 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:04.212992    5510 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6cbfd1c6-ca09-4bee-a6d3-6461eac2c2d5" containerName="istio-proxy"
Oct 30 22:21:04.213089 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:04.213002    5510 cpu_manager.go:414] "containerMap: removing container" podUID="07ed7cb3-6111-4809-8b1f-27a41ff6cafb" containerName="istio-proxy"
Oct 30 22:21:04.213089 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:04.213008    5510 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="07ed7cb3-6111-4809-8b1f-27a41ff6cafb" containerName="istio-proxy"
Oct 30 22:21:04.213089 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:04.213018    5510 cpu_manager.go:414] "containerMap: removing container" podUID="5542b62f-bc73-47d7-8615-180f43324911" containerName="istio-proxy"
Oct 30 22:21:04.213089 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:04.213024    5510 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5542b62f-bc73-47d7-8615-180f43324911" containerName="istio-proxy"
Oct 30 22:21:04.213089 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:04.213032    5510 cpu_manager.go:414] "containerMap: removing container" podUID="5ff054ea-b27c-4c64-be2d-c645c0ca3d24" containerName="istio-proxy"
Oct 30 22:21:04.213089 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:04.213037    5510 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5ff054ea-b27c-4c64-be2d-c645c0ca3d24" containerName="istio-proxy"
Oct 30 22:21:04.213089 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:04.213051    5510 cpu_manager.go:414] "containerMap: removing container" podUID="6aa08ff6-6f78-4a6e-83bd-07cf5d43b7b0" containerName="istio-proxy"
Oct 30 22:21:04.213089 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:04.213056    5510 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6aa08ff6-6f78-4a6e-83bd-07cf5d43b7b0" containerName="istio-proxy"
Oct 30 22:21:04.213089 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:04.213068    5510 cpu_manager.go:414] "containerMap: removing container" podUID="01891eb3-3336-4e31-a26a-9a1ecb935f9c" containerName="istio-proxy"
Oct 30 22:21:04.213089 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:04.213072    5510 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="01891eb3-3336-4e31-a26a-9a1ecb935f9c" containerName="istio-proxy"
Oct 30 22:21:04.213089 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:04.213086    5510 cpu_manager.go:414] "containerMap: removing container" podUID="01891eb3-3336-4e31-a26a-9a1ecb935f9c" containerName="istio-proxy"
Oct 30 22:21:04.213089 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:04.213091    5510 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="01891eb3-3336-4e31-a26a-9a1ecb935f9c" containerName="istio-proxy"
Oct 30 22:21:04.213446 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:04.213101    5510 cpu_manager.go:414] "containerMap: removing container" podUID="b0d37b4a-b986-438a-a811-9037505f0e6a" containerName="istio-proxy"
Oct 30 22:21:04.213446 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:04.213106    5510 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b0d37b4a-b986-438a-a811-9037505f0e6a" containerName="istio-proxy"
Oct 30 22:21:04.213446 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:04.213119    5510 cpu_manager.go:414] "containerMap: removing container" podUID="5ff054ea-b27c-4c64-be2d-c645c0ca3d24" containerName="istio-proxy"
Oct 30 22:21:04.213446 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:04.213123    5510 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5ff054ea-b27c-4c64-be2d-c645c0ca3d24" containerName="istio-proxy"
Oct 30 22:21:04.213446 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:04.213224    5510 memory_manager.go:365] "RemoveStaleState removing state" podUID="01891eb3-3336-4e31-a26a-9a1ecb935f9c" containerName="istio-proxy"
Oct 30 22:21:04.213446 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:04.213232    5510 memory_manager.go:365] "RemoveStaleState removing state" podUID="01891eb3-3336-4e31-a26a-9a1ecb935f9c" containerName="istio-proxy"
Oct 30 22:21:04.213446 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:04.213241    5510 memory_manager.go:365] "RemoveStaleState removing state" podUID="b0d37b4a-b986-438a-a811-9037505f0e6a" containerName="istio-proxy"
Oct 30 22:21:04.213446 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:04.213248    5510 memory_manager.go:365] "RemoveStaleState removing state" podUID="5ff054ea-b27c-4c64-be2d-c645c0ca3d24" containerName="istio-proxy"
Oct 30 22:21:04.213446 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:04.213258    5510 memory_manager.go:365] "RemoveStaleState removing state" podUID="6cbfd1c6-ca09-4bee-a6d3-6461eac2c2d5" containerName="istio-proxy"
Oct 30 22:21:04.213446 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:04.213265    5510 memory_manager.go:365] "RemoveStaleState removing state" podUID="07ed7cb3-6111-4809-8b1f-27a41ff6cafb" containerName="istio-proxy"
Oct 30 22:21:04.213446 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:04.213270    5510 memory_manager.go:365] "RemoveStaleState removing state" podUID="5542b62f-bc73-47d7-8615-180f43324911" containerName="istio-proxy"
Oct 30 22:21:04.213446 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:04.213275    5510 memory_manager.go:365] "RemoveStaleState removing state" podUID="5ff054ea-b27c-4c64-be2d-c645c0ca3d24" containerName="istio-proxy"
Oct 30 22:21:04.213446 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:04.213283    5510 memory_manager.go:365] "RemoveStaleState removing state" podUID="6aa08ff6-6f78-4a6e-83bd-07cf5d43b7b0" containerName="istio-proxy"
Oct 30 22:21:04.218600 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:04.218569    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-3cc1b3-6b779bbcc6-dmbbd"
Oct 30 22:21:04.268033 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:04.267999    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w5-3cc1b3-serving-cert\""
Oct 30 22:21:04.318932 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:04.318892    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-3cc1b3-6b779bbcc6-dmbbd"]
Oct 30 22:21:04.367926 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:04.367871    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f7fdf681-44d0-45d0-b2bd-c8e4df1af3bd-metrics-certs\") pod \"maas-api-e2e-shared-a-w5-3cc1b3-6b779bbcc6-dmbbd\" (UID: \"f7fdf681-44d0-45d0-b2bd-c8e4df1af3bd\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-3cc1b3-6b779bbcc6-dmbbd"
Oct 30 22:21:04.368107 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:04.368077    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f7fdf681-44d0-45d0-b2bd-c8e4df1af3bd-maas-api-tls\") pod \"maas-api-e2e-shared-a-w5-3cc1b3-6b779bbcc6-dmbbd\" (UID: \"f7fdf681-44d0-45d0-b2bd-c8e4df1af3bd\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-3cc1b3-6b779bbcc6-dmbbd"
Oct 30 22:21:04.368159 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:04.368110    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cvngh\" (UniqueName: \"kubernetes.io/projected/f7fdf681-44d0-45d0-b2bd-c8e4df1af3bd-kube-api-access-cvngh\") pod \"maas-api-e2e-shared-a-w5-3cc1b3-6b779bbcc6-dmbbd\" (UID: \"f7fdf681-44d0-45d0-b2bd-c8e4df1af3bd\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-3cc1b3-6b779bbcc6-dmbbd"
Oct 30 22:21:04.368300 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:04.368276    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f7fdf681-44d0-45d0-b2bd-c8e4df1af3bd-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w5-3cc1b3-6b779bbcc6-dmbbd\" (UID: \"f7fdf681-44d0-45d0-b2bd-c8e4df1af3bd\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-3cc1b3-6b779bbcc6-dmbbd"
Oct 30 22:21:04.519733 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:04.485920    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f7fdf681-44d0-45d0-b2bd-c8e4df1af3bd-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w5-3cc1b3-6b779bbcc6-dmbbd\" (UID: \"f7fdf681-44d0-45d0-b2bd-c8e4df1af3bd\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-3cc1b3-6b779bbcc6-dmbbd"
Oct 30 22:21:04.519733 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:04.486174    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f7fdf681-44d0-45d0-b2bd-c8e4df1af3bd-metrics-certs\") pod \"maas-api-e2e-shared-a-w5-3cc1b3-6b779bbcc6-dmbbd\" (UID: \"f7fdf681-44d0-45d0-b2bd-c8e4df1af3bd\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-3cc1b3-6b779bbcc6-dmbbd"
Oct 30 22:21:04.519733 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:04.486425    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f7fdf681-44d0-45d0-b2bd-c8e4df1af3bd-maas-api-tls\") pod \"maas-api-e2e-shared-a-w5-3cc1b3-6b779bbcc6-dmbbd\" (UID: \"f7fdf681-44d0-45d0-b2bd-c8e4df1af3bd\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-3cc1b3-6b779bbcc6-dmbbd"
Oct 30 22:21:04.519733 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:04.486458    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cvngh\" (UniqueName: \"kubernetes.io/projected/f7fdf681-44d0-45d0-b2bd-c8e4df1af3bd-kube-api-access-cvngh\") pod \"maas-api-e2e-shared-a-w5-3cc1b3-6b779bbcc6-dmbbd\" (UID: \"f7fdf681-44d0-45d0-b2bd-c8e4df1af3bd\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-3cc1b3-6b779bbcc6-dmbbd"
Oct 30 22:21:04.519733 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:04.500876    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f7fdf681-44d0-45d0-b2bd-c8e4df1af3bd-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w5-3cc1b3-6b779bbcc6-dmbbd\" (UID: \"f7fdf681-44d0-45d0-b2bd-c8e4df1af3bd\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-3cc1b3-6b779bbcc6-dmbbd"
Oct 30 22:21:04.519733 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:21:04.501001    5510 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-3cc1b3-serving-cert: secret "maas-api-e2e-shared-a-w5-3cc1b3-serving-cert" not found
Oct 30 22:21:04.519733 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:21:04.501078    5510 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f7fdf681-44d0-45d0-b2bd-c8e4df1af3bd-maas-api-tls podName:f7fdf681-44d0-45d0-b2bd-c8e4df1af3bd nodeName:}" failed. No retries permitted until 2026-10-30 22:21:05.001055905 +0000 UTC m=+1845.561865893 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/f7fdf681-44d0-45d0-b2bd-c8e4df1af3bd-maas-api-tls") pod "maas-api-e2e-shared-a-w5-3cc1b3-6b779bbcc6-dmbbd" (UID: "f7fdf681-44d0-45d0-b2bd-c8e4df1af3bd") : secret "maas-api-e2e-shared-a-w5-3cc1b3-serving-cert" not found
Oct 30 22:21:04.523173 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:04.521246    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f7fdf681-44d0-45d0-b2bd-c8e4df1af3bd-metrics-certs\") pod \"maas-api-e2e-shared-a-w5-3cc1b3-6b779bbcc6-dmbbd\" (UID: \"f7fdf681-44d0-45d0-b2bd-c8e4df1af3bd\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-3cc1b3-6b779bbcc6-dmbbd"
Oct 30 22:21:04.534379 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:04.534317    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:21:04.587075 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:04.586406    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cvngh\" (UniqueName: \"kubernetes.io/projected/f7fdf681-44d0-45d0-b2bd-c8e4df1af3bd-kube-api-access-cvngh\") pod \"maas-api-e2e-shared-a-w5-3cc1b3-6b779bbcc6-dmbbd\" (UID: \"f7fdf681-44d0-45d0-b2bd-c8e4df1af3bd\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-3cc1b3-6b779bbcc6-dmbbd"
Oct 30 22:21:04.748679 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:04.742255    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:21:05.037028 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:05.036988    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f7fdf681-44d0-45d0-b2bd-c8e4df1af3bd-maas-api-tls\") pod \"maas-api-e2e-shared-a-w5-3cc1b3-6b779bbcc6-dmbbd\" (UID: \"f7fdf681-44d0-45d0-b2bd-c8e4df1af3bd\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-3cc1b3-6b779bbcc6-dmbbd"
Oct 30 22:21:05.039451 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:05.039425    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f7fdf681-44d0-45d0-b2bd-c8e4df1af3bd-maas-api-tls\") pod \"maas-api-e2e-shared-a-w5-3cc1b3-6b779bbcc6-dmbbd\" (UID: \"f7fdf681-44d0-45d0-b2bd-c8e4df1af3bd\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-3cc1b3-6b779bbcc6-dmbbd"
Oct 30 22:21:05.139263 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:05.139235    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-3cc1b3-6b779bbcc6-dmbbd"
Oct 30 22:21:05.517841 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:05.517803    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-3cc1b3-6b779bbcc6-dmbbd"]
Oct 30 22:21:05.518294 ip-10-0-136-210 kubenswrapper[5510]: W1030 22:21:05.518267    5510 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7fdf681_44d0_45d0_b2bd_c8e4df1af3bd.slice/crio-8127b28ec5d42f7492fa5b8c72ed985728837b2ab1e358a2a59ea8c397c230cd WatchSource:0}: Error finding container 8127b28ec5d42f7492fa5b8c72ed985728837b2ab1e358a2a59ea8c397c230cd: Status 404 returned error can't find the container with id 8127b28ec5d42f7492fa5b8c72ed985728837b2ab1e358a2a59ea8c397c230cd
Oct 30 22:21:05.758128 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:05.758092    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:21:05.998505 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:05.998456    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:21:06.455999 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:06.455961    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-3cc1b3-6b779bbcc6-dmbbd" event={"ID":"f7fdf681-44d0-45d0-b2bd-c8e4df1af3bd","Type":"ContainerStarted","Data":"8127b28ec5d42f7492fa5b8c72ed985728837b2ab1e358a2a59ea8c397c230cd"}
Oct 30 22:21:07.465468 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:07.465434    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-3cc1b3-6b779bbcc6-dmbbd" event={"ID":"f7fdf681-44d0-45d0-b2bd-c8e4df1af3bd","Type":"ContainerStarted","Data":"d234967cdbe954b92f8b290d34db24100d622ec488fe62088c69b1796f7ccbf4"}
Oct 30 22:21:07.465917 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:07.465620    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-3cc1b3-6b779bbcc6-dmbbd"
Oct 30 22:21:07.580535 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:07.580478    5510 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-3cc1b3-6b779bbcc6-dmbbd" podStartSLOduration=2.572369396 podStartE2EDuration="3.580463636s" totalImagesPullingTime="1.00809424s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:21:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:21:07.577078343 +0000 UTC m=+1848.137888339" watchObservedRunningTime="2026-10-30 22:21:07.580463636 +0000 UTC m=+1848.141273640"
Oct 30 22:21:08.421582 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:21:08.421542    5510 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6gjbf" containerName="extractor"
Oct 30 22:21:09.201758 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:09.198521    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:21:10.146557 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:10.146522    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:21:13.479085 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:13.479051    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-3cc1b3-6b779bbcc6-dmbbd"
Oct 30 22:21:13.687832 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:13.687793    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:21:15.332505 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:15.332467    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:21:16.254991 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:16.254956    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:21:18.480684 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:18.478815    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:21:23.034614 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:23.034578    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:21:27.619686 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:27.609425    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:21:28.926032 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:28.925987    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:21:32.441680 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:32.437899    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-779f32/distinct-w0-779f32-kserve-6dcbc948c7-2vzdf" podUID="e17cedd4-7d48-42b3-b127-b6192c1b3795" containerName="main" probeResult="failure" output="Get \"https://10.133.0.65:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:21:32.487207 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:32.487156    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-c9dbe6/distinct-w5-c9dbe6-kserve-7599fd86d9-bdhxd" podUID="e903db90-f2cd-4244-b00b-f99b21873f6e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:21:32.740783 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:32.740696    5510 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-a6b6a5-cdb55bb68-ljgn7"]
Oct 30 22:21:32.756583 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:32.756556    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-a6b6a5-cdb55bb68-ljgn7"
Oct 30 22:21:32.793436 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:32.793405    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-a6b6a5-cdb55bb68-ljgn7"]
Oct 30 22:21:32.838063 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:32.838017    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w1-a6b6a5-serving-cert\""
Oct 30 22:21:32.840227 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:32.840196    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/195dcf2f-1af0-4404-ac3c-2e3424b9b266-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w1-a6b6a5-cdb55bb68-ljgn7\" (UID: \"195dcf2f-1af0-4404-ac3c-2e3424b9b266\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-a6b6a5-cdb55bb68-ljgn7"
Oct 30 22:21:32.840552 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:32.840525    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zc7z8\" (UniqueName: \"kubernetes.io/projected/195dcf2f-1af0-4404-ac3c-2e3424b9b266-kube-api-access-zc7z8\") pod \"maas-api-e2e-shared-a-w1-a6b6a5-cdb55bb68-ljgn7\" (UID: \"195dcf2f-1af0-4404-ac3c-2e3424b9b266\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-a6b6a5-cdb55bb68-ljgn7"
Oct 30 22:21:32.840768 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:32.840747    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/195dcf2f-1af0-4404-ac3c-2e3424b9b266-maas-api-tls\") pod \"maas-api-e2e-shared-a-w1-a6b6a5-cdb55bb68-ljgn7\" (UID: \"195dcf2f-1af0-4404-ac3c-2e3424b9b266\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-a6b6a5-cdb55bb68-ljgn7"
Oct 30 22:21:32.841113 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:32.841093    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/195dcf2f-1af0-4404-ac3c-2e3424b9b266-metrics-certs\") pod \"maas-api-e2e-shared-a-w1-a6b6a5-cdb55bb68-ljgn7\" (UID: \"195dcf2f-1af0-4404-ac3c-2e3424b9b266\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-a6b6a5-cdb55bb68-ljgn7"
Oct 30 22:21:33.035678 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:33.033391    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:21:33.093809 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:33.093747    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-fb0b93/premium-w6-fb0b93-kserve-5cb9948c54-w7pnk" podUID="290ae7a2-de37-4af3-82ae-a8e855b83bb9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:21:33.097295 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:33.097257    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-779f32/distinct-2-w0-779f32-kserve-857ffdf778-gk2h9" podUID="ef500cdd-cc52-45b0-8dd7-1896897acc46" containerName="main" probeResult="failure" output="Get \"https://10.133.0.74:8000/ready\": context deadline exceeded"
Oct 30 22:21:33.316128 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:33.316030    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-bc714d/distinct-w2-bc714d-kserve-66cfb968dd-jvgqj" podUID="9e9b11b8-1cc9-4fc8-8097-1afcffa8c422" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": context deadline exceeded"
Oct 30 22:21:33.341438 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:33.341287    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:21:33.341965 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:33.341811    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-fb0b93/distinct-w6-fb0b93-kserve-58db6b6757-6mww8" podUID="8ee6579b-acea-4492-862b-9883f70d00c9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:21:33.438939 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:33.438899    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/195dcf2f-1af0-4404-ac3c-2e3424b9b266-maas-api-tls\") pod \"maas-api-e2e-shared-a-w1-a6b6a5-cdb55bb68-ljgn7\" (UID: \"195dcf2f-1af0-4404-ac3c-2e3424b9b266\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-a6b6a5-cdb55bb68-ljgn7"
Oct 30 22:21:33.439529 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:33.439467    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/195dcf2f-1af0-4404-ac3c-2e3424b9b266-metrics-certs\") pod \"maas-api-e2e-shared-a-w1-a6b6a5-cdb55bb68-ljgn7\" (UID: \"195dcf2f-1af0-4404-ac3c-2e3424b9b266\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-a6b6a5-cdb55bb68-ljgn7"
Oct 30 22:21:33.439882 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:33.439856    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/195dcf2f-1af0-4404-ac3c-2e3424b9b266-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w1-a6b6a5-cdb55bb68-ljgn7\" (UID: \"195dcf2f-1af0-4404-ac3c-2e3424b9b266\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-a6b6a5-cdb55bb68-ljgn7"
Oct 30 22:21:33.440117 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:33.440096    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zc7z8\" (UniqueName: \"kubernetes.io/projected/195dcf2f-1af0-4404-ac3c-2e3424b9b266-kube-api-access-zc7z8\") pod \"maas-api-e2e-shared-a-w1-a6b6a5-cdb55bb68-ljgn7\" (UID: \"195dcf2f-1af0-4404-ac3c-2e3424b9b266\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-a6b6a5-cdb55bb68-ljgn7"
Oct 30 22:21:33.505682 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:33.481181    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-779f32/distinct-w0-779f32-kserve-6dcbc948c7-2vzdf" podUID="e17cedd4-7d48-42b3-b127-b6192c1b3795" containerName="main" probeResult="failure" output="Get \"https://10.133.0.65:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:21:33.507429 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:33.507308    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-c9dbe6/distinct-w5-c9dbe6-kserve-7599fd86d9-bdhxd" podUID="e903db90-f2cd-4244-b00b-f99b21873f6e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:21:33.508213 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:33.508188    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-779f32/distinct-w0-779f32-kserve-6dcbc948c7-2vzdf"
Oct 30 22:21:33.508338 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:33.508264    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-c9dbe6/distinct-w5-c9dbe6-kserve-7599fd86d9-bdhxd"
Oct 30 22:21:33.585738 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:33.583706    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/195dcf2f-1af0-4404-ac3c-2e3424b9b266-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w1-a6b6a5-cdb55bb68-ljgn7\" (UID: \"195dcf2f-1af0-4404-ac3c-2e3424b9b266\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-a6b6a5-cdb55bb68-ljgn7"
Oct 30 22:21:33.585738 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:33.585585    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/195dcf2f-1af0-4404-ac3c-2e3424b9b266-metrics-certs\") pod \"maas-api-e2e-shared-a-w1-a6b6a5-cdb55bb68-ljgn7\" (UID: \"195dcf2f-1af0-4404-ac3c-2e3424b9b266\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-a6b6a5-cdb55bb68-ljgn7"
Oct 30 22:21:33.610694 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:33.589898    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:21:33.610694 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:33.609473    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/195dcf2f-1af0-4404-ac3c-2e3424b9b266-maas-api-tls\") pod \"maas-api-e2e-shared-a-w1-a6b6a5-cdb55bb68-ljgn7\" (UID: \"195dcf2f-1af0-4404-ac3c-2e3424b9b266\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-a6b6a5-cdb55bb68-ljgn7"
Oct 30 22:21:33.620603 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:33.620189    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zc7z8\" (UniqueName: \"kubernetes.io/projected/195dcf2f-1af0-4404-ac3c-2e3424b9b266-kube-api-access-zc7z8\") pod \"maas-api-e2e-shared-a-w1-a6b6a5-cdb55bb68-ljgn7\" (UID: \"195dcf2f-1af0-4404-ac3c-2e3424b9b266\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-a6b6a5-cdb55bb68-ljgn7"
Oct 30 22:21:33.623819 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:33.623796    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-c9dbe6/distinct-w5-c9dbe6-kserve-7599fd86d9-bdhxd"
Oct 30 22:21:33.624358 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:33.624333    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-779f32/distinct-w0-779f32-kserve-6dcbc948c7-2vzdf"
Oct 30 22:21:33.737521 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:33.737488    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-a6b6a5-cdb55bb68-ljgn7"
Oct 30 22:21:34.268706 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:34.267119    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-a6b6a5-cdb55bb68-ljgn7"]
Oct 30 22:21:34.804227 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:34.804194    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-a6b6a5-cdb55bb68-ljgn7" event={"ID":"195dcf2f-1af0-4404-ac3c-2e3424b9b266","Type":"ContainerStarted","Data":"5d7606da0cf2835e5c53c6481dac273aac45d156b0deb23e911b3ce3600dbc5b"}
Oct 30 22:21:35.813355 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:35.813286    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-a6b6a5-cdb55bb68-ljgn7" event={"ID":"195dcf2f-1af0-4404-ac3c-2e3424b9b266","Type":"ContainerStarted","Data":"3c7c302174804a1909818ec8b4442d44876684c989a0be28a2419784b2a32b62"}
Oct 30 22:21:35.813909 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:35.813523    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-a6b6a5-cdb55bb68-ljgn7"
Oct 30 22:21:35.943686 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:35.941528    5510 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-a6b6a5-cdb55bb68-ljgn7" podStartSLOduration=2.914414552 podStartE2EDuration="3.941506859s" totalImagesPullingTime="1.027092307s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:21:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:21:35.941038507 +0000 UTC m=+1876.501848539" watchObservedRunningTime="2026-10-30 22:21:35.941506859 +0000 UTC m=+1876.502316864"
Oct 30 22:21:36.830839 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:36.830794    5510 patch_prober.go:29] interesting pod/e2e-mt-4da07739-openshift-default-6bd77c67d9-7wknn container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.107:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 22:21:36.831306 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:36.830870    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-4da07739-openshift-default-6bd77c67d9-7wknn" podUID="f4d4ae01-a1b3-41ea-9e54-88edbe45642c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.107:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:21:37.043542 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:37.043454    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:21:38.023686 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:38.009379    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:21:38.420266 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:21:38.420182    5510 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6gjbf" containerName="extractor"
Oct 30 22:21:38.871725 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:38.871687    5510 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w2-cf1826-openshift-default-cf9479bdc-jgkhc"]
Oct 30 22:21:38.880592 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:38.880563    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-cf1826-openshift-default-cf9479bdc-jgkhc"
Oct 30 22:21:38.891593 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:38.891224    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/13760c22-8ba0-45ed-b2e4-4e23d5273901-istio-data\") pod \"e2e-shared-a-w2-cf1826-openshift-default-cf9479bdc-jgkhc\" (UID: \"13760c22-8ba0-45ed-b2e4-4e23d5273901\") " pod="openshift-ingress/e2e-shared-a-w2-cf1826-openshift-default-cf9479bdc-jgkhc"
Oct 30 22:21:38.891593 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:38.891310    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f6tfk\" (UniqueName: \"kubernetes.io/projected/13760c22-8ba0-45ed-b2e4-4e23d5273901-kube-api-access-f6tfk\") pod \"e2e-shared-a-w2-cf1826-openshift-default-cf9479bdc-jgkhc\" (UID: \"13760c22-8ba0-45ed-b2e4-4e23d5273901\") " pod="openshift-ingress/e2e-shared-a-w2-cf1826-openshift-default-cf9479bdc-jgkhc"
Oct 30 22:21:38.891593 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:38.891422    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/13760c22-8ba0-45ed-b2e4-4e23d5273901-istio-envoy\") pod \"e2e-shared-a-w2-cf1826-openshift-default-cf9479bdc-jgkhc\" (UID: \"13760c22-8ba0-45ed-b2e4-4e23d5273901\") " pod="openshift-ingress/e2e-shared-a-w2-cf1826-openshift-default-cf9479bdc-jgkhc"
Oct 30 22:21:38.891593 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:38.891503    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/13760c22-8ba0-45ed-b2e4-4e23d5273901-workload-certs\") pod \"e2e-shared-a-w2-cf1826-openshift-default-cf9479bdc-jgkhc\" (UID: \"13760c22-8ba0-45ed-b2e4-4e23d5273901\") " pod="openshift-ingress/e2e-shared-a-w2-cf1826-openshift-default-cf9479bdc-jgkhc"
Oct 30 22:21:38.891593 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:38.891551    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/13760c22-8ba0-45ed-b2e4-4e23d5273901-istio-token\") pod \"e2e-shared-a-w2-cf1826-openshift-default-cf9479bdc-jgkhc\" (UID: \"13760c22-8ba0-45ed-b2e4-4e23d5273901\") " pod="openshift-ingress/e2e-shared-a-w2-cf1826-openshift-default-cf9479bdc-jgkhc"
Oct 30 22:21:38.891593 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:38.891586    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/13760c22-8ba0-45ed-b2e4-4e23d5273901-istiod-ca-cert\") pod \"e2e-shared-a-w2-cf1826-openshift-default-cf9479bdc-jgkhc\" (UID: \"13760c22-8ba0-45ed-b2e4-4e23d5273901\") " pod="openshift-ingress/e2e-shared-a-w2-cf1826-openshift-default-cf9479bdc-jgkhc"
Oct 30 22:21:38.892060 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:38.891609    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/13760c22-8ba0-45ed-b2e4-4e23d5273901-credential-socket\") pod \"e2e-shared-a-w2-cf1826-openshift-default-cf9479bdc-jgkhc\" (UID: \"13760c22-8ba0-45ed-b2e4-4e23d5273901\") " pod="openshift-ingress/e2e-shared-a-w2-cf1826-openshift-default-cf9479bdc-jgkhc"
Oct 30 22:21:38.892060 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:38.891763    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/13760c22-8ba0-45ed-b2e4-4e23d5273901-workload-socket\") pod \"e2e-shared-a-w2-cf1826-openshift-default-cf9479bdc-jgkhc\" (UID: \"13760c22-8ba0-45ed-b2e4-4e23d5273901\") " pod="openshift-ingress/e2e-shared-a-w2-cf1826-openshift-default-cf9479bdc-jgkhc"
Oct 30 22:21:38.892169 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:38.892109    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/13760c22-8ba0-45ed-b2e4-4e23d5273901-istio-podinfo\") pod \"e2e-shared-a-w2-cf1826-openshift-default-cf9479bdc-jgkhc\" (UID: \"13760c22-8ba0-45ed-b2e4-4e23d5273901\") " pod="openshift-ingress/e2e-shared-a-w2-cf1826-openshift-default-cf9479bdc-jgkhc"
Oct 30 22:21:38.934514 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:38.934479    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w2-cf1826-openshift-default-dockercfg-ngt2h\""
Oct 30 22:21:38.982281 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:38.982245    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-cf1826-openshift-default-cf9479bdc-jgkhc"]
Oct 30 22:21:38.993978 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:38.993943    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/13760c22-8ba0-45ed-b2e4-4e23d5273901-istio-data\") pod \"e2e-shared-a-w2-cf1826-openshift-default-cf9479bdc-jgkhc\" (UID: \"13760c22-8ba0-45ed-b2e4-4e23d5273901\") " pod="openshift-ingress/e2e-shared-a-w2-cf1826-openshift-default-cf9479bdc-jgkhc"
Oct 30 22:21:38.994171 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:38.994035    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f6tfk\" (UniqueName: \"kubernetes.io/projected/13760c22-8ba0-45ed-b2e4-4e23d5273901-kube-api-access-f6tfk\") pod \"e2e-shared-a-w2-cf1826-openshift-default-cf9479bdc-jgkhc\" (UID: \"13760c22-8ba0-45ed-b2e4-4e23d5273901\") " pod="openshift-ingress/e2e-shared-a-w2-cf1826-openshift-default-cf9479bdc-jgkhc"
Oct 30 22:21:38.994171 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:38.994075    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/13760c22-8ba0-45ed-b2e4-4e23d5273901-istio-envoy\") pod \"e2e-shared-a-w2-cf1826-openshift-default-cf9479bdc-jgkhc\" (UID: \"13760c22-8ba0-45ed-b2e4-4e23d5273901\") " pod="openshift-ingress/e2e-shared-a-w2-cf1826-openshift-default-cf9479bdc-jgkhc"
Oct 30 22:21:38.994171 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:38.994148    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/13760c22-8ba0-45ed-b2e4-4e23d5273901-workload-certs\") pod \"e2e-shared-a-w2-cf1826-openshift-default-cf9479bdc-jgkhc\" (UID: \"13760c22-8ba0-45ed-b2e4-4e23d5273901\") " pod="openshift-ingress/e2e-shared-a-w2-cf1826-openshift-default-cf9479bdc-jgkhc"
Oct 30 22:21:38.994598 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:38.994509    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/13760c22-8ba0-45ed-b2e4-4e23d5273901-istio-data\") pod \"e2e-shared-a-w2-cf1826-openshift-default-cf9479bdc-jgkhc\" (UID: \"13760c22-8ba0-45ed-b2e4-4e23d5273901\") " pod="openshift-ingress/e2e-shared-a-w2-cf1826-openshift-default-cf9479bdc-jgkhc"
Oct 30 22:21:38.994921 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:38.994182    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/13760c22-8ba0-45ed-b2e4-4e23d5273901-istio-token\") pod \"e2e-shared-a-w2-cf1826-openshift-default-cf9479bdc-jgkhc\" (UID: \"13760c22-8ba0-45ed-b2e4-4e23d5273901\") " pod="openshift-ingress/e2e-shared-a-w2-cf1826-openshift-default-cf9479bdc-jgkhc"
Oct 30 22:21:38.995018 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:38.994960    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/13760c22-8ba0-45ed-b2e4-4e23d5273901-istiod-ca-cert\") pod \"e2e-shared-a-w2-cf1826-openshift-default-cf9479bdc-jgkhc\" (UID: \"13760c22-8ba0-45ed-b2e4-4e23d5273901\") " pod="openshift-ingress/e2e-shared-a-w2-cf1826-openshift-default-cf9479bdc-jgkhc"
Oct 30 22:21:38.995018 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:38.994972    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/13760c22-8ba0-45ed-b2e4-4e23d5273901-workload-certs\") pod \"e2e-shared-a-w2-cf1826-openshift-default-cf9479bdc-jgkhc\" (UID: \"13760c22-8ba0-45ed-b2e4-4e23d5273901\") " pod="openshift-ingress/e2e-shared-a-w2-cf1826-openshift-default-cf9479bdc-jgkhc"
Oct 30 22:21:38.995018 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:38.995000    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/13760c22-8ba0-45ed-b2e4-4e23d5273901-credential-socket\") pod \"e2e-shared-a-w2-cf1826-openshift-default-cf9479bdc-jgkhc\" (UID: \"13760c22-8ba0-45ed-b2e4-4e23d5273901\") " pod="openshift-ingress/e2e-shared-a-w2-cf1826-openshift-default-cf9479bdc-jgkhc"
Oct 30 22:21:38.995174 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:38.995081    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/13760c22-8ba0-45ed-b2e4-4e23d5273901-workload-socket\") pod \"e2e-shared-a-w2-cf1826-openshift-default-cf9479bdc-jgkhc\" (UID: \"13760c22-8ba0-45ed-b2e4-4e23d5273901\") " pod="openshift-ingress/e2e-shared-a-w2-cf1826-openshift-default-cf9479bdc-jgkhc"
Oct 30 22:21:38.995283 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:38.995265    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/13760c22-8ba0-45ed-b2e4-4e23d5273901-istio-podinfo\") pod \"e2e-shared-a-w2-cf1826-openshift-default-cf9479bdc-jgkhc\" (UID: \"13760c22-8ba0-45ed-b2e4-4e23d5273901\") " pod="openshift-ingress/e2e-shared-a-w2-cf1826-openshift-default-cf9479bdc-jgkhc"
Oct 30 22:21:38.995342 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:38.995313    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/13760c22-8ba0-45ed-b2e4-4e23d5273901-credential-socket\") pod \"e2e-shared-a-w2-cf1826-openshift-default-cf9479bdc-jgkhc\" (UID: \"13760c22-8ba0-45ed-b2e4-4e23d5273901\") " pod="openshift-ingress/e2e-shared-a-w2-cf1826-openshift-default-cf9479bdc-jgkhc"
Oct 30 22:21:38.995644 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:38.995622    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/13760c22-8ba0-45ed-b2e4-4e23d5273901-istiod-ca-cert\") pod \"e2e-shared-a-w2-cf1826-openshift-default-cf9479bdc-jgkhc\" (UID: \"13760c22-8ba0-45ed-b2e4-4e23d5273901\") " pod="openshift-ingress/e2e-shared-a-w2-cf1826-openshift-default-cf9479bdc-jgkhc"
Oct 30 22:21:38.995751 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:38.995733    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/13760c22-8ba0-45ed-b2e4-4e23d5273901-workload-socket\") pod \"e2e-shared-a-w2-cf1826-openshift-default-cf9479bdc-jgkhc\" (UID: \"13760c22-8ba0-45ed-b2e4-4e23d5273901\") " pod="openshift-ingress/e2e-shared-a-w2-cf1826-openshift-default-cf9479bdc-jgkhc"
Oct 30 22:21:38.997820 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:38.997488    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/13760c22-8ba0-45ed-b2e4-4e23d5273901-istio-envoy\") pod \"e2e-shared-a-w2-cf1826-openshift-default-cf9479bdc-jgkhc\" (UID: \"13760c22-8ba0-45ed-b2e4-4e23d5273901\") " pod="openshift-ingress/e2e-shared-a-w2-cf1826-openshift-default-cf9479bdc-jgkhc"
Oct 30 22:21:38.998122 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:38.998101    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/13760c22-8ba0-45ed-b2e4-4e23d5273901-istio-podinfo\") pod \"e2e-shared-a-w2-cf1826-openshift-default-cf9479bdc-jgkhc\" (UID: \"13760c22-8ba0-45ed-b2e4-4e23d5273901\") " pod="openshift-ingress/e2e-shared-a-w2-cf1826-openshift-default-cf9479bdc-jgkhc"
Oct 30 22:21:39.017997 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:39.015004    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:21:39.135523 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:39.135058    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/13760c22-8ba0-45ed-b2e4-4e23d5273901-istio-token\") pod \"e2e-shared-a-w2-cf1826-openshift-default-cf9479bdc-jgkhc\" (UID: \"13760c22-8ba0-45ed-b2e4-4e23d5273901\") " pod="openshift-ingress/e2e-shared-a-w2-cf1826-openshift-default-cf9479bdc-jgkhc"
Oct 30 22:21:39.167736 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:39.148285    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f6tfk\" (UniqueName: \"kubernetes.io/projected/13760c22-8ba0-45ed-b2e4-4e23d5273901-kube-api-access-f6tfk\") pod \"e2e-shared-a-w2-cf1826-openshift-default-cf9479bdc-jgkhc\" (UID: \"13760c22-8ba0-45ed-b2e4-4e23d5273901\") " pod="openshift-ingress/e2e-shared-a-w2-cf1826-openshift-default-cf9479bdc-jgkhc"
Oct 30 22:21:39.225101 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:39.225060    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-cf1826-openshift-default-cf9479bdc-jgkhc"
Oct 30 22:21:39.229492 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:39.229454    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:21:39.711541 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:39.711506    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-cf1826-openshift-default-cf9479bdc-jgkhc"]
Oct 30 22:21:39.712008 ip-10-0-136-210 kubenswrapper[5510]: W1030 22:21:39.711985    5510 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13760c22_8ba0_45ed_b2e4_4e23d5273901.slice/crio-660a35f724c314692b44f27205866eb8ce26685930826dd76ce54bc20019cdcc WatchSource:0}: Error finding container 660a35f724c314692b44f27205866eb8ce26685930826dd76ce54bc20019cdcc: Status 404 returned error can't find the container with id 660a35f724c314692b44f27205866eb8ce26685930826dd76ce54bc20019cdcc
Oct 30 22:21:39.715161 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:39.715129    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:21:39.715255 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:39.715199    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:21:39.715255 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:39.715238    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:21:39.849587 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:39.849530    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-cf1826-openshift-default-cf9479bdc-jgkhc" event={"ID":"13760c22-8ba0-45ed-b2e4-4e23d5273901","Type":"ContainerStarted","Data":"f572ac7fae6b82a9bf28a0c16eee7c0515f93747676b59f2afe6e114fda3a496"}
Oct 30 22:21:39.849784 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:39.849606    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-cf1826-openshift-default-cf9479bdc-jgkhc" event={"ID":"13760c22-8ba0-45ed-b2e4-4e23d5273901","Type":"ContainerStarted","Data":"660a35f724c314692b44f27205866eb8ce26685930826dd76ce54bc20019cdcc"}
Oct 30 22:21:39.969254 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:39.969146    5510 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w2-cf1826-openshift-default-cf9479bdc-jgkhc" podStartSLOduration=1.969129284 podStartE2EDuration="1.969129284s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:21:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:21:39.964599398 +0000 UTC m=+1880.525409434" watchObservedRunningTime="2026-10-30 22:21:39.969129284 +0000 UTC m=+1880.529939291"
Oct 30 22:21:40.232092 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:40.230794    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w2-cf1826-openshift-default-cf9479bdc-jgkhc"
Oct 30 22:21:40.232092 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:40.230926    5510 patch_prober.go:29] interesting pod/e2e-shared-a-w2-cf1826-openshift-default-cf9479bdc-jgkhc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.118:15021/healthz/ready\": dial tcp 10.133.0.118:15021: connect: connection refused" start-of-body=
Oct 30 22:21:40.232092 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:40.230967    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-cf1826-openshift-default-cf9479bdc-jgkhc" podUID="13760c22-8ba0-45ed-b2e4-4e23d5273901" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.118:15021/healthz/ready\": dial tcp 10.133.0.118:15021: connect: connection refused"
Oct 30 22:21:40.859341 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:40.859304    5510 generic.go:356] "Generic (PLEG): container finished" podID="f4d4ae01-a1b3-41ea-9e54-88edbe45642c" containerID="0266b6992b04d91261c6136773c8e9e08aa1dbfe4c76a25afc7fb6a7a7eb3f49" exitCode=0
Oct 30 22:21:40.859558 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:40.859376    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-4da07739-openshift-default-6bd77c67d9-7wknn" event={"ID":"f4d4ae01-a1b3-41ea-9e54-88edbe45642c","Type":"ContainerDied","Data":"0266b6992b04d91261c6136773c8e9e08aa1dbfe4c76a25afc7fb6a7a7eb3f49"}
Oct 30 22:21:40.859829 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:40.859809    5510 scope.go:122] "RemoveContainer" containerID="0266b6992b04d91261c6136773c8e9e08aa1dbfe4c76a25afc7fb6a7a7eb3f49"
Oct 30 22:21:40.861208 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:40.861181    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:21:40.861268 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:40.861250    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:21:40.861304 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:40.861282    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:21:41.232110 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:41.232029    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w2-cf1826-openshift-default-cf9479bdc-jgkhc"
Oct 30 22:21:41.827932 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:41.827901    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-a6b6a5-cdb55bb68-ljgn7"
Oct 30 22:21:41.879876 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:41.879845    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-4da07739-openshift-default-6bd77c67d9-7wknn" event={"ID":"f4d4ae01-a1b3-41ea-9e54-88edbe45642c","Type":"ContainerStarted","Data":"7ce8dd60f4818547de0c6d5dacae278101c713a7c255b98121432465fb8c4545"}
Oct 30 22:21:41.880136 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:41.880115    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w2-cf1826-openshift-default-cf9479bdc-jgkhc"
Oct 30 22:21:41.881000 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:41.880971    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w2-cf1826-openshift-default-cf9479bdc-jgkhc"
Oct 30 22:21:42.261728 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:42.257262    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-4da07739-openshift-default-6bd77c67d9-7wknn"
Oct 30 22:21:42.261728 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:42.257396    5510 patch_prober.go:29] interesting pod/e2e-mt-4da07739-openshift-default-6bd77c67d9-7wknn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.107:15021/healthz/ready\": dial tcp 10.133.0.107:15021: connect: connection refused" start-of-body=
Oct 30 22:21:42.261728 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:42.257459    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-4da07739-openshift-default-6bd77c67d9-7wknn" podUID="f4d4ae01-a1b3-41ea-9e54-88edbe45642c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.107:15021/healthz/ready\": dial tcp 10.133.0.107:15021: connect: connection refused"
Oct 30 22:21:43.198187 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:43.197997    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:21:43.281679 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:43.276013    5510 patch_prober.go:29] interesting pod/e2e-mt-4da07739-openshift-default-6bd77c67d9-7wknn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.107:15021/healthz/ready\": dial tcp 10.133.0.107:15021: connect: connection refused" start-of-body=
Oct 30 22:21:43.281679 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:43.276102    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-4da07739-openshift-default-6bd77c67d9-7wknn" podUID="f4d4ae01-a1b3-41ea-9e54-88edbe45642c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.107:15021/healthz/ready\": dial tcp 10.133.0.107:15021: connect: connection refused"
Oct 30 22:21:43.407074 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:43.407041    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:21:43.669449 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:43.669407    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:21:44.260147 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:44.260103    5510 patch_prober.go:29] interesting pod/e2e-mt-4da07739-openshift-default-6bd77c67d9-7wknn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.107:15021/healthz/ready\": dial tcp 10.133.0.107:15021: connect: connection refused" start-of-body=
Oct 30 22:21:44.260486 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:44.260455    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-4da07739-openshift-default-6bd77c67d9-7wknn" podUID="f4d4ae01-a1b3-41ea-9e54-88edbe45642c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.107:15021/healthz/ready\": dial tcp 10.133.0.107:15021: connect: connection refused"
Oct 30 22:21:45.264681 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:45.264615    5510 patch_prober.go:29] interesting pod/e2e-mt-4da07739-openshift-default-6bd77c67d9-7wknn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.107:15021/healthz/ready\": dial tcp 10.133.0.107:15021: connect: connection refused" start-of-body=
Oct 30 22:21:45.265217 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:45.264720    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-4da07739-openshift-default-6bd77c67d9-7wknn" podUID="f4d4ae01-a1b3-41ea-9e54-88edbe45642c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.107:15021/healthz/ready\": dial tcp 10.133.0.107:15021: connect: connection refused"
Oct 30 22:21:45.896019 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:45.895977    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:21:46.265167 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:46.265070    5510 patch_prober.go:29] interesting pod/e2e-mt-4da07739-openshift-default-6bd77c67d9-7wknn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.107:15021/healthz/ready\": dial tcp 10.133.0.107:15021: connect: connection refused" start-of-body=
Oct 30 22:21:46.266447 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:46.266410    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-4da07739-openshift-default-6bd77c67d9-7wknn" podUID="f4d4ae01-a1b3-41ea-9e54-88edbe45642c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.107:15021/healthz/ready\": dial tcp 10.133.0.107:15021: connect: connection refused"
Oct 30 22:21:46.477425 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:46.477379    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:21:46.918865 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:46.918820    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:21:47.125921 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:47.125870    5510 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-cf1826-5f6ddbf4c7-7mrgs"]
Oct 30 22:21:47.211749 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:47.208490    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:21:47.301930 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:47.301887    5510 patch_prober.go:29] interesting pod/e2e-mt-4da07739-openshift-default-6bd77c67d9-7wknn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.107:15021/healthz/ready\": dial tcp 10.133.0.107:15021: connect: connection refused" start-of-body=
Oct 30 22:21:47.302544 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:47.302500    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-4da07739-openshift-default-6bd77c67d9-7wknn" podUID="f4d4ae01-a1b3-41ea-9e54-88edbe45642c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.107:15021/healthz/ready\": dial tcp 10.133.0.107:15021: connect: connection refused"
Oct 30 22:21:47.463202 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:47.463111    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-cf1826-5f6ddbf4c7-7mrgs"]
Oct 30 22:21:47.463554 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:47.463537    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-cf1826-5f6ddbf4c7-7mrgs"
Oct 30 22:21:47.488094 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:47.488062    5510 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-cf1826-556566884c-krgkh"]
Oct 30 22:21:47.496266 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:47.496230    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-cf1826-556566884c-krgkh"
Oct 30 22:21:47.500699 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:47.498236    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-cf1826-556566884c-krgkh"]
Oct 30 22:21:47.541640 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:47.541599    5510 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w2-cf1826\""
Oct 30 22:21:47.571973 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:47.571936    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wvwqx\" (UniqueName: \"kubernetes.io/projected/7bdcd87e-ab76-4af6-ae7b-37e1384b6406-kube-api-access-wvwqx\") pod \"payload-processing-e2e-shared-a-w2-cf1826-5f6ddbf4c7-7mrgs\" (UID: \"7bdcd87e-ab76-4af6-ae7b-37e1384b6406\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-cf1826-5f6ddbf4c7-7mrgs"
Oct 30 22:21:47.572172 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:47.572055    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gkjlr\" (UniqueName: \"kubernetes.io/projected/4d203fdf-00d0-4d25-8a63-c2f355d8d03f-kube-api-access-gkjlr\") pod \"payload-pre-processing-e2e-shared-a-w2-cf1826-556566884c-krgkh\" (UID: \"4d203fdf-00d0-4d25-8a63-c2f355d8d03f\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-cf1826-556566884c-krgkh"
Oct 30 22:21:47.572172 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:47.572102    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4d203fdf-00d0-4d25-8a63-c2f355d8d03f-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-cf1826-556566884c-krgkh\" (UID: \"4d203fdf-00d0-4d25-8a63-c2f355d8d03f\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-cf1826-556566884c-krgkh"
Oct 30 22:21:47.572324 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:47.572301    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7bdcd87e-ab76-4af6-ae7b-37e1384b6406-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-cf1826-5f6ddbf4c7-7mrgs\" (UID: \"7bdcd87e-ab76-4af6-ae7b-37e1384b6406\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-cf1826-5f6ddbf4c7-7mrgs"
Oct 30 22:21:47.589732 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:47.589702    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w2-cf1826-dockercfg-r2q6t\""
Oct 30 22:21:47.591939 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:47.591911    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:21:47.673362 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:47.673320    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gkjlr\" (UniqueName: \"kubernetes.io/projected/4d203fdf-00d0-4d25-8a63-c2f355d8d03f-kube-api-access-gkjlr\") pod \"payload-pre-processing-e2e-shared-a-w2-cf1826-556566884c-krgkh\" (UID: \"4d203fdf-00d0-4d25-8a63-c2f355d8d03f\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-cf1826-556566884c-krgkh"
Oct 30 22:21:47.673644 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:47.673375    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4d203fdf-00d0-4d25-8a63-c2f355d8d03f-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-cf1826-556566884c-krgkh\" (UID: \"4d203fdf-00d0-4d25-8a63-c2f355d8d03f\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-cf1826-556566884c-krgkh"
Oct 30 22:21:47.673644 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:47.673587    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7bdcd87e-ab76-4af6-ae7b-37e1384b6406-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-cf1826-5f6ddbf4c7-7mrgs\" (UID: \"7bdcd87e-ab76-4af6-ae7b-37e1384b6406\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-cf1826-5f6ddbf4c7-7mrgs"
Oct 30 22:21:47.673843 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:47.673802    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wvwqx\" (UniqueName: \"kubernetes.io/projected/7bdcd87e-ab76-4af6-ae7b-37e1384b6406-kube-api-access-wvwqx\") pod \"payload-processing-e2e-shared-a-w2-cf1826-5f6ddbf4c7-7mrgs\" (UID: \"7bdcd87e-ab76-4af6-ae7b-37e1384b6406\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-cf1826-5f6ddbf4c7-7mrgs"
Oct 30 22:21:47.674268 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:47.674240    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4d203fdf-00d0-4d25-8a63-c2f355d8d03f-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-cf1826-556566884c-krgkh\" (UID: \"4d203fdf-00d0-4d25-8a63-c2f355d8d03f\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-cf1826-556566884c-krgkh"
Oct 30 22:21:47.674606 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:47.674587    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7bdcd87e-ab76-4af6-ae7b-37e1384b6406-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-cf1826-5f6ddbf4c7-7mrgs\" (UID: \"7bdcd87e-ab76-4af6-ae7b-37e1384b6406\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-cf1826-5f6ddbf4c7-7mrgs"
Oct 30 22:21:47.766742 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:47.763869    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wvwqx\" (UniqueName: \"kubernetes.io/projected/7bdcd87e-ab76-4af6-ae7b-37e1384b6406-kube-api-access-wvwqx\") pod \"payload-processing-e2e-shared-a-w2-cf1826-5f6ddbf4c7-7mrgs\" (UID: \"7bdcd87e-ab76-4af6-ae7b-37e1384b6406\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-cf1826-5f6ddbf4c7-7mrgs"
Oct 30 22:21:47.766742 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:47.763968    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gkjlr\" (UniqueName: \"kubernetes.io/projected/4d203fdf-00d0-4d25-8a63-c2f355d8d03f-kube-api-access-gkjlr\") pod \"payload-pre-processing-e2e-shared-a-w2-cf1826-556566884c-krgkh\" (UID: \"4d203fdf-00d0-4d25-8a63-c2f355d8d03f\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-cf1826-556566884c-krgkh"
Oct 30 22:21:47.795684 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:47.791976    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-cf1826-5f6ddbf4c7-7mrgs"
Oct 30 22:21:47.819684 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:47.811201    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-cf1826-556566884c-krgkh"
Oct 30 22:21:48.257690 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:48.257645    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-4da07739-openshift-default-6bd77c67d9-7wknn"
Oct 30 22:21:48.258048 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:48.258015    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-4da07739-openshift-default-6bd77c67d9-7wknn"
Oct 30 22:21:48.258693 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:48.258651    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-4da07739-openshift-default-6bd77c67d9-7wknn"
Oct 30 22:21:48.449320 ip-10-0-136-210 kubenswrapper[5510]: W1030 22:21:48.449292    5510 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7bdcd87e_ab76_4af6_ae7b_37e1384b6406.slice/crio-f48094b7cb3a8f2570286e4d21c4511aa6a48730edc65610adc4898095e97cc5 WatchSource:0}: Error finding container f48094b7cb3a8f2570286e4d21c4511aa6a48730edc65610adc4898095e97cc5: Status 404 returned error can't find the container with id f48094b7cb3a8f2570286e4d21c4511aa6a48730edc65610adc4898095e97cc5
Oct 30 22:21:48.454339 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:48.454310    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-cf1826-5f6ddbf4c7-7mrgs"]
Oct 30 22:21:48.471333 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:48.471311    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-cf1826-556566884c-krgkh"]
Oct 30 22:21:48.471473 ip-10-0-136-210 kubenswrapper[5510]: W1030 22:21:48.471452    5510 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d203fdf_00d0_4d25_8a63_c2f355d8d03f.slice/crio-e23c244757a3c640dde799f9527847e41fb477ed690e51693e4d4e02e50fa080 WatchSource:0}: Error finding container e23c244757a3c640dde799f9527847e41fb477ed690e51693e4d4e02e50fa080: Status 404 returned error can't find the container with id e23c244757a3c640dde799f9527847e41fb477ed690e51693e4d4e02e50fa080
Oct 30 22:21:48.937694 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:48.935366    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:21:49.072224 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:49.072184    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-cf1826-556566884c-krgkh" event={"ID":"4d203fdf-00d0-4d25-8a63-c2f355d8d03f","Type":"ContainerStarted","Data":"5976d932463b35a75223277b5967fb906832deb69137e65ef0ad08e49ae6c317"}
Oct 30 22:21:49.072464 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:49.072259    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-cf1826-556566884c-krgkh"
Oct 30 22:21:49.072464 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:49.072284    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-cf1826-556566884c-krgkh" event={"ID":"4d203fdf-00d0-4d25-8a63-c2f355d8d03f","Type":"ContainerStarted","Data":"e23c244757a3c640dde799f9527847e41fb477ed690e51693e4d4e02e50fa080"}
Oct 30 22:21:49.074076 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:49.074047    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-cf1826-5f6ddbf4c7-7mrgs" event={"ID":"7bdcd87e-ab76-4af6-ae7b-37e1384b6406","Type":"ContainerStarted","Data":"b77c2289be901656adec84752fdad0880eb9e5002f510d139d1a0367c9a5ee85"}
Oct 30 22:21:49.074205 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:49.074099    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-cf1826-5f6ddbf4c7-7mrgs" event={"ID":"7bdcd87e-ab76-4af6-ae7b-37e1384b6406","Type":"ContainerStarted","Data":"f48094b7cb3a8f2570286e4d21c4511aa6a48730edc65610adc4898095e97cc5"}
Oct 30 22:21:49.074205 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:49.074130    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-cf1826-5f6ddbf4c7-7mrgs"
Oct 30 22:21:49.184671 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:49.184605    5510 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-cf1826-556566884c-krgkh" podStartSLOduration=2.184591342 podStartE2EDuration="2.184591342s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:21:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:21:49.182219466 +0000 UTC m=+1889.743029482" watchObservedRunningTime="2026-10-30 22:21:49.184591342 +0000 UTC m=+1889.745401346"
Oct 30 22:21:49.297609 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:49.297546    5510 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-cf1826-5f6ddbf4c7-7mrgs" podStartSLOduration=3.297532074 podStartE2EDuration="3.297532074s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:21:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:21:49.293396527 +0000 UTC m=+1889.854206552" watchObservedRunningTime="2026-10-30 22:21:49.297532074 +0000 UTC m=+1889.858342080"
Oct 30 22:21:51.040766 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:51.040718    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:21:51.253861 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:51.253826    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:21:52.317054 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:52.317023    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-l5ndh"]
Oct 30 22:21:52.317556 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:52.317253    5510 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-l5ndh" podUID="2344709a-1a48-417c-ae98-2b462bf30516" containerName="istio-proxy" containerID="cri-o://7c524cc3c61f0a271fdc614f3ce555c8653983b2f00bc3c5273fbc29d0e93e2c" gracePeriod=30
Oct 30 22:21:52.365393 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:52.365353    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-ccsc2"]
Oct 30 22:21:52.365652 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:52.365621    5510 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-ccsc2" podUID="caef4e8d-7dcc-455a-94a2-4bb341662a88" containerName="istio-proxy" containerID="cri-o://6bcb261945056982cd70d737d4fd06cd141d27c20163e0534990323500d8654d" gracePeriod=30
Oct 30 22:21:52.369367 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:52.369336    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-n6tv4"]
Oct 30 22:21:52.369599 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:52.369562    5510 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-n6tv4" podUID="e15eace0-bc5d-43ec-b352-74536b28a56c" containerName="istio-proxy" containerID="cri-o://3e27cd938544e7aa190cb973c1a2b4a48728b899be1b90d874d262abdd407a01" gracePeriod=30
Oct 30 22:21:52.371460 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:52.371435    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:21:52.726738 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:52.726377    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:21:54.139505 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:54.138992    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:21:54.377062 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:54.377031    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:21:54.878708 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:54.876352    5510 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-7f76c4-openshift-default-5969946659-5dwws"]
Oct 30 22:21:54.927678 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:54.922323    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-7f76c4-openshift-default-5969946659-5dwws"
Oct 30 22:21:55.044695 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:55.018609    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:21:55.076456 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:55.045158    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-7f76c4-openshift-default-dockercfg-vv4bv\""
Oct 30 22:21:55.076456 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:55.055111    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/11bc04c5-986f-47f9-93d1-0f803b775eb5-workload-certs\") pod \"e2e-api-a-7f76c4-openshift-default-5969946659-5dwws\" (UID: \"11bc04c5-986f-47f9-93d1-0f803b775eb5\") " pod="openshift-ingress/e2e-api-a-7f76c4-openshift-default-5969946659-5dwws"
Oct 30 22:21:55.100892 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:55.100854    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/11bc04c5-986f-47f9-93d1-0f803b775eb5-istio-podinfo\") pod \"e2e-api-a-7f76c4-openshift-default-5969946659-5dwws\" (UID: \"11bc04c5-986f-47f9-93d1-0f803b775eb5\") " pod="openshift-ingress/e2e-api-a-7f76c4-openshift-default-5969946659-5dwws"
Oct 30 22:21:55.101169 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:55.101155    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/11bc04c5-986f-47f9-93d1-0f803b775eb5-istiod-ca-cert\") pod \"e2e-api-a-7f76c4-openshift-default-5969946659-5dwws\" (UID: \"11bc04c5-986f-47f9-93d1-0f803b775eb5\") " pod="openshift-ingress/e2e-api-a-7f76c4-openshift-default-5969946659-5dwws"
Oct 30 22:21:55.101523 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:55.101505    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/11bc04c5-986f-47f9-93d1-0f803b775eb5-credential-socket\") pod \"e2e-api-a-7f76c4-openshift-default-5969946659-5dwws\" (UID: \"11bc04c5-986f-47f9-93d1-0f803b775eb5\") " pod="openshift-ingress/e2e-api-a-7f76c4-openshift-default-5969946659-5dwws"
Oct 30 22:21:55.101851 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:55.101823    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rdx2c\" (UniqueName: \"kubernetes.io/projected/11bc04c5-986f-47f9-93d1-0f803b775eb5-kube-api-access-rdx2c\") pod \"e2e-api-a-7f76c4-openshift-default-5969946659-5dwws\" (UID: \"11bc04c5-986f-47f9-93d1-0f803b775eb5\") " pod="openshift-ingress/e2e-api-a-7f76c4-openshift-default-5969946659-5dwws"
Oct 30 22:21:55.247732 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:55.247071    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/11bc04c5-986f-47f9-93d1-0f803b775eb5-istio-data\") pod \"e2e-api-a-7f76c4-openshift-default-5969946659-5dwws\" (UID: \"11bc04c5-986f-47f9-93d1-0f803b775eb5\") " pod="openshift-ingress/e2e-api-a-7f76c4-openshift-default-5969946659-5dwws"
Oct 30 22:21:55.247732 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:55.247349    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/11bc04c5-986f-47f9-93d1-0f803b775eb5-istio-envoy\") pod \"e2e-api-a-7f76c4-openshift-default-5969946659-5dwws\" (UID: \"11bc04c5-986f-47f9-93d1-0f803b775eb5\") " pod="openshift-ingress/e2e-api-a-7f76c4-openshift-default-5969946659-5dwws"
Oct 30 22:21:55.247732 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:55.247434    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/11bc04c5-986f-47f9-93d1-0f803b775eb5-istio-token\") pod \"e2e-api-a-7f76c4-openshift-default-5969946659-5dwws\" (UID: \"11bc04c5-986f-47f9-93d1-0f803b775eb5\") " pod="openshift-ingress/e2e-api-a-7f76c4-openshift-default-5969946659-5dwws"
Oct 30 22:21:55.248315 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:55.247708    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/11bc04c5-986f-47f9-93d1-0f803b775eb5-workload-socket\") pod \"e2e-api-a-7f76c4-openshift-default-5969946659-5dwws\" (UID: \"11bc04c5-986f-47f9-93d1-0f803b775eb5\") " pod="openshift-ingress/e2e-api-a-7f76c4-openshift-default-5969946659-5dwws"
Oct 30 22:21:55.248315 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:55.144268    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-7f76c4-openshift-default-5969946659-5dwws"]
Oct 30 22:21:55.263715 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:55.263672    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:21:55.435623 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:55.435583    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/11bc04c5-986f-47f9-93d1-0f803b775eb5-workload-socket\") pod \"e2e-api-a-7f76c4-openshift-default-5969946659-5dwws\" (UID: \"11bc04c5-986f-47f9-93d1-0f803b775eb5\") " pod="openshift-ingress/e2e-api-a-7f76c4-openshift-default-5969946659-5dwws"
Oct 30 22:21:55.454677 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:55.453292    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/11bc04c5-986f-47f9-93d1-0f803b775eb5-workload-certs\") pod \"e2e-api-a-7f76c4-openshift-default-5969946659-5dwws\" (UID: \"11bc04c5-986f-47f9-93d1-0f803b775eb5\") " pod="openshift-ingress/e2e-api-a-7f76c4-openshift-default-5969946659-5dwws"
Oct 30 22:21:55.454677 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:55.453357    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/11bc04c5-986f-47f9-93d1-0f803b775eb5-istio-podinfo\") pod \"e2e-api-a-7f76c4-openshift-default-5969946659-5dwws\" (UID: \"11bc04c5-986f-47f9-93d1-0f803b775eb5\") " pod="openshift-ingress/e2e-api-a-7f76c4-openshift-default-5969946659-5dwws"
Oct 30 22:21:55.454677 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:55.436419    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/11bc04c5-986f-47f9-93d1-0f803b775eb5-workload-socket\") pod \"e2e-api-a-7f76c4-openshift-default-5969946659-5dwws\" (UID: \"11bc04c5-986f-47f9-93d1-0f803b775eb5\") " pod="openshift-ingress/e2e-api-a-7f76c4-openshift-default-5969946659-5dwws"
Oct 30 22:21:55.454677 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:55.453536    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/11bc04c5-986f-47f9-93d1-0f803b775eb5-istiod-ca-cert\") pod \"e2e-api-a-7f76c4-openshift-default-5969946659-5dwws\" (UID: \"11bc04c5-986f-47f9-93d1-0f803b775eb5\") " pod="openshift-ingress/e2e-api-a-7f76c4-openshift-default-5969946659-5dwws"
Oct 30 22:21:55.454677 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:55.453799    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/11bc04c5-986f-47f9-93d1-0f803b775eb5-credential-socket\") pod \"e2e-api-a-7f76c4-openshift-default-5969946659-5dwws\" (UID: \"11bc04c5-986f-47f9-93d1-0f803b775eb5\") " pod="openshift-ingress/e2e-api-a-7f76c4-openshift-default-5969946659-5dwws"
Oct 30 22:21:55.454677 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:55.453830    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rdx2c\" (UniqueName: \"kubernetes.io/projected/11bc04c5-986f-47f9-93d1-0f803b775eb5-kube-api-access-rdx2c\") pod \"e2e-api-a-7f76c4-openshift-default-5969946659-5dwws\" (UID: \"11bc04c5-986f-47f9-93d1-0f803b775eb5\") " pod="openshift-ingress/e2e-api-a-7f76c4-openshift-default-5969946659-5dwws"
Oct 30 22:21:55.454677 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:55.453842    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/11bc04c5-986f-47f9-93d1-0f803b775eb5-workload-certs\") pod \"e2e-api-a-7f76c4-openshift-default-5969946659-5dwws\" (UID: \"11bc04c5-986f-47f9-93d1-0f803b775eb5\") " pod="openshift-ingress/e2e-api-a-7f76c4-openshift-default-5969946659-5dwws"
Oct 30 22:21:55.454677 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:55.453860    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/11bc04c5-986f-47f9-93d1-0f803b775eb5-istio-data\") pod \"e2e-api-a-7f76c4-openshift-default-5969946659-5dwws\" (UID: \"11bc04c5-986f-47f9-93d1-0f803b775eb5\") " pod="openshift-ingress/e2e-api-a-7f76c4-openshift-default-5969946659-5dwws"
Oct 30 22:21:55.454677 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:55.454132    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/11bc04c5-986f-47f9-93d1-0f803b775eb5-istio-data\") pod \"e2e-api-a-7f76c4-openshift-default-5969946659-5dwws\" (UID: \"11bc04c5-986f-47f9-93d1-0f803b775eb5\") " pod="openshift-ingress/e2e-api-a-7f76c4-openshift-default-5969946659-5dwws"
Oct 30 22:21:55.455271 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:55.454162    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/11bc04c5-986f-47f9-93d1-0f803b775eb5-istio-envoy\") pod \"e2e-api-a-7f76c4-openshift-default-5969946659-5dwws\" (UID: \"11bc04c5-986f-47f9-93d1-0f803b775eb5\") " pod="openshift-ingress/e2e-api-a-7f76c4-openshift-default-5969946659-5dwws"
Oct 30 22:21:55.455271 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:55.454825    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/11bc04c5-986f-47f9-93d1-0f803b775eb5-istio-token\") pod \"e2e-api-a-7f76c4-openshift-default-5969946659-5dwws\" (UID: \"11bc04c5-986f-47f9-93d1-0f803b775eb5\") " pod="openshift-ingress/e2e-api-a-7f76c4-openshift-default-5969946659-5dwws"
Oct 30 22:21:55.456367 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:55.456344    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/11bc04c5-986f-47f9-93d1-0f803b775eb5-credential-socket\") pod \"e2e-api-a-7f76c4-openshift-default-5969946659-5dwws\" (UID: \"11bc04c5-986f-47f9-93d1-0f803b775eb5\") " pod="openshift-ingress/e2e-api-a-7f76c4-openshift-default-5969946659-5dwws"
Oct 30 22:21:55.459743 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:55.457164    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/11bc04c5-986f-47f9-93d1-0f803b775eb5-istiod-ca-cert\") pod \"e2e-api-a-7f76c4-openshift-default-5969946659-5dwws\" (UID: \"11bc04c5-986f-47f9-93d1-0f803b775eb5\") " pod="openshift-ingress/e2e-api-a-7f76c4-openshift-default-5969946659-5dwws"
Oct 30 22:21:55.484011 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:55.483975    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/11bc04c5-986f-47f9-93d1-0f803b775eb5-istio-envoy\") pod \"e2e-api-a-7f76c4-openshift-default-5969946659-5dwws\" (UID: \"11bc04c5-986f-47f9-93d1-0f803b775eb5\") " pod="openshift-ingress/e2e-api-a-7f76c4-openshift-default-5969946659-5dwws"
Oct 30 22:21:55.533691 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:55.532054    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/11bc04c5-986f-47f9-93d1-0f803b775eb5-istio-podinfo\") pod \"e2e-api-a-7f76c4-openshift-default-5969946659-5dwws\" (UID: \"11bc04c5-986f-47f9-93d1-0f803b775eb5\") " pod="openshift-ingress/e2e-api-a-7f76c4-openshift-default-5969946659-5dwws"
Oct 30 22:21:55.533691 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:55.533384    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:21:55.536687 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:55.535279    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rdx2c\" (UniqueName: \"kubernetes.io/projected/11bc04c5-986f-47f9-93d1-0f803b775eb5-kube-api-access-rdx2c\") pod \"e2e-api-a-7f76c4-openshift-default-5969946659-5dwws\" (UID: \"11bc04c5-986f-47f9-93d1-0f803b775eb5\") " pod="openshift-ingress/e2e-api-a-7f76c4-openshift-default-5969946659-5dwws"
Oct 30 22:21:55.539042 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:55.538990    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/11bc04c5-986f-47f9-93d1-0f803b775eb5-istio-token\") pod \"e2e-api-a-7f76c4-openshift-default-5969946659-5dwws\" (UID: \"11bc04c5-986f-47f9-93d1-0f803b775eb5\") " pod="openshift-ingress/e2e-api-a-7f76c4-openshift-default-5969946659-5dwws"
Oct 30 22:21:55.554296 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:55.554268    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-7f76c4-openshift-default-5969946659-5dwws"
Oct 30 22:21:55.986809 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:55.986732    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-7f76c4-openshift-default-5969946659-5dwws"]
Oct 30 22:21:56.018092 ip-10-0-136-210 kubenswrapper[5510]: W1030 22:21:56.018065    5510 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11bc04c5_986f_47f9_93d1_0f803b775eb5.slice/crio-f54a19eb0a6b8a12aaf386c5b58af2a50880c997015b29a2dae1d949fc66fac8 WatchSource:0}: Error finding container f54a19eb0a6b8a12aaf386c5b58af2a50880c997015b29a2dae1d949fc66fac8: Status 404 returned error can't find the container with id f54a19eb0a6b8a12aaf386c5b58af2a50880c997015b29a2dae1d949fc66fac8
Oct 30 22:21:56.020990 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:56.020957    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:21:56.021068 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:56.021025    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:21:56.021068 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:56.021055    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:21:56.068403 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:56.068369    5510 patch_prober.go:29] interesting pod/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-n6tv4 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 22:21:56.068557 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:56.068461    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-n6tv4" podUID="e15eace0-bc5d-43ec-b352-74536b28a56c" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 22:21:56.401080 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:56.401042    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-7f76c4-openshift-default-5969946659-5dwws" event={"ID":"11bc04c5-986f-47f9-93d1-0f803b775eb5","Type":"ContainerStarted","Data":"128a7a0a8c823b9a180f97a1b06a35c9fe3cc8c144345631ac24025deac1a841"}
Oct 30 22:21:56.401529 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:56.401108    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-7f76c4-openshift-default-5969946659-5dwws" event={"ID":"11bc04c5-986f-47f9-93d1-0f803b775eb5","Type":"ContainerStarted","Data":"f54a19eb0a6b8a12aaf386c5b58af2a50880c997015b29a2dae1d949fc66fac8"}
Oct 30 22:21:56.519772 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:56.519724    5510 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-7f76c4-openshift-default-5969946659-5dwws" podStartSLOduration=2.519708572 podStartE2EDuration="2.519708572s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:21:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:21:56.516651675 +0000 UTC m=+1897.077461681" watchObservedRunningTime="2026-10-30 22:21:56.519708572 +0000 UTC m=+1897.080518576"
Oct 30 22:21:56.554448 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:56.554415    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-7f76c4-openshift-default-5969946659-5dwws"
Oct 30 22:21:57.568851 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:57.568571    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-7f76c4-openshift-default-5969946659-5dwws"
Oct 30 22:21:57.702175 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:57.702148    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-l5ndh"
Oct 30 22:21:57.749342 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:57.749314    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-n6tv4"
Oct 30 22:21:57.753511 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:57.753487    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-ccsc2"
Oct 30 22:21:57.859131 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:57.859047    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e15eace0-bc5d-43ec-b352-74536b28a56c-kube-api-access-lq4gs\" (UniqueName: \"kubernetes.io/projected/e15eace0-bc5d-43ec-b352-74536b28a56c-kube-api-access-lq4gs\") pod \"e15eace0-bc5d-43ec-b352-74536b28a56c\" (UID: \"e15eace0-bc5d-43ec-b352-74536b28a56c\") "
Oct 30 22:21:57.859131 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:57.859085    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2344709a-1a48-417c-ae98-2b462bf30516-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2344709a-1a48-417c-ae98-2b462bf30516-workload-certs\") pod \"2344709a-1a48-417c-ae98-2b462bf30516\" (UID: \"2344709a-1a48-417c-ae98-2b462bf30516\") "
Oct 30 22:21:57.859377 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:57.859151    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e15eace0-bc5d-43ec-b352-74536b28a56c-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e15eace0-bc5d-43ec-b352-74536b28a56c-istiod-ca-cert\") pod \"e15eace0-bc5d-43ec-b352-74536b28a56c\" (UID: \"e15eace0-bc5d-43ec-b352-74536b28a56c\") "
Oct 30 22:21:57.859377 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:57.859177    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2344709a-1a48-417c-ae98-2b462bf30516-kube-api-access-4k7rz\" (UniqueName: \"kubernetes.io/projected/2344709a-1a48-417c-ae98-2b462bf30516-kube-api-access-4k7rz\") pod \"2344709a-1a48-417c-ae98-2b462bf30516\" (UID: \"2344709a-1a48-417c-ae98-2b462bf30516\") "
Oct 30 22:21:57.859377 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:57.859233    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2344709a-1a48-417c-ae98-2b462bf30516-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2344709a-1a48-417c-ae98-2b462bf30516-istiod-ca-cert\") pod \"2344709a-1a48-417c-ae98-2b462bf30516\" (UID: \"2344709a-1a48-417c-ae98-2b462bf30516\") "
Oct 30 22:21:57.859377 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:57.859297    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e15eace0-bc5d-43ec-b352-74536b28a56c-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e15eace0-bc5d-43ec-b352-74536b28a56c-istio-data\") pod \"e15eace0-bc5d-43ec-b352-74536b28a56c\" (UID: \"e15eace0-bc5d-43ec-b352-74536b28a56c\") "
Oct 30 22:21:57.859377 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:57.859324    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e15eace0-bc5d-43ec-b352-74536b28a56c-istio-token\" (UniqueName: \"kubernetes.io/projected/e15eace0-bc5d-43ec-b352-74536b28a56c-istio-token\") pod \"e15eace0-bc5d-43ec-b352-74536b28a56c\" (UID: \"e15eace0-bc5d-43ec-b352-74536b28a56c\") "
Oct 30 22:21:57.859377 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:57.859362    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e15eace0-bc5d-43ec-b352-74536b28a56c-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e15eace0-bc5d-43ec-b352-74536b28a56c-credential-socket\") pod \"e15eace0-bc5d-43ec-b352-74536b28a56c\" (UID: \"e15eace0-bc5d-43ec-b352-74536b28a56c\") "
Oct 30 22:21:57.859733 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:57.859402    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/caef4e8d-7dcc-455a-94a2-4bb341662a88-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/caef4e8d-7dcc-455a-94a2-4bb341662a88-istio-data\") pod \"caef4e8d-7dcc-455a-94a2-4bb341662a88\" (UID: \"caef4e8d-7dcc-455a-94a2-4bb341662a88\") "
Oct 30 22:21:57.859733 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:57.859423    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2344709a-1a48-417c-ae98-2b462bf30516-workload-certs" pod "2344709a-1a48-417c-ae98-2b462bf30516" (UID: "2344709a-1a48-417c-ae98-2b462bf30516"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:21:57.859733 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:57.859444    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/caef4e8d-7dcc-455a-94a2-4bb341662a88-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/caef4e8d-7dcc-455a-94a2-4bb341662a88-istio-envoy\") pod \"caef4e8d-7dcc-455a-94a2-4bb341662a88\" (UID: \"caef4e8d-7dcc-455a-94a2-4bb341662a88\") "
Oct 30 22:21:57.859733 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:57.859479    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e15eace0-bc5d-43ec-b352-74536b28a56c-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e15eace0-bc5d-43ec-b352-74536b28a56c-workload-socket\") pod \"e15eace0-bc5d-43ec-b352-74536b28a56c\" (UID: \"e15eace0-bc5d-43ec-b352-74536b28a56c\") "
Oct 30 22:21:57.859733 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:57.859548    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2344709a-1a48-417c-ae98-2b462bf30516-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2344709a-1a48-417c-ae98-2b462bf30516-istio-envoy\") pod \"2344709a-1a48-417c-ae98-2b462bf30516\" (UID: \"2344709a-1a48-417c-ae98-2b462bf30516\") "
Oct 30 22:21:57.859733 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:57.859592    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/caef4e8d-7dcc-455a-94a2-4bb341662a88-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/caef4e8d-7dcc-455a-94a2-4bb341662a88-istio-podinfo\") pod \"caef4e8d-7dcc-455a-94a2-4bb341662a88\" (UID: \"caef4e8d-7dcc-455a-94a2-4bb341662a88\") "
Oct 30 22:21:57.859733 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:57.859641    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e15eace0-bc5d-43ec-b352-74536b28a56c-istio-data" pod "e15eace0-bc5d-43ec-b352-74536b28a56c" (UID: "e15eace0-bc5d-43ec-b352-74536b28a56c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:21:57.859733 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:57.859690    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/caef4e8d-7dcc-455a-94a2-4bb341662a88-istio-token\" (UniqueName: \"kubernetes.io/projected/caef4e8d-7dcc-455a-94a2-4bb341662a88-istio-token\") pod \"caef4e8d-7dcc-455a-94a2-4bb341662a88\" (UID: \"caef4e8d-7dcc-455a-94a2-4bb341662a88\") "
Oct 30 22:21:57.860161 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:57.859757    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2344709a-1a48-417c-ae98-2b462bf30516-istio-token\" (UniqueName: \"kubernetes.io/projected/2344709a-1a48-417c-ae98-2b462bf30516-istio-token\") pod \"2344709a-1a48-417c-ae98-2b462bf30516\" (UID: \"2344709a-1a48-417c-ae98-2b462bf30516\") "
Oct 30 22:21:57.860161 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:57.859791    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/caef4e8d-7dcc-455a-94a2-4bb341662a88-kube-api-access-qmscm\" (UniqueName: \"kubernetes.io/projected/caef4e8d-7dcc-455a-94a2-4bb341662a88-kube-api-access-qmscm\") pod \"caef4e8d-7dcc-455a-94a2-4bb341662a88\" (UID: \"caef4e8d-7dcc-455a-94a2-4bb341662a88\") "
Oct 30 22:21:57.860161 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:57.859831    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2344709a-1a48-417c-ae98-2b462bf30516-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2344709a-1a48-417c-ae98-2b462bf30516-credential-socket\") pod \"2344709a-1a48-417c-ae98-2b462bf30516\" (UID: \"2344709a-1a48-417c-ae98-2b462bf30516\") "
Oct 30 22:21:57.860161 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:57.859870    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/caef4e8d-7dcc-455a-94a2-4bb341662a88-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/caef4e8d-7dcc-455a-94a2-4bb341662a88-workload-certs\") pod \"caef4e8d-7dcc-455a-94a2-4bb341662a88\" (UID: \"caef4e8d-7dcc-455a-94a2-4bb341662a88\") "
Oct 30 22:21:57.860161 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:57.859900    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/2344709a-1a48-417c-ae98-2b462bf30516-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2344709a-1a48-417c-ae98-2b462bf30516-istio-podinfo\") pod \"2344709a-1a48-417c-ae98-2b462bf30516\" (UID: \"2344709a-1a48-417c-ae98-2b462bf30516\") "
Oct 30 22:21:57.860161 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:57.859932    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e15eace0-bc5d-43ec-b352-74536b28a56c-istiod-ca-cert" pod "e15eace0-bc5d-43ec-b352-74536b28a56c" (UID: "e15eace0-bc5d-43ec-b352-74536b28a56c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:21:57.860161 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:57.859959    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/caef4e8d-7dcc-455a-94a2-4bb341662a88-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/caef4e8d-7dcc-455a-94a2-4bb341662a88-workload-socket\") pod \"caef4e8d-7dcc-455a-94a2-4bb341662a88\" (UID: \"caef4e8d-7dcc-455a-94a2-4bb341662a88\") "
Oct 30 22:21:57.860161 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:57.859998    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/caef4e8d-7dcc-455a-94a2-4bb341662a88-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/caef4e8d-7dcc-455a-94a2-4bb341662a88-credential-socket\") pod \"caef4e8d-7dcc-455a-94a2-4bb341662a88\" (UID: \"caef4e8d-7dcc-455a-94a2-4bb341662a88\") "
Oct 30 22:21:57.860161 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:57.860086    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e15eace0-bc5d-43ec-b352-74536b28a56c-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e15eace0-bc5d-43ec-b352-74536b28a56c-istio-envoy\") pod \"e15eace0-bc5d-43ec-b352-74536b28a56c\" (UID: \"e15eace0-bc5d-43ec-b352-74536b28a56c\") "
Oct 30 22:21:57.860161 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:57.860124    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2344709a-1a48-417c-ae98-2b462bf30516-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2344709a-1a48-417c-ae98-2b462bf30516-istio-data\") pod \"2344709a-1a48-417c-ae98-2b462bf30516\" (UID: \"2344709a-1a48-417c-ae98-2b462bf30516\") "
Oct 30 22:21:57.860161 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:57.860157    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/e15eace0-bc5d-43ec-b352-74536b28a56c-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e15eace0-bc5d-43ec-b352-74536b28a56c-istio-podinfo\") pod \"e15eace0-bc5d-43ec-b352-74536b28a56c\" (UID: \"e15eace0-bc5d-43ec-b352-74536b28a56c\") "
Oct 30 22:21:57.860745 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:57.860225    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e15eace0-bc5d-43ec-b352-74536b28a56c-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e15eace0-bc5d-43ec-b352-74536b28a56c-workload-certs\") pod \"e15eace0-bc5d-43ec-b352-74536b28a56c\" (UID: \"e15eace0-bc5d-43ec-b352-74536b28a56c\") "
Oct 30 22:21:57.860745 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:57.860305    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2344709a-1a48-417c-ae98-2b462bf30516-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2344709a-1a48-417c-ae98-2b462bf30516-workload-socket\") pod \"2344709a-1a48-417c-ae98-2b462bf30516\" (UID: \"2344709a-1a48-417c-ae98-2b462bf30516\") "
Oct 30 22:21:57.860745 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:57.860414    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/caef4e8d-7dcc-455a-94a2-4bb341662a88-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/caef4e8d-7dcc-455a-94a2-4bb341662a88-istiod-ca-cert\") pod \"caef4e8d-7dcc-455a-94a2-4bb341662a88\" (UID: \"caef4e8d-7dcc-455a-94a2-4bb341662a88\") "
Oct 30 22:21:57.861473 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:57.861444    5510 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2344709a-1a48-417c-ae98-2b462bf30516-workload-certs\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:21:57.861631 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:57.861478    5510 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e15eace0-bc5d-43ec-b352-74536b28a56c-istiod-ca-cert\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:21:57.861631 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:57.861495    5510 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e15eace0-bc5d-43ec-b352-74536b28a56c-istio-data\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:21:57.862710 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:57.861989    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/caef4e8d-7dcc-455a-94a2-4bb341662a88-credential-socket" pod "caef4e8d-7dcc-455a-94a2-4bb341662a88" (UID: "caef4e8d-7dcc-455a-94a2-4bb341662a88"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:21:57.862710 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:57.862133    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/caef4e8d-7dcc-455a-94a2-4bb341662a88-workload-certs" pod "caef4e8d-7dcc-455a-94a2-4bb341662a88" (UID: "caef4e8d-7dcc-455a-94a2-4bb341662a88"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:21:57.862710 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:57.862332    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2344709a-1a48-417c-ae98-2b462bf30516-credential-socket" pod "2344709a-1a48-417c-ae98-2b462bf30516" (UID: "2344709a-1a48-417c-ae98-2b462bf30516"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:21:57.862710 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:57.862456    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e15eace0-bc5d-43ec-b352-74536b28a56c-workload-socket" pod "e15eace0-bc5d-43ec-b352-74536b28a56c" (UID: "e15eace0-bc5d-43ec-b352-74536b28a56c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:21:57.862710 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:57.862472    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e15eace0-bc5d-43ec-b352-74536b28a56c-kube-api-access-lq4gs" pod "e15eace0-bc5d-43ec-b352-74536b28a56c" (UID: "e15eace0-bc5d-43ec-b352-74536b28a56c"). InnerVolumeSpecName "kube-api-access-lq4gs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:21:57.863050 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:57.862710    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/caef4e8d-7dcc-455a-94a2-4bb341662a88-istio-data" pod "caef4e8d-7dcc-455a-94a2-4bb341662a88" (UID: "caef4e8d-7dcc-455a-94a2-4bb341662a88"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:21:57.863050 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:57.862909    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e15eace0-bc5d-43ec-b352-74536b28a56c-credential-socket" pod "e15eace0-bc5d-43ec-b352-74536b28a56c" (UID: "e15eace0-bc5d-43ec-b352-74536b28a56c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:21:57.863050 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:57.862917    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/caef4e8d-7dcc-455a-94a2-4bb341662a88-istiod-ca-cert" pod "caef4e8d-7dcc-455a-94a2-4bb341662a88" (UID: "caef4e8d-7dcc-455a-94a2-4bb341662a88"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:21:57.863218 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:57.863113    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/caef4e8d-7dcc-455a-94a2-4bb341662a88-workload-socket" pod "caef4e8d-7dcc-455a-94a2-4bb341662a88" (UID: "caef4e8d-7dcc-455a-94a2-4bb341662a88"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:21:57.863218 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:57.863168    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e15eace0-bc5d-43ec-b352-74536b28a56c-workload-certs" pod "e15eace0-bc5d-43ec-b352-74536b28a56c" (UID: "e15eace0-bc5d-43ec-b352-74536b28a56c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:21:57.863394 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:57.863374    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2344709a-1a48-417c-ae98-2b462bf30516-workload-socket" pod "2344709a-1a48-417c-ae98-2b462bf30516" (UID: "2344709a-1a48-417c-ae98-2b462bf30516"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:21:57.864940 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:57.864305    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2344709a-1a48-417c-ae98-2b462bf30516-kube-api-access-4k7rz" pod "2344709a-1a48-417c-ae98-2b462bf30516" (UID: "2344709a-1a48-417c-ae98-2b462bf30516"). InnerVolumeSpecName "kube-api-access-4k7rz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:21:57.864940 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:57.864616    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2344709a-1a48-417c-ae98-2b462bf30516-istiod-ca-cert" pod "2344709a-1a48-417c-ae98-2b462bf30516" (UID: "2344709a-1a48-417c-ae98-2b462bf30516"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:21:57.864940 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:57.864861    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2344709a-1a48-417c-ae98-2b462bf30516-istio-data" pod "2344709a-1a48-417c-ae98-2b462bf30516" (UID: "2344709a-1a48-417c-ae98-2b462bf30516"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:21:57.866991 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:57.866970    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2344709a-1a48-417c-ae98-2b462bf30516-istio-envoy" pod "2344709a-1a48-417c-ae98-2b462bf30516" (UID: "2344709a-1a48-417c-ae98-2b462bf30516"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:21:57.867091 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:57.867065    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/caef4e8d-7dcc-455a-94a2-4bb341662a88-kube-api-access-qmscm" pod "caef4e8d-7dcc-455a-94a2-4bb341662a88" (UID: "caef4e8d-7dcc-455a-94a2-4bb341662a88"). InnerVolumeSpecName "kube-api-access-qmscm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:21:57.867160 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:57.867120    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e15eace0-bc5d-43ec-b352-74536b28a56c-istio-podinfo" pod "e15eace0-bc5d-43ec-b352-74536b28a56c" (UID: "e15eace0-bc5d-43ec-b352-74536b28a56c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:21:57.867223 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:57.867168    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/caef4e8d-7dcc-455a-94a2-4bb341662a88-istio-podinfo" pod "caef4e8d-7dcc-455a-94a2-4bb341662a88" (UID: "caef4e8d-7dcc-455a-94a2-4bb341662a88"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:21:57.867600 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:57.867575    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/caef4e8d-7dcc-455a-94a2-4bb341662a88-istio-token" pod "caef4e8d-7dcc-455a-94a2-4bb341662a88" (UID: "caef4e8d-7dcc-455a-94a2-4bb341662a88"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:21:57.868368 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:57.867811    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/caef4e8d-7dcc-455a-94a2-4bb341662a88-istio-envoy" pod "caef4e8d-7dcc-455a-94a2-4bb341662a88" (UID: "caef4e8d-7dcc-455a-94a2-4bb341662a88"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:21:57.868368 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:57.868022    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2344709a-1a48-417c-ae98-2b462bf30516-istio-podinfo" pod "2344709a-1a48-417c-ae98-2b462bf30516" (UID: "2344709a-1a48-417c-ae98-2b462bf30516"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:21:57.869795 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:57.869770    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e15eace0-bc5d-43ec-b352-74536b28a56c-istio-token" pod "e15eace0-bc5d-43ec-b352-74536b28a56c" (UID: "e15eace0-bc5d-43ec-b352-74536b28a56c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:21:57.869915 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:57.869777    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e15eace0-bc5d-43ec-b352-74536b28a56c-istio-envoy" pod "e15eace0-bc5d-43ec-b352-74536b28a56c" (UID: "e15eace0-bc5d-43ec-b352-74536b28a56c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:21:57.869974 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:57.869936    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2344709a-1a48-417c-ae98-2b462bf30516-istio-token" pod "2344709a-1a48-417c-ae98-2b462bf30516" (UID: "2344709a-1a48-417c-ae98-2b462bf30516"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:21:57.970843 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:57.970811    5510 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e15eace0-bc5d-43ec-b352-74536b28a56c-credential-socket\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:21:57.970843 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:57.970843    5510 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/caef4e8d-7dcc-455a-94a2-4bb341662a88-istio-data\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:21:57.971192 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:57.970857    5510 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/caef4e8d-7dcc-455a-94a2-4bb341662a88-istio-envoy\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:21:57.971192 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:57.970871    5510 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e15eace0-bc5d-43ec-b352-74536b28a56c-workload-socket\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:21:57.971192 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:57.970886    5510 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2344709a-1a48-417c-ae98-2b462bf30516-istio-envoy\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:21:57.971192 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:57.970899    5510 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/caef4e8d-7dcc-455a-94a2-4bb341662a88-istio-podinfo\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:21:57.971192 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:57.970911    5510 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/caef4e8d-7dcc-455a-94a2-4bb341662a88-istio-token\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:21:57.971192 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:57.970924    5510 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2344709a-1a48-417c-ae98-2b462bf30516-istio-token\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:21:57.971192 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:57.970939    5510 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qmscm\" (UniqueName: \"kubernetes.io/projected/caef4e8d-7dcc-455a-94a2-4bb341662a88-kube-api-access-qmscm\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:21:57.971192 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:57.970953    5510 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2344709a-1a48-417c-ae98-2b462bf30516-credential-socket\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:21:57.971192 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:57.970967    5510 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/caef4e8d-7dcc-455a-94a2-4bb341662a88-workload-certs\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:21:57.971192 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:57.970979    5510 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2344709a-1a48-417c-ae98-2b462bf30516-istio-podinfo\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:21:57.971192 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:57.970993    5510 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/caef4e8d-7dcc-455a-94a2-4bb341662a88-workload-socket\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:21:57.971192 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:57.971006    5510 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/caef4e8d-7dcc-455a-94a2-4bb341662a88-credential-socket\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:21:57.971192 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:57.971019    5510 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e15eace0-bc5d-43ec-b352-74536b28a56c-istio-envoy\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:21:57.971192 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:57.971031    5510 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2344709a-1a48-417c-ae98-2b462bf30516-istio-data\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:21:57.971192 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:57.971043    5510 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e15eace0-bc5d-43ec-b352-74536b28a56c-istio-podinfo\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:21:57.971192 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:57.971056    5510 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e15eace0-bc5d-43ec-b352-74536b28a56c-workload-certs\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:21:57.971192 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:57.971069    5510 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2344709a-1a48-417c-ae98-2b462bf30516-workload-socket\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:21:57.971192 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:57.971082    5510 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/caef4e8d-7dcc-455a-94a2-4bb341662a88-istiod-ca-cert\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:21:57.971192 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:57.971095    5510 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lq4gs\" (UniqueName: \"kubernetes.io/projected/e15eace0-bc5d-43ec-b352-74536b28a56c-kube-api-access-lq4gs\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:21:57.971192 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:57.971110    5510 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4k7rz\" (UniqueName: \"kubernetes.io/projected/2344709a-1a48-417c-ae98-2b462bf30516-kube-api-access-4k7rz\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:21:57.971192 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:57.971124    5510 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2344709a-1a48-417c-ae98-2b462bf30516-istiod-ca-cert\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:21:57.971192 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:57.971138    5510 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e15eace0-bc5d-43ec-b352-74536b28a56c-istio-token\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:21:58.558679 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:58.552080    5510 generic.go:356] "Generic (PLEG): container finished" podID="caef4e8d-7dcc-455a-94a2-4bb341662a88" containerID="6bcb261945056982cd70d737d4fd06cd141d27c20163e0534990323500d8654d" exitCode=0
Oct 30 22:21:58.558679 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:58.552272    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-ccsc2" event={"ID":"caef4e8d-7dcc-455a-94a2-4bb341662a88","Type":"ContainerDied","Data":"6bcb261945056982cd70d737d4fd06cd141d27c20163e0534990323500d8654d"}
Oct 30 22:21:58.558679 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:58.552316    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-ccsc2" event={"ID":"caef4e8d-7dcc-455a-94a2-4bb341662a88","Type":"ContainerDied","Data":"d228450133636e3fce07ad40ff3ab39cb70c8888f1b7a9dbc5ef2ae073c4dcc5"}
Oct 30 22:21:58.558679 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:58.552346    5510 scope.go:122] "RemoveContainer" containerID="6bcb261945056982cd70d737d4fd06cd141d27c20163e0534990323500d8654d"
Oct 30 22:21:58.558679 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:58.552511    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-ccsc2"
Oct 30 22:21:58.571857 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:58.571824    5510 generic.go:356] "Generic (PLEG): container finished" podID="e15eace0-bc5d-43ec-b352-74536b28a56c" containerID="3e27cd938544e7aa190cb973c1a2b4a48728b899be1b90d874d262abdd407a01" exitCode=0
Oct 30 22:21:58.574693 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:58.572485    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-n6tv4" event={"ID":"e15eace0-bc5d-43ec-b352-74536b28a56c","Type":"ContainerDied","Data":"3e27cd938544e7aa190cb973c1a2b4a48728b899be1b90d874d262abdd407a01"}
Oct 30 22:21:58.574693 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:58.572552    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-n6tv4" event={"ID":"e15eace0-bc5d-43ec-b352-74536b28a56c","Type":"ContainerDied","Data":"874ae3914468dfb0d1ac37c819397d91444ab864245bba5659fd8761abd9d873"}
Oct 30 22:21:58.574693 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:58.572640    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-n6tv4"
Oct 30 22:21:58.606014 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:58.605990    5510 scope.go:122] "RemoveContainer" containerID="0bae0294bc1a482d1e9171428212b20af595fa61ee28059daa35724bb102a910"
Oct 30 22:21:58.608494 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:58.608467    5510 generic.go:356] "Generic (PLEG): container finished" podID="2344709a-1a48-417c-ae98-2b462bf30516" containerID="7c524cc3c61f0a271fdc614f3ce555c8653983b2f00bc3c5273fbc29d0e93e2c" exitCode=0
Oct 30 22:21:58.608986 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:58.608956    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-l5ndh"
Oct 30 22:21:58.609808 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:58.609785    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-l5ndh" event={"ID":"2344709a-1a48-417c-ae98-2b462bf30516","Type":"ContainerDied","Data":"7c524cc3c61f0a271fdc614f3ce555c8653983b2f00bc3c5273fbc29d0e93e2c"}
Oct 30 22:21:58.609999 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:58.609835    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-l5ndh" event={"ID":"2344709a-1a48-417c-ae98-2b462bf30516","Type":"ContainerDied","Data":"73b33eaac8def512e1877ae3ae20913170663d58911665bc47ebef15b60528bf"}
Oct 30 22:21:58.609999 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:58.609878    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-7f76c4-openshift-default-5969946659-5dwws"
Oct 30 22:21:58.610888 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:58.610841    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-7f76c4-openshift-default-5969946659-5dwws"
Oct 30 22:21:58.624004 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:58.623986    5510 scope.go:122] "RemoveContainer" containerID="6bcb261945056982cd70d737d4fd06cd141d27c20163e0534990323500d8654d"
Oct 30 22:21:58.624344 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:21:58.624298    5510 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6bcb261945056982cd70d737d4fd06cd141d27c20163e0534990323500d8654d\": container with ID starting with 6bcb261945056982cd70d737d4fd06cd141d27c20163e0534990323500d8654d not found: ID does not exist" containerID="6bcb261945056982cd70d737d4fd06cd141d27c20163e0534990323500d8654d"
Oct 30 22:21:58.624404 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:58.624354    5510 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6bcb261945056982cd70d737d4fd06cd141d27c20163e0534990323500d8654d"} err="failed to get container status \"6bcb261945056982cd70d737d4fd06cd141d27c20163e0534990323500d8654d\": rpc error: code = NotFound desc = could not find container \"6bcb261945056982cd70d737d4fd06cd141d27c20163e0534990323500d8654d\": container with ID starting with 6bcb261945056982cd70d737d4fd06cd141d27c20163e0534990323500d8654d not found: ID does not exist"
Oct 30 22:21:58.624404 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:58.624379    5510 scope.go:122] "RemoveContainer" containerID="0bae0294bc1a482d1e9171428212b20af595fa61ee28059daa35724bb102a910"
Oct 30 22:21:58.624666 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:21:58.624632    5510 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0bae0294bc1a482d1e9171428212b20af595fa61ee28059daa35724bb102a910\": container with ID starting with 0bae0294bc1a482d1e9171428212b20af595fa61ee28059daa35724bb102a910 not found: ID does not exist" containerID="0bae0294bc1a482d1e9171428212b20af595fa61ee28059daa35724bb102a910"
Oct 30 22:21:58.624752 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:58.624674    5510 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0bae0294bc1a482d1e9171428212b20af595fa61ee28059daa35724bb102a910"} err="failed to get container status \"0bae0294bc1a482d1e9171428212b20af595fa61ee28059daa35724bb102a910\": rpc error: code = NotFound desc = could not find container \"0bae0294bc1a482d1e9171428212b20af595fa61ee28059daa35724bb102a910\": container with ID starting with 0bae0294bc1a482d1e9171428212b20af595fa61ee28059daa35724bb102a910 not found: ID does not exist"
Oct 30 22:21:58.624752 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:58.624695    5510 scope.go:122] "RemoveContainer" containerID="3e27cd938544e7aa190cb973c1a2b4a48728b899be1b90d874d262abdd407a01"
Oct 30 22:21:58.636781 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:58.636756    5510 scope.go:122] "RemoveContainer" containerID="3e27cd938544e7aa190cb973c1a2b4a48728b899be1b90d874d262abdd407a01"
Oct 30 22:21:58.637063 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:21:58.637042    5510 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3e27cd938544e7aa190cb973c1a2b4a48728b899be1b90d874d262abdd407a01\": container with ID starting with 3e27cd938544e7aa190cb973c1a2b4a48728b899be1b90d874d262abdd407a01 not found: ID does not exist" containerID="3e27cd938544e7aa190cb973c1a2b4a48728b899be1b90d874d262abdd407a01"
Oct 30 22:21:58.637136 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:58.637068    5510 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3e27cd938544e7aa190cb973c1a2b4a48728b899be1b90d874d262abdd407a01"} err="failed to get container status \"3e27cd938544e7aa190cb973c1a2b4a48728b899be1b90d874d262abdd407a01\": rpc error: code = NotFound desc = could not find container \"3e27cd938544e7aa190cb973c1a2b4a48728b899be1b90d874d262abdd407a01\": container with ID starting with 3e27cd938544e7aa190cb973c1a2b4a48728b899be1b90d874d262abdd407a01 not found: ID does not exist"
Oct 30 22:21:58.637136 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:58.637084    5510 scope.go:122] "RemoveContainer" containerID="7c524cc3c61f0a271fdc614f3ce555c8653983b2f00bc3c5273fbc29d0e93e2c"
Oct 30 22:21:58.648008 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:58.647990    5510 scope.go:122] "RemoveContainer" containerID="7c524cc3c61f0a271fdc614f3ce555c8653983b2f00bc3c5273fbc29d0e93e2c"
Oct 30 22:21:58.648345 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:21:58.648324    5510 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7c524cc3c61f0a271fdc614f3ce555c8653983b2f00bc3c5273fbc29d0e93e2c\": container with ID starting with 7c524cc3c61f0a271fdc614f3ce555c8653983b2f00bc3c5273fbc29d0e93e2c not found: ID does not exist" containerID="7c524cc3c61f0a271fdc614f3ce555c8653983b2f00bc3c5273fbc29d0e93e2c"
Oct 30 22:21:58.648416 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:58.648353    5510 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7c524cc3c61f0a271fdc614f3ce555c8653983b2f00bc3c5273fbc29d0e93e2c"} err="failed to get container status \"7c524cc3c61f0a271fdc614f3ce555c8653983b2f00bc3c5273fbc29d0e93e2c\": rpc error: code = NotFound desc = could not find container \"7c524cc3c61f0a271fdc614f3ce555c8653983b2f00bc3c5273fbc29d0e93e2c\": container with ID starting with 7c524cc3c61f0a271fdc614f3ce555c8653983b2f00bc3c5273fbc29d0e93e2c not found: ID does not exist"
Oct 30 22:21:58.770644 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:58.770611    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-n6tv4"]
Oct 30 22:21:58.814651 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:58.814575    5510 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-n6tv4"]
Oct 30 22:21:58.882333 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:58.882298    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-ccsc2"]
Oct 30 22:21:58.892230 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:58.892200    5510 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-ccsc2"]
Oct 30 22:21:58.997714 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:58.995522    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-l5ndh"]
Oct 30 22:21:58.999577 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:58.999545    5510 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-l5ndh"]
Oct 30 22:21:59.401200 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:59.401154    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:21:59.578229 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:59.578194    5510 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2344709a-1a48-417c-ae98-2b462bf30516" path="/var/lib/kubelet/pods/2344709a-1a48-417c-ae98-2b462bf30516/volumes"
Oct 30 22:21:59.578806 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:59.578787    5510 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="caef4e8d-7dcc-455a-94a2-4bb341662a88" path="/var/lib/kubelet/pods/caef4e8d-7dcc-455a-94a2-4bb341662a88/volumes"
Oct 30 22:21:59.579264 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:59.579251    5510 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e15eace0-bc5d-43ec-b352-74536b28a56c" path="/var/lib/kubelet/pods/e15eace0-bc5d-43ec-b352-74536b28a56c/volumes"
Oct 30 22:21:59.670196 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:21:59.670109    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:22:00.084462 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:00.084429    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-cf1826-5f6ddbf4c7-7mrgs"
Oct 30 22:22:00.084632 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:00.084507    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-cf1826-556566884c-krgkh"
Oct 30 22:22:02.670352 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:02.670315    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:22:02.969734 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:02.967760    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:22:03.039691 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:03.037696    5510 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-7f76c4-6c578b69ff-hgkjj"]
Oct 30 22:22:03.058693 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:03.056972    5510 cpu_manager.go:414] "containerMap: removing container" podUID="2344709a-1a48-417c-ae98-2b462bf30516" containerName="istio-proxy"
Oct 30 22:22:03.058693 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:03.057008    5510 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2344709a-1a48-417c-ae98-2b462bf30516" containerName="istio-proxy"
Oct 30 22:22:03.058693 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:03.057059    5510 cpu_manager.go:414] "containerMap: removing container" podUID="e15eace0-bc5d-43ec-b352-74536b28a56c" containerName="istio-proxy"
Oct 30 22:22:03.058693 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:03.057067    5510 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e15eace0-bc5d-43ec-b352-74536b28a56c" containerName="istio-proxy"
Oct 30 22:22:03.058693 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:03.057090    5510 cpu_manager.go:414] "containerMap: removing container" podUID="caef4e8d-7dcc-455a-94a2-4bb341662a88" containerName="istio-proxy"
Oct 30 22:22:03.058693 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:03.057098    5510 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="caef4e8d-7dcc-455a-94a2-4bb341662a88" containerName="istio-proxy"
Oct 30 22:22:03.058693 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:03.057346    5510 memory_manager.go:365] "RemoveStaleState removing state" podUID="e15eace0-bc5d-43ec-b352-74536b28a56c" containerName="istio-proxy"
Oct 30 22:22:03.058693 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:03.057364    5510 memory_manager.go:365] "RemoveStaleState removing state" podUID="caef4e8d-7dcc-455a-94a2-4bb341662a88" containerName="istio-proxy"
Oct 30 22:22:03.058693 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:03.057385    5510 memory_manager.go:365] "RemoveStaleState removing state" podUID="2344709a-1a48-417c-ae98-2b462bf30516" containerName="istio-proxy"
Oct 30 22:22:03.098682 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:03.071343    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-7f76c4-6c578b69ff-hgkjj"
Oct 30 22:22:03.180680 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:03.179924    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-7f76c4-6c578b69ff-hgkjj"]
Oct 30 22:22:03.206689 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:03.200040    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-7f76c4-dockercfg-qh69t\""
Oct 30 22:22:03.263730 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:03.210846    5510 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-7f76c4\""
Oct 30 22:22:03.263730 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:03.260285    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2bbmj\" (UniqueName: \"kubernetes.io/projected/446aae1e-35a8-49da-9780-db1c369d7efd-kube-api-access-2bbmj\") pod \"payload-pre-processing-e2e-api-a-7f76c4-6c578b69ff-hgkjj\" (UID: \"446aae1e-35a8-49da-9780-db1c369d7efd\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-7f76c4-6c578b69ff-hgkjj"
Oct 30 22:22:03.263730 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:03.262383    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/446aae1e-35a8-49da-9780-db1c369d7efd-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-7f76c4-6c578b69ff-hgkjj\" (UID: \"446aae1e-35a8-49da-9780-db1c369d7efd\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-7f76c4-6c578b69ff-hgkjj"
Oct 30 22:22:03.368149 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:03.368107    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2bbmj\" (UniqueName: \"kubernetes.io/projected/446aae1e-35a8-49da-9780-db1c369d7efd-kube-api-access-2bbmj\") pod \"payload-pre-processing-e2e-api-a-7f76c4-6c578b69ff-hgkjj\" (UID: \"446aae1e-35a8-49da-9780-db1c369d7efd\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-7f76c4-6c578b69ff-hgkjj"
Oct 30 22:22:03.437191 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:03.434038    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/446aae1e-35a8-49da-9780-db1c369d7efd-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-7f76c4-6c578b69ff-hgkjj\" (UID: \"446aae1e-35a8-49da-9780-db1c369d7efd\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-7f76c4-6c578b69ff-hgkjj"
Oct 30 22:22:03.498696 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:03.486199    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/446aae1e-35a8-49da-9780-db1c369d7efd-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-7f76c4-6c578b69ff-hgkjj\" (UID: \"446aae1e-35a8-49da-9780-db1c369d7efd\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-7f76c4-6c578b69ff-hgkjj"
Oct 30 22:22:03.560685 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:03.551854    5510 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-7f76c4-56f886bdb9-w57ph"]
Oct 30 22:22:03.560685 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:03.555604    5510 cpu_manager.go:414] "containerMap: removing container" podUID="caef4e8d-7dcc-455a-94a2-4bb341662a88" containerName="istio-proxy"
Oct 30 22:22:03.560685 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:03.555627    5510 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="caef4e8d-7dcc-455a-94a2-4bb341662a88" containerName="istio-proxy"
Oct 30 22:22:03.560685 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:03.556069    5510 memory_manager.go:365] "RemoveStaleState removing state" podUID="caef4e8d-7dcc-455a-94a2-4bb341662a88" containerName="istio-proxy"
Oct 30 22:22:03.609972 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:03.566799    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-7f76c4-56f886bdb9-w57ph"
Oct 30 22:22:03.609972 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:03.600016    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-7f76c4-56f886bdb9-w57ph"]
Oct 30 22:22:03.664683 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:03.639410    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2bbmj\" (UniqueName: \"kubernetes.io/projected/446aae1e-35a8-49da-9780-db1c369d7efd-kube-api-access-2bbmj\") pod \"payload-pre-processing-e2e-api-a-7f76c4-6c578b69ff-hgkjj\" (UID: \"446aae1e-35a8-49da-9780-db1c369d7efd\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-7f76c4-6c578b69ff-hgkjj"
Oct 30 22:22:03.693263 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:03.693224    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7ffv2\" (UniqueName: \"kubernetes.io/projected/4e4794d9-9186-4127-b515-3e89b9882981-kube-api-access-7ffv2\") pod \"payload-processing-e2e-api-a-7f76c4-56f886bdb9-w57ph\" (UID: \"4e4794d9-9186-4127-b515-3e89b9882981\") " pod="openshift-ingress/payload-processing-e2e-api-a-7f76c4-56f886bdb9-w57ph"
Oct 30 22:22:03.693765 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:03.693382    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4e4794d9-9186-4127-b515-3e89b9882981-plugins-config-volume\") pod \"payload-processing-e2e-api-a-7f76c4-56f886bdb9-w57ph\" (UID: \"4e4794d9-9186-4127-b515-3e89b9882981\") " pod="openshift-ingress/payload-processing-e2e-api-a-7f76c4-56f886bdb9-w57ph"
Oct 30 22:22:03.736196 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:03.736171    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-7f76c4-6c578b69ff-hgkjj"
Oct 30 22:22:03.795365 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:03.795332    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7ffv2\" (UniqueName: \"kubernetes.io/projected/4e4794d9-9186-4127-b515-3e89b9882981-kube-api-access-7ffv2\") pod \"payload-processing-e2e-api-a-7f76c4-56f886bdb9-w57ph\" (UID: \"4e4794d9-9186-4127-b515-3e89b9882981\") " pod="openshift-ingress/payload-processing-e2e-api-a-7f76c4-56f886bdb9-w57ph"
Oct 30 22:22:03.795521 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:03.795417    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4e4794d9-9186-4127-b515-3e89b9882981-plugins-config-volume\") pod \"payload-processing-e2e-api-a-7f76c4-56f886bdb9-w57ph\" (UID: \"4e4794d9-9186-4127-b515-3e89b9882981\") " pod="openshift-ingress/payload-processing-e2e-api-a-7f76c4-56f886bdb9-w57ph"
Oct 30 22:22:03.796299 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:03.796275    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4e4794d9-9186-4127-b515-3e89b9882981-plugins-config-volume\") pod \"payload-processing-e2e-api-a-7f76c4-56f886bdb9-w57ph\" (UID: \"4e4794d9-9186-4127-b515-3e89b9882981\") " pod="openshift-ingress/payload-processing-e2e-api-a-7f76c4-56f886bdb9-w57ph"
Oct 30 22:22:03.872512 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:03.872471    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7ffv2\" (UniqueName: \"kubernetes.io/projected/4e4794d9-9186-4127-b515-3e89b9882981-kube-api-access-7ffv2\") pod \"payload-processing-e2e-api-a-7f76c4-56f886bdb9-w57ph\" (UID: \"4e4794d9-9186-4127-b515-3e89b9882981\") " pod="openshift-ingress/payload-processing-e2e-api-a-7f76c4-56f886bdb9-w57ph"
Oct 30 22:22:03.989140 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:03.989104    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-7f76c4-56f886bdb9-w57ph"
Oct 30 22:22:04.140075 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:04.140039    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-7f76c4-6c578b69ff-hgkjj"]
Oct 30 22:22:04.142641 ip-10-0-136-210 kubenswrapper[5510]: W1030 22:22:04.142615    5510 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod446aae1e_35a8_49da_9780_db1c369d7efd.slice/crio-83cd2953c39b177970955f9f6099d9269b744fb9f7235830db10832808db40d5 WatchSource:0}: Error finding container 83cd2953c39b177970955f9f6099d9269b744fb9f7235830db10832808db40d5: Status 404 returned error can't find the container with id 83cd2953c39b177970955f9f6099d9269b744fb9f7235830db10832808db40d5
Oct 30 22:22:04.385379 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:04.383842    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-7f76c4-56f886bdb9-w57ph"]
Oct 30 22:22:04.387455 ip-10-0-136-210 kubenswrapper[5510]: W1030 22:22:04.387419    5510 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e4794d9_9186_4127_b515_3e89b9882981.slice/crio-597b383926b6e10e0c22fca960984699268dcfefe6df38454718b82d18d1f1c6 WatchSource:0}: Error finding container 597b383926b6e10e0c22fca960984699268dcfefe6df38454718b82d18d1f1c6: Status 404 returned error can't find the container with id 597b383926b6e10e0c22fca960984699268dcfefe6df38454718b82d18d1f1c6
Oct 30 22:22:04.705954 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:04.705857    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-7f76c4-6c578b69ff-hgkjj" event={"ID":"446aae1e-35a8-49da-9780-db1c369d7efd","Type":"ContainerStarted","Data":"72a9726d0d1876846b3b24cf4236b315afcfee647cff03f2c45c6cd75632efed"}
Oct 30 22:22:04.705954 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:04.705920    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-7f76c4-6c578b69ff-hgkjj"
Oct 30 22:22:04.705954 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:04.705946    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-7f76c4-6c578b69ff-hgkjj" event={"ID":"446aae1e-35a8-49da-9780-db1c369d7efd","Type":"ContainerStarted","Data":"83cd2953c39b177970955f9f6099d9269b744fb9f7235830db10832808db40d5"}
Oct 30 22:22:04.707854 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:04.707827    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-7f76c4-56f886bdb9-w57ph" event={"ID":"4e4794d9-9186-4127-b515-3e89b9882981","Type":"ContainerStarted","Data":"c8393a23596fe9ffe4530603d9bceb1fbecb8ca5be75f66d2ce0d59ae5cd0d92"}
Oct 30 22:22:04.707974 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:04.707878    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-7f76c4-56f886bdb9-w57ph" event={"ID":"4e4794d9-9186-4127-b515-3e89b9882981","Type":"ContainerStarted","Data":"597b383926b6e10e0c22fca960984699268dcfefe6df38454718b82d18d1f1c6"}
Oct 30 22:22:04.708396 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:04.708370    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-7f76c4-56f886bdb9-w57ph"
Oct 30 22:22:04.814680 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:04.809692    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:22:04.844677 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:04.840295    5510 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-7f76c4-6c578b69ff-hgkjj" podStartSLOduration=2.840271302 podStartE2EDuration="2.840271302s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:22:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:22:04.817570726 +0000 UTC m=+1905.378380732" watchObservedRunningTime="2026-10-30 22:22:04.840271302 +0000 UTC m=+1905.401081307"
Oct 30 22:22:04.970736 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:04.964330    5510 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-7f76c4-56f886bdb9-w57ph" podStartSLOduration=1.964308749 podStartE2EDuration="1.964308749s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:22:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:22:04.927808149 +0000 UTC m=+1905.488618154" watchObservedRunningTime="2026-10-30 22:22:04.964308749 +0000 UTC m=+1905.525118755"
Oct 30 22:22:08.422453 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:22:08.422404    5510 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6gjbf" containerName="extractor"
Oct 30 22:22:09.418691 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:09.414867    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:22:09.749766 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:09.749683    5510 generic.go:356] "Generic (PLEG): container finished" podID="61fdf6bd-8922-4db6-a4cf-4fa9ea967b20" containerID="244da90203c70fbbaf03359fbaf0ae00b6517b196db93176f7390a74b61d45e6" exitCode=0
Oct 30 22:22:09.749766 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:09.749739    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w1-a6b6a5-openshift-default-f7cd49795-zlkqg" event={"ID":"61fdf6bd-8922-4db6-a4cf-4fa9ea967b20","Type":"ContainerDied","Data":"244da90203c70fbbaf03359fbaf0ae00b6517b196db93176f7390a74b61d45e6"}
Oct 30 22:22:09.750281 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:09.750199    5510 scope.go:122] "RemoveContainer" containerID="244da90203c70fbbaf03359fbaf0ae00b6517b196db93176f7390a74b61d45e6"
Oct 30 22:22:09.751931 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:09.751900    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:22:09.752025 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:09.751972    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:22:09.752025 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:09.752003    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:22:10.444698 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:10.444636    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:22:10.776373 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:10.776335    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w1-a6b6a5-openshift-default-f7cd49795-zlkqg" event={"ID":"61fdf6bd-8922-4db6-a4cf-4fa9ea967b20","Type":"ContainerStarted","Data":"33a8fb2c026ec7712d6106b877ef39350263d933eb1f22bb8130b0091637edc1"}
Oct 30 22:22:10.967682 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:10.967064    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w1-a6b6a5-openshift-default-f7cd49795-zlkqg"
Oct 30 22:22:10.969857 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:10.969816    5510 patch_prober.go:29] interesting pod/e2e-shared-a-w1-a6b6a5-openshift-default-f7cd49795-zlkqg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.111:15021/healthz/ready\": dial tcp 10.133.0.111:15021: connect: connection refused" start-of-body=
Oct 30 22:22:10.970146 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:10.970111    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w1-a6b6a5-openshift-default-f7cd49795-zlkqg" podUID="61fdf6bd-8922-4db6-a4cf-4fa9ea967b20" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.111:15021/healthz/ready\": dial tcp 10.133.0.111:15021: connect: connection refused"
Oct 30 22:22:11.964161 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:11.964118    5510 patch_prober.go:29] interesting pod/e2e-shared-a-w1-a6b6a5-openshift-default-f7cd49795-zlkqg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.111:15021/healthz/ready\": dial tcp 10.133.0.111:15021: connect: connection refused" start-of-body=
Oct 30 22:22:11.964826 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:11.964788    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w1-a6b6a5-openshift-default-f7cd49795-zlkqg" podUID="61fdf6bd-8922-4db6-a4cf-4fa9ea967b20" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.111:15021/healthz/ready\": dial tcp 10.133.0.111:15021: connect: connection refused"
Oct 30 22:22:12.969124 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:12.969077    5510 patch_prober.go:29] interesting pod/e2e-shared-a-w1-a6b6a5-openshift-default-f7cd49795-zlkqg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.111:15021/healthz/ready\": dial tcp 10.133.0.111:15021: connect: connection refused" start-of-body=
Oct 30 22:22:12.969568 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:12.969151    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w1-a6b6a5-openshift-default-f7cd49795-zlkqg" podUID="61fdf6bd-8922-4db6-a4cf-4fa9ea967b20" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.111:15021/healthz/ready\": dial tcp 10.133.0.111:15021: connect: connection refused"
Oct 30 22:22:13.968200 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:13.968155    5510 patch_prober.go:29] interesting pod/e2e-shared-a-w1-a6b6a5-openshift-default-f7cd49795-zlkqg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.111:15021/healthz/ready\": dial tcp 10.133.0.111:15021: connect: connection refused" start-of-body=
Oct 30 22:22:13.968512 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:13.968485    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w1-a6b6a5-openshift-default-f7cd49795-zlkqg" podUID="61fdf6bd-8922-4db6-a4cf-4fa9ea967b20" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.111:15021/healthz/ready\": dial tcp 10.133.0.111:15021: connect: connection refused"
Oct 30 22:22:14.463683 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:14.463010    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:22:14.962125 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:14.962081    5510 patch_prober.go:29] interesting pod/e2e-shared-a-w1-a6b6a5-openshift-default-f7cd49795-zlkqg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.111:15021/healthz/ready\": dial tcp 10.133.0.111:15021: connect: connection refused" start-of-body=
Oct 30 22:22:14.962433 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:14.962405    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w1-a6b6a5-openshift-default-f7cd49795-zlkqg" podUID="61fdf6bd-8922-4db6-a4cf-4fa9ea967b20" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.111:15021/healthz/ready\": dial tcp 10.133.0.111:15021: connect: connection refused"
Oct 30 22:22:15.719459 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:15.719426    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-7f76c4-6c578b69ff-hgkjj"
Oct 30 22:22:15.720262 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:15.720227    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-7f76c4-56f886bdb9-w57ph"
Oct 30 22:22:15.853298 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:15.853256    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w1-a6b6a5-openshift-default-f7cd49795-zlkqg"
Oct 30 22:22:15.867382 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:15.867349    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:22:15.986991 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:15.986888    5510 patch_prober.go:29] interesting pod/e2e-shared-a-w1-a6b6a5-openshift-default-f7cd49795-zlkqg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.111:15021/healthz/ready\": dial tcp 10.133.0.111:15021: connect: connection refused" start-of-body=
Oct 30 22:22:15.987270 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:15.987227    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w1-a6b6a5-openshift-default-f7cd49795-zlkqg" podUID="61fdf6bd-8922-4db6-a4cf-4fa9ea967b20" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.111:15021/healthz/ready\": dial tcp 10.133.0.111:15021: connect: connection refused"
Oct 30 22:22:16.311597 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:16.311560    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:22:17.043682 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:17.037989    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w1-a6b6a5-openshift-default-f7cd49795-zlkqg"
Oct 30 22:22:17.062678 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:17.054492    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w1-a6b6a5-openshift-default-f7cd49795-zlkqg"
Oct 30 22:22:17.300091 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:17.299998    5510 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w5-99304b-openshift-default-665c757b9-6t2xl"]
Oct 30 22:22:17.303857 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:17.303828    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w5-99304b-openshift-default-665c757b9-6t2xl"
Oct 30 22:22:17.354187 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:17.354153    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w5-99304b-openshift-default-dockercfg-fhf9d\""
Oct 30 22:22:17.360699 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:17.360649    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w5-99304b-openshift-default-665c757b9-6t2xl"]
Oct 30 22:22:17.398806 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:17.398749    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c4152cef-f9cf-46c1-a9c4-78484ab58efe-istio-envoy\") pod \"e2e-shared-b-w5-99304b-openshift-default-665c757b9-6t2xl\" (UID: \"c4152cef-f9cf-46c1-a9c4-78484ab58efe\") " pod="openshift-ingress/e2e-shared-b-w5-99304b-openshift-default-665c757b9-6t2xl"
Oct 30 22:22:17.398986 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:17.398911    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4152cef-f9cf-46c1-a9c4-78484ab58efe-workload-socket\") pod \"e2e-shared-b-w5-99304b-openshift-default-665c757b9-6t2xl\" (UID: \"c4152cef-f9cf-46c1-a9c4-78484ab58efe\") " pod="openshift-ingress/e2e-shared-b-w5-99304b-openshift-default-665c757b9-6t2xl"
Oct 30 22:22:17.398986 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:17.398943    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c4152cef-f9cf-46c1-a9c4-78484ab58efe-workload-certs\") pod \"e2e-shared-b-w5-99304b-openshift-default-665c757b9-6t2xl\" (UID: \"c4152cef-f9cf-46c1-a9c4-78484ab58efe\") " pod="openshift-ingress/e2e-shared-b-w5-99304b-openshift-default-665c757b9-6t2xl"
Oct 30 22:22:17.398986 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:17.398968    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-srwlx\" (UniqueName: \"kubernetes.io/projected/c4152cef-f9cf-46c1-a9c4-78484ab58efe-kube-api-access-srwlx\") pod \"e2e-shared-b-w5-99304b-openshift-default-665c757b9-6t2xl\" (UID: \"c4152cef-f9cf-46c1-a9c4-78484ab58efe\") " pod="openshift-ingress/e2e-shared-b-w5-99304b-openshift-default-665c757b9-6t2xl"
Oct 30 22:22:17.399134 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:17.399036    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c4152cef-f9cf-46c1-a9c4-78484ab58efe-istiod-ca-cert\") pod \"e2e-shared-b-w5-99304b-openshift-default-665c757b9-6t2xl\" (UID: \"c4152cef-f9cf-46c1-a9c4-78484ab58efe\") " pod="openshift-ingress/e2e-shared-b-w5-99304b-openshift-default-665c757b9-6t2xl"
Oct 30 22:22:17.399294 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:17.399263    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c4152cef-f9cf-46c1-a9c4-78484ab58efe-istio-data\") pod \"e2e-shared-b-w5-99304b-openshift-default-665c757b9-6t2xl\" (UID: \"c4152cef-f9cf-46c1-a9c4-78484ab58efe\") " pod="openshift-ingress/e2e-shared-b-w5-99304b-openshift-default-665c757b9-6t2xl"
Oct 30 22:22:17.399369 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:17.399345    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4152cef-f9cf-46c1-a9c4-78484ab58efe-credential-socket\") pod \"e2e-shared-b-w5-99304b-openshift-default-665c757b9-6t2xl\" (UID: \"c4152cef-f9cf-46c1-a9c4-78484ab58efe\") " pod="openshift-ingress/e2e-shared-b-w5-99304b-openshift-default-665c757b9-6t2xl"
Oct 30 22:22:17.399506 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:17.399489    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c4152cef-f9cf-46c1-a9c4-78484ab58efe-istio-token\") pod \"e2e-shared-b-w5-99304b-openshift-default-665c757b9-6t2xl\" (UID: \"c4152cef-f9cf-46c1-a9c4-78484ab58efe\") " pod="openshift-ingress/e2e-shared-b-w5-99304b-openshift-default-665c757b9-6t2xl"
Oct 30 22:22:17.399610 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:17.399548    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c4152cef-f9cf-46c1-a9c4-78484ab58efe-istio-podinfo\") pod \"e2e-shared-b-w5-99304b-openshift-default-665c757b9-6t2xl\" (UID: \"c4152cef-f9cf-46c1-a9c4-78484ab58efe\") " pod="openshift-ingress/e2e-shared-b-w5-99304b-openshift-default-665c757b9-6t2xl"
Oct 30 22:22:17.465313 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:17.464935    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:22:17.505972 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:17.502147    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c4152cef-f9cf-46c1-a9c4-78484ab58efe-istio-data\") pod \"e2e-shared-b-w5-99304b-openshift-default-665c757b9-6t2xl\" (UID: \"c4152cef-f9cf-46c1-a9c4-78484ab58efe\") " pod="openshift-ingress/e2e-shared-b-w5-99304b-openshift-default-665c757b9-6t2xl"
Oct 30 22:22:17.505972 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:17.504528    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4152cef-f9cf-46c1-a9c4-78484ab58efe-credential-socket\") pod \"e2e-shared-b-w5-99304b-openshift-default-665c757b9-6t2xl\" (UID: \"c4152cef-f9cf-46c1-a9c4-78484ab58efe\") " pod="openshift-ingress/e2e-shared-b-w5-99304b-openshift-default-665c757b9-6t2xl"
Oct 30 22:22:17.505972 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:17.504672    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c4152cef-f9cf-46c1-a9c4-78484ab58efe-istio-token\") pod \"e2e-shared-b-w5-99304b-openshift-default-665c757b9-6t2xl\" (UID: \"c4152cef-f9cf-46c1-a9c4-78484ab58efe\") " pod="openshift-ingress/e2e-shared-b-w5-99304b-openshift-default-665c757b9-6t2xl"
Oct 30 22:22:17.505972 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:17.504733    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c4152cef-f9cf-46c1-a9c4-78484ab58efe-istio-podinfo\") pod \"e2e-shared-b-w5-99304b-openshift-default-665c757b9-6t2xl\" (UID: \"c4152cef-f9cf-46c1-a9c4-78484ab58efe\") " pod="openshift-ingress/e2e-shared-b-w5-99304b-openshift-default-665c757b9-6t2xl"
Oct 30 22:22:17.505972 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:17.504860    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c4152cef-f9cf-46c1-a9c4-78484ab58efe-istio-envoy\") pod \"e2e-shared-b-w5-99304b-openshift-default-665c757b9-6t2xl\" (UID: \"c4152cef-f9cf-46c1-a9c4-78484ab58efe\") " pod="openshift-ingress/e2e-shared-b-w5-99304b-openshift-default-665c757b9-6t2xl"
Oct 30 22:22:17.505972 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:17.504960    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4152cef-f9cf-46c1-a9c4-78484ab58efe-workload-socket\") pod \"e2e-shared-b-w5-99304b-openshift-default-665c757b9-6t2xl\" (UID: \"c4152cef-f9cf-46c1-a9c4-78484ab58efe\") " pod="openshift-ingress/e2e-shared-b-w5-99304b-openshift-default-665c757b9-6t2xl"
Oct 30 22:22:17.505972 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:17.504990    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c4152cef-f9cf-46c1-a9c4-78484ab58efe-workload-certs\") pod \"e2e-shared-b-w5-99304b-openshift-default-665c757b9-6t2xl\" (UID: \"c4152cef-f9cf-46c1-a9c4-78484ab58efe\") " pod="openshift-ingress/e2e-shared-b-w5-99304b-openshift-default-665c757b9-6t2xl"
Oct 30 22:22:17.505972 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:17.505016    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-srwlx\" (UniqueName: \"kubernetes.io/projected/c4152cef-f9cf-46c1-a9c4-78484ab58efe-kube-api-access-srwlx\") pod \"e2e-shared-b-w5-99304b-openshift-default-665c757b9-6t2xl\" (UID: \"c4152cef-f9cf-46c1-a9c4-78484ab58efe\") " pod="openshift-ingress/e2e-shared-b-w5-99304b-openshift-default-665c757b9-6t2xl"
Oct 30 22:22:17.505972 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:17.505043    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c4152cef-f9cf-46c1-a9c4-78484ab58efe-istiod-ca-cert\") pod \"e2e-shared-b-w5-99304b-openshift-default-665c757b9-6t2xl\" (UID: \"c4152cef-f9cf-46c1-a9c4-78484ab58efe\") " pod="openshift-ingress/e2e-shared-b-w5-99304b-openshift-default-665c757b9-6t2xl"
Oct 30 22:22:17.505972 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:17.505185    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4152cef-f9cf-46c1-a9c4-78484ab58efe-credential-socket\") pod \"e2e-shared-b-w5-99304b-openshift-default-665c757b9-6t2xl\" (UID: \"c4152cef-f9cf-46c1-a9c4-78484ab58efe\") " pod="openshift-ingress/e2e-shared-b-w5-99304b-openshift-default-665c757b9-6t2xl"
Oct 30 22:22:17.506594 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:17.506150    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c4152cef-f9cf-46c1-a9c4-78484ab58efe-istiod-ca-cert\") pod \"e2e-shared-b-w5-99304b-openshift-default-665c757b9-6t2xl\" (UID: \"c4152cef-f9cf-46c1-a9c4-78484ab58efe\") " pod="openshift-ingress/e2e-shared-b-w5-99304b-openshift-default-665c757b9-6t2xl"
Oct 30 22:22:17.506594 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:17.506406    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c4152cef-f9cf-46c1-a9c4-78484ab58efe-istio-data\") pod \"e2e-shared-b-w5-99304b-openshift-default-665c757b9-6t2xl\" (UID: \"c4152cef-f9cf-46c1-a9c4-78484ab58efe\") " pod="openshift-ingress/e2e-shared-b-w5-99304b-openshift-default-665c757b9-6t2xl"
Oct 30 22:22:17.506713 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:17.506639    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4152cef-f9cf-46c1-a9c4-78484ab58efe-workload-socket\") pod \"e2e-shared-b-w5-99304b-openshift-default-665c757b9-6t2xl\" (UID: \"c4152cef-f9cf-46c1-a9c4-78484ab58efe\") " pod="openshift-ingress/e2e-shared-b-w5-99304b-openshift-default-665c757b9-6t2xl"
Oct 30 22:22:17.508470 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:17.508447    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c4152cef-f9cf-46c1-a9c4-78484ab58efe-workload-certs\") pod \"e2e-shared-b-w5-99304b-openshift-default-665c757b9-6t2xl\" (UID: \"c4152cef-f9cf-46c1-a9c4-78484ab58efe\") " pod="openshift-ingress/e2e-shared-b-w5-99304b-openshift-default-665c757b9-6t2xl"
Oct 30 22:22:17.565738 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:17.565603    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c4152cef-f9cf-46c1-a9c4-78484ab58efe-istio-podinfo\") pod \"e2e-shared-b-w5-99304b-openshift-default-665c757b9-6t2xl\" (UID: \"c4152cef-f9cf-46c1-a9c4-78484ab58efe\") " pod="openshift-ingress/e2e-shared-b-w5-99304b-openshift-default-665c757b9-6t2xl"
Oct 30 22:22:17.635281 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:17.635161    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c4152cef-f9cf-46c1-a9c4-78484ab58efe-istio-envoy\") pod \"e2e-shared-b-w5-99304b-openshift-default-665c757b9-6t2xl\" (UID: \"c4152cef-f9cf-46c1-a9c4-78484ab58efe\") " pod="openshift-ingress/e2e-shared-b-w5-99304b-openshift-default-665c757b9-6t2xl"
Oct 30 22:22:17.736460 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:17.733433    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-srwlx\" (UniqueName: \"kubernetes.io/projected/c4152cef-f9cf-46c1-a9c4-78484ab58efe-kube-api-access-srwlx\") pod \"e2e-shared-b-w5-99304b-openshift-default-665c757b9-6t2xl\" (UID: \"c4152cef-f9cf-46c1-a9c4-78484ab58efe\") " pod="openshift-ingress/e2e-shared-b-w5-99304b-openshift-default-665c757b9-6t2xl"
Oct 30 22:22:17.738258 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:17.738190    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:22:17.743262 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:17.742979    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c4152cef-f9cf-46c1-a9c4-78484ab58efe-istio-token\") pod \"e2e-shared-b-w5-99304b-openshift-default-665c757b9-6t2xl\" (UID: \"c4152cef-f9cf-46c1-a9c4-78484ab58efe\") " pod="openshift-ingress/e2e-shared-b-w5-99304b-openshift-default-665c757b9-6t2xl"
Oct 30 22:22:17.919403 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:17.919315    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w5-99304b-openshift-default-665c757b9-6t2xl"
Oct 30 22:22:18.014208 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:18.014169    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:22:18.346285 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:18.346243    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w5-99304b-openshift-default-665c757b9-6t2xl"]
Oct 30 22:22:18.346702 ip-10-0-136-210 kubenswrapper[5510]: W1030 22:22:18.346528    5510 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4152cef_f9cf_46c1_a9c4_78484ab58efe.slice/crio-231c680116133f8e0e387f7e7569c5429921f7f2837a0ec60966df23af2f6e69 WatchSource:0}: Error finding container 231c680116133f8e0e387f7e7569c5429921f7f2837a0ec60966df23af2f6e69: Status 404 returned error can't find the container with id 231c680116133f8e0e387f7e7569c5429921f7f2837a0ec60966df23af2f6e69
Oct 30 22:22:18.349164 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:18.349128    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:22:18.349266 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:18.349195    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:22:18.349266 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:18.349226    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:22:18.835160 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:18.835121    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w5-99304b-openshift-default-665c757b9-6t2xl" event={"ID":"c4152cef-f9cf-46c1-a9c4-78484ab58efe","Type":"ContainerStarted","Data":"e97bc2ef207378d6ceb74ad4321a4a3e7a692e93e16ed58c50e1d900ac569c18"}
Oct 30 22:22:18.835365 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:18.835181    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w5-99304b-openshift-default-665c757b9-6t2xl" event={"ID":"c4152cef-f9cf-46c1-a9c4-78484ab58efe","Type":"ContainerStarted","Data":"231c680116133f8e0e387f7e7569c5429921f7f2837a0ec60966df23af2f6e69"}
Oct 30 22:22:18.919897 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:18.919864    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w5-99304b-openshift-default-665c757b9-6t2xl"
Oct 30 22:22:18.951648 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:18.951584    5510 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w5-99304b-openshift-default-665c757b9-6t2xl" podStartSLOduration=1.951565148 podStartE2EDuration="1.951565148s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:22:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:22:18.949647787 +0000 UTC m=+1919.510457812" watchObservedRunningTime="2026-10-30 22:22:18.951565148 +0000 UTC m=+1919.512375155"
Oct 30 22:22:19.529774 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:19.526336    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:22:19.926118 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:19.926037    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w5-99304b-openshift-default-665c757b9-6t2xl"
Oct 30 22:22:20.849013 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:20.848979    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w5-99304b-openshift-default-665c757b9-6t2xl"
Oct 30 22:22:20.850147 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:20.850115    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w5-99304b-openshift-default-665c757b9-6t2xl"
Oct 30 22:22:21.898344 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:21.898309    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:22:22.185687 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:22.185556    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:22:24.631012 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:24.630902    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:22:25.343238 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:25.305313    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:22:25.429049 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:25.428997    5510 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w5-99304b-7dc47b455-f2xqz"]
Oct 30 22:22:25.463728 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:25.438718    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w5-99304b-7dc47b455-f2xqz"
Oct 30 22:22:25.518972 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:25.512534    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:22:25.529902 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:25.520107    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w5-99304b-dockercfg-cb2zp\""
Oct 30 22:22:25.530189 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:25.530149    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w5-99304b-7dc47b455-f2xqz"]
Oct 30 22:22:25.587483 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:25.587447    5510 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w5-99304b\""
Oct 30 22:22:25.599672 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:25.599572    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-svttq\" (UniqueName: \"kubernetes.io/projected/9151b5e0-534d-4f10-8c6e-6a0f0a36a942-kube-api-access-svttq\") pod \"payload-pre-processing-e2e-shared-b-w5-99304b-7dc47b455-f2xqz\" (UID: \"9151b5e0-534d-4f10-8c6e-6a0f0a36a942\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w5-99304b-7dc47b455-f2xqz"
Oct 30 22:22:25.600186 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:25.600160    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9151b5e0-534d-4f10-8c6e-6a0f0a36a942-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w5-99304b-7dc47b455-f2xqz\" (UID: \"9151b5e0-534d-4f10-8c6e-6a0f0a36a942\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w5-99304b-7dc47b455-f2xqz"
Oct 30 22:22:25.754221 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:25.753267    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-svttq\" (UniqueName: \"kubernetes.io/projected/9151b5e0-534d-4f10-8c6e-6a0f0a36a942-kube-api-access-svttq\") pod \"payload-pre-processing-e2e-shared-b-w5-99304b-7dc47b455-f2xqz\" (UID: \"9151b5e0-534d-4f10-8c6e-6a0f0a36a942\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w5-99304b-7dc47b455-f2xqz"
Oct 30 22:22:25.754221 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:25.753318    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9151b5e0-534d-4f10-8c6e-6a0f0a36a942-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w5-99304b-7dc47b455-f2xqz\" (UID: \"9151b5e0-534d-4f10-8c6e-6a0f0a36a942\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w5-99304b-7dc47b455-f2xqz"
Oct 30 22:22:25.788689 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:25.784444    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9151b5e0-534d-4f10-8c6e-6a0f0a36a942-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w5-99304b-7dc47b455-f2xqz\" (UID: \"9151b5e0-534d-4f10-8c6e-6a0f0a36a942\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w5-99304b-7dc47b455-f2xqz"
Oct 30 22:22:25.879148 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:25.879080    5510 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w5-99304b-6c6775f47f-bd9fp"]
Oct 30 22:22:25.886997 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:25.886963    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w5-99304b-6c6775f47f-bd9fp"
Oct 30 22:22:25.908304 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:25.888070    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w5-99304b-6c6775f47f-bd9fp"]
Oct 30 22:22:25.908700 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:25.889642    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-svttq\" (UniqueName: \"kubernetes.io/projected/9151b5e0-534d-4f10-8c6e-6a0f0a36a942-kube-api-access-svttq\") pod \"payload-pre-processing-e2e-shared-b-w5-99304b-7dc47b455-f2xqz\" (UID: \"9151b5e0-534d-4f10-8c6e-6a0f0a36a942\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w5-99304b-7dc47b455-f2xqz"
Oct 30 22:22:25.995932 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:25.995886    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1ddea0c3-4738-4353-a0df-726c86a635be-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w5-99304b-6c6775f47f-bd9fp\" (UID: \"1ddea0c3-4738-4353-a0df-726c86a635be\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w5-99304b-6c6775f47f-bd9fp"
Oct 30 22:22:25.996290 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:25.996270    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lzmwn\" (UniqueName: \"kubernetes.io/projected/1ddea0c3-4738-4353-a0df-726c86a635be-kube-api-access-lzmwn\") pod \"payload-processing-e2e-shared-b-w5-99304b-6c6775f47f-bd9fp\" (UID: \"1ddea0c3-4738-4353-a0df-726c86a635be\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w5-99304b-6c6775f47f-bd9fp"
Oct 30 22:22:26.097748 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:26.097715    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1ddea0c3-4738-4353-a0df-726c86a635be-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w5-99304b-6c6775f47f-bd9fp\" (UID: \"1ddea0c3-4738-4353-a0df-726c86a635be\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w5-99304b-6c6775f47f-bd9fp"
Oct 30 22:22:26.097917 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:26.097837    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lzmwn\" (UniqueName: \"kubernetes.io/projected/1ddea0c3-4738-4353-a0df-726c86a635be-kube-api-access-lzmwn\") pod \"payload-processing-e2e-shared-b-w5-99304b-6c6775f47f-bd9fp\" (UID: \"1ddea0c3-4738-4353-a0df-726c86a635be\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w5-99304b-6c6775f47f-bd9fp"
Oct 30 22:22:26.098402 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:26.098381    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1ddea0c3-4738-4353-a0df-726c86a635be-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w5-99304b-6c6775f47f-bd9fp\" (UID: \"1ddea0c3-4738-4353-a0df-726c86a635be\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w5-99304b-6c6775f47f-bd9fp"
Oct 30 22:22:26.157136 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:26.157067    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lzmwn\" (UniqueName: \"kubernetes.io/projected/1ddea0c3-4738-4353-a0df-726c86a635be-kube-api-access-lzmwn\") pod \"payload-processing-e2e-shared-b-w5-99304b-6c6775f47f-bd9fp\" (UID: \"1ddea0c3-4738-4353-a0df-726c86a635be\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w5-99304b-6c6775f47f-bd9fp"
Oct 30 22:22:26.192179 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:26.192149    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w5-99304b-7dc47b455-f2xqz"
Oct 30 22:22:26.227750 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:26.227721    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w5-99304b-6c6775f47f-bd9fp"
Oct 30 22:22:26.588822 ip-10-0-136-210 kubenswrapper[5510]: W1030 22:22:26.588788    5510 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9151b5e0_534d_4f10_8c6e_6a0f0a36a942.slice/crio-a1960d31da11f4285a446406a508f28034361614cd2664950cb9c18df29394e3 WatchSource:0}: Error finding container a1960d31da11f4285a446406a508f28034361614cd2664950cb9c18df29394e3: Status 404 returned error can't find the container with id a1960d31da11f4285a446406a508f28034361614cd2664950cb9c18df29394e3
Oct 30 22:22:26.588954 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:26.588845    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w5-99304b-7dc47b455-f2xqz"]
Oct 30 22:22:26.637704 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:26.637678    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w5-99304b-6c6775f47f-bd9fp"]
Oct 30 22:22:26.640231 ip-10-0-136-210 kubenswrapper[5510]: W1030 22:22:26.640203    5510 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ddea0c3_4738_4353_a0df_726c86a635be.slice/crio-72b3a05304c1094348366ce2368de8a149333f30525bf02e50ce7fad035490fb WatchSource:0}: Error finding container 72b3a05304c1094348366ce2368de8a149333f30525bf02e50ce7fad035490fb: Status 404 returned error can't find the container with id 72b3a05304c1094348366ce2368de8a149333f30525bf02e50ce7fad035490fb
Oct 30 22:22:26.903789 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:26.903688    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w5-99304b-7dc47b455-f2xqz" event={"ID":"9151b5e0-534d-4f10-8c6e-6a0f0a36a942","Type":"ContainerStarted","Data":"cd9742ea1b0ad2686ab3222f6d7164fa2af51430b113e3d311a92aa8dd3b3703"}
Oct 30 22:22:26.903789 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:26.903743    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w5-99304b-7dc47b455-f2xqz" event={"ID":"9151b5e0-534d-4f10-8c6e-6a0f0a36a942","Type":"ContainerStarted","Data":"a1960d31da11f4285a446406a508f28034361614cd2664950cb9c18df29394e3"}
Oct 30 22:22:26.904273 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:26.903931    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w5-99304b-7dc47b455-f2xqz"
Oct 30 22:22:26.905517 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:26.905491    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w5-99304b-6c6775f47f-bd9fp" event={"ID":"1ddea0c3-4738-4353-a0df-726c86a635be","Type":"ContainerStarted","Data":"e96ec942921f6989eeaf6b31328c91ddabd21c8aa58b21c623a9c003b7b9be2a"}
Oct 30 22:22:26.905647 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:26.905536    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w5-99304b-6c6775f47f-bd9fp" event={"ID":"1ddea0c3-4738-4353-a0df-726c86a635be","Type":"ContainerStarted","Data":"72b3a05304c1094348366ce2368de8a149333f30525bf02e50ce7fad035490fb"}
Oct 30 22:22:26.905647 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:26.905560    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w5-99304b-6c6775f47f-bd9fp"
Oct 30 22:22:27.013751 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:27.013691    5510 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w5-99304b-7dc47b455-f2xqz" podStartSLOduration=2.013672157 podStartE2EDuration="2.013672157s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:22:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:22:27.013453334 +0000 UTC m=+1927.574263341" watchObservedRunningTime="2026-10-30 22:22:27.013672157 +0000 UTC m=+1927.574482163"
Oct 30 22:22:27.180735 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:27.179751    5510 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w5-99304b-6c6775f47f-bd9fp" podStartSLOduration=2.179730399 podStartE2EDuration="2.179730399s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:22:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:22:27.153204913 +0000 UTC m=+1927.714014924" watchObservedRunningTime="2026-10-30 22:22:27.179730399 +0000 UTC m=+1927.740540417"
Oct 30 22:22:29.246110 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:29.246072    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:22:29.495736 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:29.495686    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:22:29.687063 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:29.687018    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:22:31.012317 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:31.012279    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:22:31.205339 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:31.205298    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:22:32.606637 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:32.606604    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:22:32.840845 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:32.840811    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:22:34.105224 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:34.105185    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:22:35.847683 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:35.844419    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-g64nz"]
Oct 30 22:22:35.859681 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:35.857167    5510 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-g64nz" podUID="71b34137-bc68-4228-a7e0-30fda97183aa" containerName="istio-proxy" containerID="cri-o://39d70433e536b73933ad2fb97c7e017a6f0de14f8c2b04088965c5afeaa31ebb" gracePeriod=30
Oct 30 22:22:35.935828 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:35.935792    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-86dps"]
Oct 30 22:22:35.936093 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:35.936053    5510 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-86dps" podUID="10d681c2-b6b1-40a1-8c97-1988a0e91d05" containerName="istio-proxy" containerID="cri-o://f791c34bc62d549080d2963539c7ef6fffcf7a4cd452bd82810ba5d43aae9b08" gracePeriod=30
Oct 30 22:22:37.916426 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:37.916396    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w5-99304b-6c6775f47f-bd9fp"
Oct 30 22:22:37.916920 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:37.916688    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w5-99304b-7dc47b455-f2xqz"
Oct 30 22:22:37.966645 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:37.966605    5510 patch_prober.go:29] interesting pod/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-g64nz container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 22:22:37.966816 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:37.966703    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-g64nz" podUID="71b34137-bc68-4228-a7e0-30fda97183aa" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 22:22:38.421724 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:22:38.421650    5510 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6gjbf" containerName="extractor"
Oct 30 22:22:38.834562 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:38.833137    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:22:41.122931 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:41.122518    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-g64nz"
Oct 30 22:22:41.185975 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:41.185888    5510 generic.go:356] "Generic (PLEG): container finished" podID="10d681c2-b6b1-40a1-8c97-1988a0e91d05" containerID="f791c34bc62d549080d2963539c7ef6fffcf7a4cd452bd82810ba5d43aae9b08" exitCode=0
Oct 30 22:22:41.185975 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:41.185965    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-86dps" event={"ID":"10d681c2-b6b1-40a1-8c97-1988a0e91d05","Type":"ContainerDied","Data":"f791c34bc62d549080d2963539c7ef6fffcf7a4cd452bd82810ba5d43aae9b08"}
Oct 30 22:22:41.187699 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:41.187673    5510 generic.go:356] "Generic (PLEG): container finished" podID="71b34137-bc68-4228-a7e0-30fda97183aa" containerID="39d70433e536b73933ad2fb97c7e017a6f0de14f8c2b04088965c5afeaa31ebb" exitCode=0
Oct 30 22:22:41.187858 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:41.187711    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-g64nz" event={"ID":"71b34137-bc68-4228-a7e0-30fda97183aa","Type":"ContainerDied","Data":"39d70433e536b73933ad2fb97c7e017a6f0de14f8c2b04088965c5afeaa31ebb"}
Oct 30 22:22:41.187906 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:41.187870    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-g64nz" event={"ID":"71b34137-bc68-4228-a7e0-30fda97183aa","Type":"ContainerDied","Data":"ec05d0c6797ccbf7e66f71ace3ab0e03d94d51d6b94c385e599fec6096663472"}
Oct 30 22:22:41.187906 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:41.187896    5510 scope.go:122] "RemoveContainer" containerID="39d70433e536b73933ad2fb97c7e017a6f0de14f8c2b04088965c5afeaa31ebb"
Oct 30 22:22:41.187995 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:41.187744    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-g64nz"
Oct 30 22:22:41.202150 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:41.202126    5510 scope.go:122] "RemoveContainer" containerID="39d70433e536b73933ad2fb97c7e017a6f0de14f8c2b04088965c5afeaa31ebb"
Oct 30 22:22:41.202492 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:22:41.202468    5510 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"39d70433e536b73933ad2fb97c7e017a6f0de14f8c2b04088965c5afeaa31ebb\": container with ID starting with 39d70433e536b73933ad2fb97c7e017a6f0de14f8c2b04088965c5afeaa31ebb not found: ID does not exist" containerID="39d70433e536b73933ad2fb97c7e017a6f0de14f8c2b04088965c5afeaa31ebb"
Oct 30 22:22:41.202627 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:41.202499    5510 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"39d70433e536b73933ad2fb97c7e017a6f0de14f8c2b04088965c5afeaa31ebb"} err="failed to get container status \"39d70433e536b73933ad2fb97c7e017a6f0de14f8c2b04088965c5afeaa31ebb\": rpc error: code = NotFound desc = could not find container \"39d70433e536b73933ad2fb97c7e017a6f0de14f8c2b04088965c5afeaa31ebb\": container with ID starting with 39d70433e536b73933ad2fb97c7e017a6f0de14f8c2b04088965c5afeaa31ebb not found: ID does not exist"
Oct 30 22:22:41.235903 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:41.235876    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-86dps"
Oct 30 22:22:41.237849 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:41.237821    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/71b34137-bc68-4228-a7e0-30fda97183aa-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/71b34137-bc68-4228-a7e0-30fda97183aa-istio-data\") pod \"71b34137-bc68-4228-a7e0-30fda97183aa\" (UID: \"71b34137-bc68-4228-a7e0-30fda97183aa\") "
Oct 30 22:22:41.237973 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:41.237892    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/71b34137-bc68-4228-a7e0-30fda97183aa-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/71b34137-bc68-4228-a7e0-30fda97183aa-workload-certs\") pod \"71b34137-bc68-4228-a7e0-30fda97183aa\" (UID: \"71b34137-bc68-4228-a7e0-30fda97183aa\") "
Oct 30 22:22:41.238070 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:41.238041    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/71b34137-bc68-4228-a7e0-30fda97183aa-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/71b34137-bc68-4228-a7e0-30fda97183aa-istio-envoy\") pod \"71b34137-bc68-4228-a7e0-30fda97183aa\" (UID: \"71b34137-bc68-4228-a7e0-30fda97183aa\") "
Oct 30 22:22:41.238136 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:41.238113    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71b34137-bc68-4228-a7e0-30fda97183aa-istio-data" pod "71b34137-bc68-4228-a7e0-30fda97183aa" (UID: "71b34137-bc68-4228-a7e0-30fda97183aa"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:22:41.238188 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:41.238157    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71b34137-bc68-4228-a7e0-30fda97183aa-workload-certs" pod "71b34137-bc68-4228-a7e0-30fda97183aa" (UID: "71b34137-bc68-4228-a7e0-30fda97183aa"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:22:41.238238 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:41.238207    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/71b34137-bc68-4228-a7e0-30fda97183aa-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/71b34137-bc68-4228-a7e0-30fda97183aa-credential-socket\") pod \"71b34137-bc68-4228-a7e0-30fda97183aa\" (UID: \"71b34137-bc68-4228-a7e0-30fda97183aa\") "
Oct 30 22:22:41.238292 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:41.238271    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/71b34137-bc68-4228-a7e0-30fda97183aa-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/71b34137-bc68-4228-a7e0-30fda97183aa-istiod-ca-cert\") pod \"71b34137-bc68-4228-a7e0-30fda97183aa\" (UID: \"71b34137-bc68-4228-a7e0-30fda97183aa\") "
Oct 30 22:22:41.238348 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:41.238318    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/71b34137-bc68-4228-a7e0-30fda97183aa-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/71b34137-bc68-4228-a7e0-30fda97183aa-workload-socket\") pod \"71b34137-bc68-4228-a7e0-30fda97183aa\" (UID: \"71b34137-bc68-4228-a7e0-30fda97183aa\") "
Oct 30 22:22:41.238406 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:41.238348    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/71b34137-bc68-4228-a7e0-30fda97183aa-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/71b34137-bc68-4228-a7e0-30fda97183aa-istio-podinfo\") pod \"71b34137-bc68-4228-a7e0-30fda97183aa\" (UID: \"71b34137-bc68-4228-a7e0-30fda97183aa\") "
Oct 30 22:22:41.238406 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:41.238384    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/71b34137-bc68-4228-a7e0-30fda97183aa-istio-token\" (UniqueName: \"kubernetes.io/projected/71b34137-bc68-4228-a7e0-30fda97183aa-istio-token\") pod \"71b34137-bc68-4228-a7e0-30fda97183aa\" (UID: \"71b34137-bc68-4228-a7e0-30fda97183aa\") "
Oct 30 22:22:41.238502 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:41.238409    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71b34137-bc68-4228-a7e0-30fda97183aa-credential-socket" pod "71b34137-bc68-4228-a7e0-30fda97183aa" (UID: "71b34137-bc68-4228-a7e0-30fda97183aa"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:22:41.238502 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:41.238427    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/71b34137-bc68-4228-a7e0-30fda97183aa-kube-api-access-mmbqj\" (UniqueName: \"kubernetes.io/projected/71b34137-bc68-4228-a7e0-30fda97183aa-kube-api-access-mmbqj\") pod \"71b34137-bc68-4228-a7e0-30fda97183aa\" (UID: \"71b34137-bc68-4228-a7e0-30fda97183aa\") "
Oct 30 22:22:41.238648 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:41.238624    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71b34137-bc68-4228-a7e0-30fda97183aa-workload-socket" pod "71b34137-bc68-4228-a7e0-30fda97183aa" (UID: "71b34137-bc68-4228-a7e0-30fda97183aa"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:22:41.238739 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:41.238713    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/71b34137-bc68-4228-a7e0-30fda97183aa-istiod-ca-cert" pod "71b34137-bc68-4228-a7e0-30fda97183aa" (UID: "71b34137-bc68-4228-a7e0-30fda97183aa"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:22:41.239275 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:41.239249    5510 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/71b34137-bc68-4228-a7e0-30fda97183aa-workload-socket\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:22:41.239399 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:41.239278    5510 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/71b34137-bc68-4228-a7e0-30fda97183aa-istio-data\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:22:41.239399 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:41.239292    5510 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/71b34137-bc68-4228-a7e0-30fda97183aa-workload-certs\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:22:41.239399 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:41.239307    5510 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/71b34137-bc68-4228-a7e0-30fda97183aa-credential-socket\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:22:41.239399 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:41.239321    5510 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/71b34137-bc68-4228-a7e0-30fda97183aa-istiod-ca-cert\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:22:41.240358 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:41.240336    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71b34137-bc68-4228-a7e0-30fda97183aa-istio-envoy" pod "71b34137-bc68-4228-a7e0-30fda97183aa" (UID: "71b34137-bc68-4228-a7e0-30fda97183aa"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:22:41.240578 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:41.240556    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/71b34137-bc68-4228-a7e0-30fda97183aa-istio-token" pod "71b34137-bc68-4228-a7e0-30fda97183aa" (UID: "71b34137-bc68-4228-a7e0-30fda97183aa"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:22:41.240724 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:41.240706    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/71b34137-bc68-4228-a7e0-30fda97183aa-kube-api-access-mmbqj" pod "71b34137-bc68-4228-a7e0-30fda97183aa" (UID: "71b34137-bc68-4228-a7e0-30fda97183aa"). InnerVolumeSpecName "kube-api-access-mmbqj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:22:41.241217 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:41.241197    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/71b34137-bc68-4228-a7e0-30fda97183aa-istio-podinfo" pod "71b34137-bc68-4228-a7e0-30fda97183aa" (UID: "71b34137-bc68-4228-a7e0-30fda97183aa"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:22:41.343216 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:41.343181    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/10d681c2-b6b1-40a1-8c97-1988a0e91d05-kube-api-access-zl46q\" (UniqueName: \"kubernetes.io/projected/10d681c2-b6b1-40a1-8c97-1988a0e91d05-kube-api-access-zl46q\") pod \"10d681c2-b6b1-40a1-8c97-1988a0e91d05\" (UID: \"10d681c2-b6b1-40a1-8c97-1988a0e91d05\") "
Oct 30 22:22:41.343431 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:41.343331    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/10d681c2-b6b1-40a1-8c97-1988a0e91d05-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/10d681c2-b6b1-40a1-8c97-1988a0e91d05-istio-envoy\") pod \"10d681c2-b6b1-40a1-8c97-1988a0e91d05\" (UID: \"10d681c2-b6b1-40a1-8c97-1988a0e91d05\") "
Oct 30 22:22:41.343506 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:41.343427    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/10d681c2-b6b1-40a1-8c97-1988a0e91d05-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/10d681c2-b6b1-40a1-8c97-1988a0e91d05-istio-podinfo\") pod \"10d681c2-b6b1-40a1-8c97-1988a0e91d05\" (UID: \"10d681c2-b6b1-40a1-8c97-1988a0e91d05\") "
Oct 30 22:22:41.343566 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:41.343516    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/10d681c2-b6b1-40a1-8c97-1988a0e91d05-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/10d681c2-b6b1-40a1-8c97-1988a0e91d05-workload-socket\") pod \"10d681c2-b6b1-40a1-8c97-1988a0e91d05\" (UID: \"10d681c2-b6b1-40a1-8c97-1988a0e91d05\") "
Oct 30 22:22:41.343628 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:41.343591    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/10d681c2-b6b1-40a1-8c97-1988a0e91d05-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/10d681c2-b6b1-40a1-8c97-1988a0e91d05-istiod-ca-cert\") pod \"10d681c2-b6b1-40a1-8c97-1988a0e91d05\" (UID: \"10d681c2-b6b1-40a1-8c97-1988a0e91d05\") "
Oct 30 22:22:41.343723 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:41.343642    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/10d681c2-b6b1-40a1-8c97-1988a0e91d05-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/10d681c2-b6b1-40a1-8c97-1988a0e91d05-credential-socket\") pod \"10d681c2-b6b1-40a1-8c97-1988a0e91d05\" (UID: \"10d681c2-b6b1-40a1-8c97-1988a0e91d05\") "
Oct 30 22:22:41.343805 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:41.343739    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/10d681c2-b6b1-40a1-8c97-1988a0e91d05-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/10d681c2-b6b1-40a1-8c97-1988a0e91d05-istio-data\") pod \"10d681c2-b6b1-40a1-8c97-1988a0e91d05\" (UID: \"10d681c2-b6b1-40a1-8c97-1988a0e91d05\") "
Oct 30 22:22:41.343805 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:41.343771    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/10d681c2-b6b1-40a1-8c97-1988a0e91d05-istio-token\" (UniqueName: \"kubernetes.io/projected/10d681c2-b6b1-40a1-8c97-1988a0e91d05-istio-token\") pod \"10d681c2-b6b1-40a1-8c97-1988a0e91d05\" (UID: \"10d681c2-b6b1-40a1-8c97-1988a0e91d05\") "
Oct 30 22:22:41.343908 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:41.343823    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/10d681c2-b6b1-40a1-8c97-1988a0e91d05-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/10d681c2-b6b1-40a1-8c97-1988a0e91d05-workload-certs\") pod \"10d681c2-b6b1-40a1-8c97-1988a0e91d05\" (UID: \"10d681c2-b6b1-40a1-8c97-1988a0e91d05\") "
Oct 30 22:22:41.344876 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:41.344317    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/10d681c2-b6b1-40a1-8c97-1988a0e91d05-istiod-ca-cert" pod "10d681c2-b6b1-40a1-8c97-1988a0e91d05" (UID: "10d681c2-b6b1-40a1-8c97-1988a0e91d05"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:22:41.344876 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:41.344557    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/10d681c2-b6b1-40a1-8c97-1988a0e91d05-workload-certs" pod "10d681c2-b6b1-40a1-8c97-1988a0e91d05" (UID: "10d681c2-b6b1-40a1-8c97-1988a0e91d05"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:22:41.344876 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:41.344788    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/10d681c2-b6b1-40a1-8c97-1988a0e91d05-credential-socket" pod "10d681c2-b6b1-40a1-8c97-1988a0e91d05" (UID: "10d681c2-b6b1-40a1-8c97-1988a0e91d05"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:22:41.345108 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:41.344967    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/10d681c2-b6b1-40a1-8c97-1988a0e91d05-istio-data" pod "10d681c2-b6b1-40a1-8c97-1988a0e91d05" (UID: "10d681c2-b6b1-40a1-8c97-1988a0e91d05"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:22:41.346595 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:41.346567    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/10d681c2-b6b1-40a1-8c97-1988a0e91d05-workload-socket" pod "10d681c2-b6b1-40a1-8c97-1988a0e91d05" (UID: "10d681c2-b6b1-40a1-8c97-1988a0e91d05"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:22:41.347072 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:41.347041    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/10d681c2-b6b1-40a1-8c97-1988a0e91d05-kube-api-access-zl46q" pod "10d681c2-b6b1-40a1-8c97-1988a0e91d05" (UID: "10d681c2-b6b1-40a1-8c97-1988a0e91d05"). InnerVolumeSpecName "kube-api-access-zl46q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:22:41.347407 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:41.347366    5510 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/10d681c2-b6b1-40a1-8c97-1988a0e91d05-istio-data\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:22:41.347407 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:41.347391    5510 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/10d681c2-b6b1-40a1-8c97-1988a0e91d05-workload-certs\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:22:41.347407 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:41.347405    5510 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zl46q\" (UniqueName: \"kubernetes.io/projected/10d681c2-b6b1-40a1-8c97-1988a0e91d05-kube-api-access-zl46q\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:22:41.347746 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:41.347421    5510 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/71b34137-bc68-4228-a7e0-30fda97183aa-istio-podinfo\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:22:41.347746 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:41.347436    5510 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/71b34137-bc68-4228-a7e0-30fda97183aa-istio-token\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:22:41.347746 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:41.347450    5510 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mmbqj\" (UniqueName: \"kubernetes.io/projected/71b34137-bc68-4228-a7e0-30fda97183aa-kube-api-access-mmbqj\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:22:41.347746 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:41.347454    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/10d681c2-b6b1-40a1-8c97-1988a0e91d05-istio-token" pod "10d681c2-b6b1-40a1-8c97-1988a0e91d05" (UID: "10d681c2-b6b1-40a1-8c97-1988a0e91d05"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:22:41.347746 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:41.347464    5510 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/10d681c2-b6b1-40a1-8c97-1988a0e91d05-workload-socket\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:22:41.347746 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:41.347511    5510 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/10d681c2-b6b1-40a1-8c97-1988a0e91d05-istiod-ca-cert\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:22:41.347746 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:41.347529    5510 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/71b34137-bc68-4228-a7e0-30fda97183aa-istio-envoy\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:22:41.347746 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:41.347545    5510 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/10d681c2-b6b1-40a1-8c97-1988a0e91d05-credential-socket\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:22:41.347746 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:41.347719    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/10d681c2-b6b1-40a1-8c97-1988a0e91d05-istio-envoy" pod "10d681c2-b6b1-40a1-8c97-1988a0e91d05" (UID: "10d681c2-b6b1-40a1-8c97-1988a0e91d05"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:22:41.349545 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:41.349517    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/10d681c2-b6b1-40a1-8c97-1988a0e91d05-istio-podinfo" pod "10d681c2-b6b1-40a1-8c97-1988a0e91d05" (UID: "10d681c2-b6b1-40a1-8c97-1988a0e91d05"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:22:41.449049 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:41.448954    5510 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/10d681c2-b6b1-40a1-8c97-1988a0e91d05-istio-token\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:22:41.449049 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:41.448985    5510 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/10d681c2-b6b1-40a1-8c97-1988a0e91d05-istio-envoy\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:22:41.449049 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:41.448994    5510 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/10d681c2-b6b1-40a1-8c97-1988a0e91d05-istio-podinfo\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:22:41.606319 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:41.606284    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-g64nz"]
Oct 30 22:22:41.610592 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:41.610561    5510 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-g64nz"]
Oct 30 22:22:42.168012 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:42.167926    5510 patch_prober.go:29] interesting pod/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-86dps container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.82:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 22:22:42.168012 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:42.168000    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-86dps" podUID="10d681c2-b6b1-40a1-8c97-1988a0e91d05" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.82:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:22:42.196923 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:42.196890    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-86dps"
Oct 30 22:22:42.196923 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:42.196915    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-86dps" event={"ID":"10d681c2-b6b1-40a1-8c97-1988a0e91d05","Type":"ContainerDied","Data":"e71dd109b65c5a1427d9172043c5e0bd42199d0f329ab8d348b985f2154430dc"}
Oct 30 22:22:42.197156 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:42.196975    5510 scope.go:122] "RemoveContainer" containerID="f791c34bc62d549080d2963539c7ef6fffcf7a4cd452bd82810ba5d43aae9b08"
Oct 30 22:22:42.376558 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:42.376523    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-86dps"]
Oct 30 22:22:42.423198 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:42.423110    5510 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-86dps"]
Oct 30 22:22:43.555735 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:43.555687    5510 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="10d681c2-b6b1-40a1-8c97-1988a0e91d05" path="/var/lib/kubelet/pods/10d681c2-b6b1-40a1-8c97-1988a0e91d05/volumes"
Oct 30 22:22:43.556254 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:43.556239    5510 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="71b34137-bc68-4228-a7e0-30fda97183aa" path="/var/lib/kubelet/pods/71b34137-bc68-4228-a7e0-30fda97183aa/volumes"
Oct 30 22:22:43.626396 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:43.621902    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:22:45.043307 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:45.043275    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:22:45.452414 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:45.452322    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:22:45.949570 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:45.949531    5510 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w1-943d8f-openshift-default-6c57fb7747-z6x89"]
Oct 30 22:22:45.951027 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:45.950997    5510 cpu_manager.go:414] "containerMap: removing container" podUID="71b34137-bc68-4228-a7e0-30fda97183aa" containerName="istio-proxy"
Oct 30 22:22:45.951220 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:45.951204    5510 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="71b34137-bc68-4228-a7e0-30fda97183aa" containerName="istio-proxy"
Oct 30 22:22:45.951364 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:45.951352    5510 cpu_manager.go:414] "containerMap: removing container" podUID="10d681c2-b6b1-40a1-8c97-1988a0e91d05" containerName="istio-proxy"
Oct 30 22:22:45.951543 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:45.951528    5510 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="10d681c2-b6b1-40a1-8c97-1988a0e91d05" containerName="istio-proxy"
Oct 30 22:22:45.951853 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:45.951836    5510 memory_manager.go:365] "RemoveStaleState removing state" podUID="71b34137-bc68-4228-a7e0-30fda97183aa" containerName="istio-proxy"
Oct 30 22:22:45.951970 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:45.951957    5510 memory_manager.go:365] "RemoveStaleState removing state" podUID="10d681c2-b6b1-40a1-8c97-1988a0e91d05" containerName="istio-proxy"
Oct 30 22:22:45.965892 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:45.965859    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w1-943d8f-openshift-default-6c57fb7747-z6x89"
Oct 30 22:22:46.028126 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:46.028086    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w1-943d8f-openshift-default-dockercfg-tjv5r\""
Oct 30 22:22:46.052645 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:46.044916    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dcbe788e-cc5c-4807-af27-b98a507ea810-istio-token\") pod \"e2e-shared-b-w1-943d8f-openshift-default-6c57fb7747-z6x89\" (UID: \"dcbe788e-cc5c-4807-af27-b98a507ea810\") " pod="openshift-ingress/e2e-shared-b-w1-943d8f-openshift-default-6c57fb7747-z6x89"
Oct 30 22:22:46.053492 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:46.053462    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dcbe788e-cc5c-4807-af27-b98a507ea810-workload-socket\") pod \"e2e-shared-b-w1-943d8f-openshift-default-6c57fb7747-z6x89\" (UID: \"dcbe788e-cc5c-4807-af27-b98a507ea810\") " pod="openshift-ingress/e2e-shared-b-w1-943d8f-openshift-default-6c57fb7747-z6x89"
Oct 30 22:22:46.053761 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:46.053704    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dcbe788e-cc5c-4807-af27-b98a507ea810-credential-socket\") pod \"e2e-shared-b-w1-943d8f-openshift-default-6c57fb7747-z6x89\" (UID: \"dcbe788e-cc5c-4807-af27-b98a507ea810\") " pod="openshift-ingress/e2e-shared-b-w1-943d8f-openshift-default-6c57fb7747-z6x89"
Oct 30 22:22:46.054310 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:46.054257    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dcbe788e-cc5c-4807-af27-b98a507ea810-istio-envoy\") pod \"e2e-shared-b-w1-943d8f-openshift-default-6c57fb7747-z6x89\" (UID: \"dcbe788e-cc5c-4807-af27-b98a507ea810\") " pod="openshift-ingress/e2e-shared-b-w1-943d8f-openshift-default-6c57fb7747-z6x89"
Oct 30 22:22:46.054559 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:46.054531    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jt726\" (UniqueName: \"kubernetes.io/projected/dcbe788e-cc5c-4807-af27-b98a507ea810-kube-api-access-jt726\") pod \"e2e-shared-b-w1-943d8f-openshift-default-6c57fb7747-z6x89\" (UID: \"dcbe788e-cc5c-4807-af27-b98a507ea810\") " pod="openshift-ingress/e2e-shared-b-w1-943d8f-openshift-default-6c57fb7747-z6x89"
Oct 30 22:22:46.054816 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:46.054791    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dcbe788e-cc5c-4807-af27-b98a507ea810-istiod-ca-cert\") pod \"e2e-shared-b-w1-943d8f-openshift-default-6c57fb7747-z6x89\" (UID: \"dcbe788e-cc5c-4807-af27-b98a507ea810\") " pod="openshift-ingress/e2e-shared-b-w1-943d8f-openshift-default-6c57fb7747-z6x89"
Oct 30 22:22:46.055053 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:46.055031    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dcbe788e-cc5c-4807-af27-b98a507ea810-istio-podinfo\") pod \"e2e-shared-b-w1-943d8f-openshift-default-6c57fb7747-z6x89\" (UID: \"dcbe788e-cc5c-4807-af27-b98a507ea810\") " pod="openshift-ingress/e2e-shared-b-w1-943d8f-openshift-default-6c57fb7747-z6x89"
Oct 30 22:22:46.055300 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:46.055280    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dcbe788e-cc5c-4807-af27-b98a507ea810-workload-certs\") pod \"e2e-shared-b-w1-943d8f-openshift-default-6c57fb7747-z6x89\" (UID: \"dcbe788e-cc5c-4807-af27-b98a507ea810\") " pod="openshift-ingress/e2e-shared-b-w1-943d8f-openshift-default-6c57fb7747-z6x89"
Oct 30 22:22:46.055497 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:46.055481    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dcbe788e-cc5c-4807-af27-b98a507ea810-istio-data\") pod \"e2e-shared-b-w1-943d8f-openshift-default-6c57fb7747-z6x89\" (UID: \"dcbe788e-cc5c-4807-af27-b98a507ea810\") " pod="openshift-ingress/e2e-shared-b-w1-943d8f-openshift-default-6c57fb7747-z6x89"
Oct 30 22:22:46.134484 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:46.134408    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:22:46.161352 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:46.161294    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dcbe788e-cc5c-4807-af27-b98a507ea810-istio-token\") pod \"e2e-shared-b-w1-943d8f-openshift-default-6c57fb7747-z6x89\" (UID: \"dcbe788e-cc5c-4807-af27-b98a507ea810\") " pod="openshift-ingress/e2e-shared-b-w1-943d8f-openshift-default-6c57fb7747-z6x89"
Oct 30 22:22:46.161644 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:46.161622    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dcbe788e-cc5c-4807-af27-b98a507ea810-workload-socket\") pod \"e2e-shared-b-w1-943d8f-openshift-default-6c57fb7747-z6x89\" (UID: \"dcbe788e-cc5c-4807-af27-b98a507ea810\") " pod="openshift-ingress/e2e-shared-b-w1-943d8f-openshift-default-6c57fb7747-z6x89"
Oct 30 22:22:46.206789 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:46.199797    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dcbe788e-cc5c-4807-af27-b98a507ea810-credential-socket\") pod \"e2e-shared-b-w1-943d8f-openshift-default-6c57fb7747-z6x89\" (UID: \"dcbe788e-cc5c-4807-af27-b98a507ea810\") " pod="openshift-ingress/e2e-shared-b-w1-943d8f-openshift-default-6c57fb7747-z6x89"
Oct 30 22:22:46.206789 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:46.200017    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dcbe788e-cc5c-4807-af27-b98a507ea810-istio-envoy\") pod \"e2e-shared-b-w1-943d8f-openshift-default-6c57fb7747-z6x89\" (UID: \"dcbe788e-cc5c-4807-af27-b98a507ea810\") " pod="openshift-ingress/e2e-shared-b-w1-943d8f-openshift-default-6c57fb7747-z6x89"
Oct 30 22:22:46.206789 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:46.200050    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jt726\" (UniqueName: \"kubernetes.io/projected/dcbe788e-cc5c-4807-af27-b98a507ea810-kube-api-access-jt726\") pod \"e2e-shared-b-w1-943d8f-openshift-default-6c57fb7747-z6x89\" (UID: \"dcbe788e-cc5c-4807-af27-b98a507ea810\") " pod="openshift-ingress/e2e-shared-b-w1-943d8f-openshift-default-6c57fb7747-z6x89"
Oct 30 22:22:46.206789 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:46.200096    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dcbe788e-cc5c-4807-af27-b98a507ea810-istiod-ca-cert\") pod \"e2e-shared-b-w1-943d8f-openshift-default-6c57fb7747-z6x89\" (UID: \"dcbe788e-cc5c-4807-af27-b98a507ea810\") " pod="openshift-ingress/e2e-shared-b-w1-943d8f-openshift-default-6c57fb7747-z6x89"
Oct 30 22:22:46.206789 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:46.200125    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dcbe788e-cc5c-4807-af27-b98a507ea810-istio-podinfo\") pod \"e2e-shared-b-w1-943d8f-openshift-default-6c57fb7747-z6x89\" (UID: \"dcbe788e-cc5c-4807-af27-b98a507ea810\") " pod="openshift-ingress/e2e-shared-b-w1-943d8f-openshift-default-6c57fb7747-z6x89"
Oct 30 22:22:46.206789 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:46.200194    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dcbe788e-cc5c-4807-af27-b98a507ea810-workload-certs\") pod \"e2e-shared-b-w1-943d8f-openshift-default-6c57fb7747-z6x89\" (UID: \"dcbe788e-cc5c-4807-af27-b98a507ea810\") " pod="openshift-ingress/e2e-shared-b-w1-943d8f-openshift-default-6c57fb7747-z6x89"
Oct 30 22:22:46.206789 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:46.200230    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dcbe788e-cc5c-4807-af27-b98a507ea810-istio-data\") pod \"e2e-shared-b-w1-943d8f-openshift-default-6c57fb7747-z6x89\" (UID: \"dcbe788e-cc5c-4807-af27-b98a507ea810\") " pod="openshift-ingress/e2e-shared-b-w1-943d8f-openshift-default-6c57fb7747-z6x89"
Oct 30 22:22:46.206789 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:46.200792    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dcbe788e-cc5c-4807-af27-b98a507ea810-istio-data\") pod \"e2e-shared-b-w1-943d8f-openshift-default-6c57fb7747-z6x89\" (UID: \"dcbe788e-cc5c-4807-af27-b98a507ea810\") " pod="openshift-ingress/e2e-shared-b-w1-943d8f-openshift-default-6c57fb7747-z6x89"
Oct 30 22:22:46.206789 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:46.204977    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dcbe788e-cc5c-4807-af27-b98a507ea810-workload-certs\") pod \"e2e-shared-b-w1-943d8f-openshift-default-6c57fb7747-z6x89\" (UID: \"dcbe788e-cc5c-4807-af27-b98a507ea810\") " pod="openshift-ingress/e2e-shared-b-w1-943d8f-openshift-default-6c57fb7747-z6x89"
Oct 30 22:22:46.206789 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:46.205181    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dcbe788e-cc5c-4807-af27-b98a507ea810-credential-socket\") pod \"e2e-shared-b-w1-943d8f-openshift-default-6c57fb7747-z6x89\" (UID: \"dcbe788e-cc5c-4807-af27-b98a507ea810\") " pod="openshift-ingress/e2e-shared-b-w1-943d8f-openshift-default-6c57fb7747-z6x89"
Oct 30 22:22:46.249772 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:46.249276    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dcbe788e-cc5c-4807-af27-b98a507ea810-istiod-ca-cert\") pod \"e2e-shared-b-w1-943d8f-openshift-default-6c57fb7747-z6x89\" (UID: \"dcbe788e-cc5c-4807-af27-b98a507ea810\") " pod="openshift-ingress/e2e-shared-b-w1-943d8f-openshift-default-6c57fb7747-z6x89"
Oct 30 22:22:46.341333 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:46.341299    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dcbe788e-cc5c-4807-af27-b98a507ea810-workload-socket\") pod \"e2e-shared-b-w1-943d8f-openshift-default-6c57fb7747-z6x89\" (UID: \"dcbe788e-cc5c-4807-af27-b98a507ea810\") " pod="openshift-ingress/e2e-shared-b-w1-943d8f-openshift-default-6c57fb7747-z6x89"
Oct 30 22:22:46.365709 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:46.347266    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w1-943d8f-openshift-default-6c57fb7747-z6x89"]
Oct 30 22:22:46.365709 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:46.349271    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dcbe788e-cc5c-4807-af27-b98a507ea810-istio-token\") pod \"e2e-shared-b-w1-943d8f-openshift-default-6c57fb7747-z6x89\" (UID: \"dcbe788e-cc5c-4807-af27-b98a507ea810\") " pod="openshift-ingress/e2e-shared-b-w1-943d8f-openshift-default-6c57fb7747-z6x89"
Oct 30 22:22:46.402829 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:46.402792    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jt726\" (UniqueName: \"kubernetes.io/projected/dcbe788e-cc5c-4807-af27-b98a507ea810-kube-api-access-jt726\") pod \"e2e-shared-b-w1-943d8f-openshift-default-6c57fb7747-z6x89\" (UID: \"dcbe788e-cc5c-4807-af27-b98a507ea810\") " pod="openshift-ingress/e2e-shared-b-w1-943d8f-openshift-default-6c57fb7747-z6x89"
Oct 30 22:22:46.406387 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:46.406354    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dcbe788e-cc5c-4807-af27-b98a507ea810-istio-envoy\") pod \"e2e-shared-b-w1-943d8f-openshift-default-6c57fb7747-z6x89\" (UID: \"dcbe788e-cc5c-4807-af27-b98a507ea810\") " pod="openshift-ingress/e2e-shared-b-w1-943d8f-openshift-default-6c57fb7747-z6x89"
Oct 30 22:22:46.411281 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:46.411216    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dcbe788e-cc5c-4807-af27-b98a507ea810-istio-podinfo\") pod \"e2e-shared-b-w1-943d8f-openshift-default-6c57fb7747-z6x89\" (UID: \"dcbe788e-cc5c-4807-af27-b98a507ea810\") " pod="openshift-ingress/e2e-shared-b-w1-943d8f-openshift-default-6c57fb7747-z6x89"
Oct 30 22:22:46.612101 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:46.606718    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:22:46.612101 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:46.606760    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w1-943d8f-openshift-default-6c57fb7747-z6x89"
Oct 30 22:22:47.044278 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:47.044244    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w1-943d8f-openshift-default-6c57fb7747-z6x89"]
Oct 30 22:22:47.044744 ip-10-0-136-210 kubenswrapper[5510]: W1030 22:22:47.044716    5510 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddcbe788e_cc5c_4807_af27_b98a507ea810.slice/crio-a097d38a60706d1740c4dea99d27bab8bce52b1031a17d69723ab7fc6fb7caf4 WatchSource:0}: Error finding container a097d38a60706d1740c4dea99d27bab8bce52b1031a17d69723ab7fc6fb7caf4: Status 404 returned error can't find the container with id a097d38a60706d1740c4dea99d27bab8bce52b1031a17d69723ab7fc6fb7caf4
Oct 30 22:22:47.047596 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:47.047559    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:22:47.047694 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:47.047626    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:22:47.047694 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:47.047671    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:22:47.410317 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:47.410219    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w1-943d8f-openshift-default-6c57fb7747-z6x89" event={"ID":"dcbe788e-cc5c-4807-af27-b98a507ea810","Type":"ContainerStarted","Data":"d7f8bddcd52f0750ee71642815306a33677a593752edf6eb4fdc0e6bdbe20a33"}
Oct 30 22:22:47.410317 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:47.410289    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w1-943d8f-openshift-default-6c57fb7747-z6x89" event={"ID":"dcbe788e-cc5c-4807-af27-b98a507ea810","Type":"ContainerStarted","Data":"a097d38a60706d1740c4dea99d27bab8bce52b1031a17d69723ab7fc6fb7caf4"}
Oct 30 22:22:47.530185 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:47.530119    5510 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w1-943d8f-openshift-default-6c57fb7747-z6x89" podStartSLOduration=2.530102359 podStartE2EDuration="2.530102359s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:22:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:22:47.526036326 +0000 UTC m=+1948.086846330" watchObservedRunningTime="2026-10-30 22:22:47.530102359 +0000 UTC m=+1948.090912432"
Oct 30 22:22:47.607457 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:47.607421    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w1-943d8f-openshift-default-6c57fb7747-z6x89"
Oct 30 22:22:48.312258 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:48.312226    5510 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-cf1826-54f5f6c8c4-d972s"]
Oct 30 22:22:48.317055 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:48.317035    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-cf1826-54f5f6c8c4-d972s"
Oct 30 22:22:48.414814 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:48.414784    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w2-cf1826-serving-cert\""
Oct 30 22:22:48.418865 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:48.418837    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-cf1826-54f5f6c8c4-d972s"]
Oct 30 22:22:48.426347 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:48.426323    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/58572a25-fdb3-417c-9ade-f20e2d714b78-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-cf1826-54f5f6c8c4-d972s\" (UID: \"58572a25-fdb3-417c-9ade-f20e2d714b78\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-cf1826-54f5f6c8c4-d972s"
Oct 30 22:22:48.426428 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:48.426397    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-drgk9\" (UniqueName: \"kubernetes.io/projected/58572a25-fdb3-417c-9ade-f20e2d714b78-kube-api-access-drgk9\") pod \"maas-api-e2e-shared-a-w2-cf1826-54f5f6c8c4-d972s\" (UID: \"58572a25-fdb3-417c-9ade-f20e2d714b78\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-cf1826-54f5f6c8c4-d972s"
Oct 30 22:22:48.426554 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:48.426531    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/58572a25-fdb3-417c-9ade-f20e2d714b78-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-cf1826-54f5f6c8c4-d972s\" (UID: \"58572a25-fdb3-417c-9ade-f20e2d714b78\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-cf1826-54f5f6c8c4-d972s"
Oct 30 22:22:48.426632 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:48.426612    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/58572a25-fdb3-417c-9ade-f20e2d714b78-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-cf1826-54f5f6c8c4-d972s\" (UID: \"58572a25-fdb3-417c-9ade-f20e2d714b78\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-cf1826-54f5f6c8c4-d972s"
Oct 30 22:22:48.528136 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:48.528102    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-drgk9\" (UniqueName: \"kubernetes.io/projected/58572a25-fdb3-417c-9ade-f20e2d714b78-kube-api-access-drgk9\") pod \"maas-api-e2e-shared-a-w2-cf1826-54f5f6c8c4-d972s\" (UID: \"58572a25-fdb3-417c-9ade-f20e2d714b78\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-cf1826-54f5f6c8c4-d972s"
Oct 30 22:22:48.528362 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:48.528338    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/58572a25-fdb3-417c-9ade-f20e2d714b78-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-cf1826-54f5f6c8c4-d972s\" (UID: \"58572a25-fdb3-417c-9ade-f20e2d714b78\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-cf1826-54f5f6c8c4-d972s"
Oct 30 22:22:48.528422 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:48.528388    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/58572a25-fdb3-417c-9ade-f20e2d714b78-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-cf1826-54f5f6c8c4-d972s\" (UID: \"58572a25-fdb3-417c-9ade-f20e2d714b78\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-cf1826-54f5f6c8c4-d972s"
Oct 30 22:22:48.528475 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:22:48.528462    5510 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-cf1826-serving-cert: secret "maas-api-e2e-shared-a-w2-cf1826-serving-cert" not found
Oct 30 22:22:48.528526 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:22:48.528516    5510 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/58572a25-fdb3-417c-9ade-f20e2d714b78-maas-api-tls podName:58572a25-fdb3-417c-9ade-f20e2d714b78 nodeName:}" failed. No retries permitted until 2026-10-30 22:22:49.028500177 +0000 UTC m=+1949.589310165 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/58572a25-fdb3-417c-9ade-f20e2d714b78-maas-api-tls") pod "maas-api-e2e-shared-a-w2-cf1826-54f5f6c8c4-d972s" (UID: "58572a25-fdb3-417c-9ade-f20e2d714b78") : secret "maas-api-e2e-shared-a-w2-cf1826-serving-cert" not found
Oct 30 22:22:48.528759 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:48.528745    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/58572a25-fdb3-417c-9ade-f20e2d714b78-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-cf1826-54f5f6c8c4-d972s\" (UID: \"58572a25-fdb3-417c-9ade-f20e2d714b78\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-cf1826-54f5f6c8c4-d972s"
Oct 30 22:22:48.529337 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:48.529311    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/58572a25-fdb3-417c-9ade-f20e2d714b78-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-cf1826-54f5f6c8c4-d972s\" (UID: \"58572a25-fdb3-417c-9ade-f20e2d714b78\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-cf1826-54f5f6c8c4-d972s"
Oct 30 22:22:48.530860 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:48.530837    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/58572a25-fdb3-417c-9ade-f20e2d714b78-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-cf1826-54f5f6c8c4-d972s\" (UID: \"58572a25-fdb3-417c-9ade-f20e2d714b78\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-cf1826-54f5f6c8c4-d972s"
Oct 30 22:22:48.569742 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:48.564696    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:22:48.627770 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:48.625593    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-drgk9\" (UniqueName: \"kubernetes.io/projected/58572a25-fdb3-417c-9ade-f20e2d714b78-kube-api-access-drgk9\") pod \"maas-api-e2e-shared-a-w2-cf1826-54f5f6c8c4-d972s\" (UID: \"58572a25-fdb3-417c-9ade-f20e2d714b78\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-cf1826-54f5f6c8c4-d972s"
Oct 30 22:22:48.631697 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:48.630533    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w1-943d8f-openshift-default-6c57fb7747-z6x89"
Oct 30 22:22:48.785696 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:48.781740    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:22:49.046453 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:49.046414    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/58572a25-fdb3-417c-9ade-f20e2d714b78-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-cf1826-54f5f6c8c4-d972s\" (UID: \"58572a25-fdb3-417c-9ade-f20e2d714b78\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-cf1826-54f5f6c8c4-d972s"
Oct 30 22:22:49.048974 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:49.048945    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/58572a25-fdb3-417c-9ade-f20e2d714b78-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-cf1826-54f5f6c8c4-d972s\" (UID: \"58572a25-fdb3-417c-9ade-f20e2d714b78\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-cf1826-54f5f6c8c4-d972s"
Oct 30 22:22:49.232006 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:49.231964    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-cf1826-54f5f6c8c4-d972s"
Oct 30 22:22:49.425138 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:49.425107    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w1-943d8f-openshift-default-6c57fb7747-z6x89"
Oct 30 22:22:49.426135 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:49.426113    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w1-943d8f-openshift-default-6c57fb7747-z6x89"
Oct 30 22:22:49.619443 ip-10-0-136-210 kubenswrapper[5510]: W1030 22:22:49.619413    5510 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod58572a25_fdb3_417c_9ade_f20e2d714b78.slice/crio-847c81f4764479569b7b66d71c7986c27dad1734a26cf19e7d30fa81a510f9aa WatchSource:0}: Error finding container 847c81f4764479569b7b66d71c7986c27dad1734a26cf19e7d30fa81a510f9aa: Status 404 returned error can't find the container with id 847c81f4764479569b7b66d71c7986c27dad1734a26cf19e7d30fa81a510f9aa
Oct 30 22:22:49.619923 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:49.619900    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-cf1826-54f5f6c8c4-d972s"]
Oct 30 22:22:50.433967 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:50.433925    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-cf1826-54f5f6c8c4-d972s" event={"ID":"58572a25-fdb3-417c-9ade-f20e2d714b78","Type":"ContainerStarted","Data":"847c81f4764479569b7b66d71c7986c27dad1734a26cf19e7d30fa81a510f9aa"}
Oct 30 22:22:50.513303 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:50.513272    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:22:50.766843 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:50.766740    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:22:51.444867 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:51.444833    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-cf1826-54f5f6c8c4-d972s" event={"ID":"58572a25-fdb3-417c-9ade-f20e2d714b78","Type":"ContainerStarted","Data":"0deb080acbc693aa01ed07d33c34b48459d268dcc7c5fa87f46fb0e3b1333450"}
Oct 30 22:22:51.445338 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:51.444915    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-cf1826-54f5f6c8c4-d972s"
Oct 30 22:22:53.832678 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:53.829145    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:22:54.126747 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:54.126185    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:22:54.129977 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:54.128372    5510 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-cf1826-54f5f6c8c4-d972s" podStartSLOduration=5.264702717 podStartE2EDuration="6.128298216s" totalImagesPullingTime="863.595499ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:22:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:22:51.579090642 +0000 UTC m=+1952.139900648" watchObservedRunningTime="2026-10-30 22:22:54.128298216 +0000 UTC m=+1954.689108223"
Oct 30 22:22:54.132534 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:54.132506    5510 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w1-943d8f-7b5555d7f9-v5c6w"]
Oct 30 22:22:54.168703 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:54.168640    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-943d8f-7b5555d7f9-v5c6w"
Oct 30 22:22:54.177684 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:54.174088    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-njt4t\" (UniqueName: \"kubernetes.io/projected/d303451d-0d81-4a9b-8bc6-91f019cfc4a0-kube-api-access-njt4t\") pod \"payload-pre-processing-e2e-shared-b-w1-943d8f-7b5555d7f9-v5c6w\" (UID: \"d303451d-0d81-4a9b-8bc6-91f019cfc4a0\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-943d8f-7b5555d7f9-v5c6w"
Oct 30 22:22:54.177684 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:54.174321    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d303451d-0d81-4a9b-8bc6-91f019cfc4a0-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w1-943d8f-7b5555d7f9-v5c6w\" (UID: \"d303451d-0d81-4a9b-8bc6-91f019cfc4a0\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-943d8f-7b5555d7f9-v5c6w"
Oct 30 22:22:54.263033 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:54.208488    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w1-943d8f-7b5555d7f9-v5c6w"]
Oct 30 22:22:54.263867 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:54.225574    5510 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w1-943d8f\""
Oct 30 22:22:54.300679 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:54.281117    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-njt4t\" (UniqueName: \"kubernetes.io/projected/d303451d-0d81-4a9b-8bc6-91f019cfc4a0-kube-api-access-njt4t\") pod \"payload-pre-processing-e2e-shared-b-w1-943d8f-7b5555d7f9-v5c6w\" (UID: \"d303451d-0d81-4a9b-8bc6-91f019cfc4a0\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-943d8f-7b5555d7f9-v5c6w"
Oct 30 22:22:54.408101 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:54.392008    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d303451d-0d81-4a9b-8bc6-91f019cfc4a0-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w1-943d8f-7b5555d7f9-v5c6w\" (UID: \"d303451d-0d81-4a9b-8bc6-91f019cfc4a0\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-943d8f-7b5555d7f9-v5c6w"
Oct 30 22:22:54.408101 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:54.393188    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d303451d-0d81-4a9b-8bc6-91f019cfc4a0-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w1-943d8f-7b5555d7f9-v5c6w\" (UID: \"d303451d-0d81-4a9b-8bc6-91f019cfc4a0\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-943d8f-7b5555d7f9-v5c6w"
Oct 30 22:22:54.502536 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:54.471648    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-njt4t\" (UniqueName: \"kubernetes.io/projected/d303451d-0d81-4a9b-8bc6-91f019cfc4a0-kube-api-access-njt4t\") pod \"payload-pre-processing-e2e-shared-b-w1-943d8f-7b5555d7f9-v5c6w\" (UID: \"d303451d-0d81-4a9b-8bc6-91f019cfc4a0\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-943d8f-7b5555d7f9-v5c6w"
Oct 30 22:22:54.519993 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:54.519836    5510 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w1-943d8f-7556597485-mznq2"]
Oct 30 22:22:54.525579 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:54.525548    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w1-943d8f-7556597485-mznq2"]
Oct 30 22:22:54.525740 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:54.525670    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-943d8f-7556597485-mznq2"
Oct 30 22:22:54.528100 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:54.528077    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-943d8f-7b5555d7f9-v5c6w"
Oct 30 22:22:54.576715 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:54.576690    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w1-943d8f-dockercfg-khkm6\""
Oct 30 22:22:54.596012 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:54.595837    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jz6p4\" (UniqueName: \"kubernetes.io/projected/1cd4d179-c02d-439e-afea-7dbf8b89d3ea-kube-api-access-jz6p4\") pod \"payload-processing-e2e-shared-b-w1-943d8f-7556597485-mznq2\" (UID: \"1cd4d179-c02d-439e-afea-7dbf8b89d3ea\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w1-943d8f-7556597485-mznq2"
Oct 30 22:22:54.596012 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:54.595991    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1cd4d179-c02d-439e-afea-7dbf8b89d3ea-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w1-943d8f-7556597485-mznq2\" (UID: \"1cd4d179-c02d-439e-afea-7dbf8b89d3ea\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w1-943d8f-7556597485-mznq2"
Oct 30 22:22:54.697414 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:54.697334    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jz6p4\" (UniqueName: \"kubernetes.io/projected/1cd4d179-c02d-439e-afea-7dbf8b89d3ea-kube-api-access-jz6p4\") pod \"payload-processing-e2e-shared-b-w1-943d8f-7556597485-mznq2\" (UID: \"1cd4d179-c02d-439e-afea-7dbf8b89d3ea\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w1-943d8f-7556597485-mznq2"
Oct 30 22:22:54.697573 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:54.697434    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1cd4d179-c02d-439e-afea-7dbf8b89d3ea-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w1-943d8f-7556597485-mznq2\" (UID: \"1cd4d179-c02d-439e-afea-7dbf8b89d3ea\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w1-943d8f-7556597485-mznq2"
Oct 30 22:22:54.698283 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:54.698262    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1cd4d179-c02d-439e-afea-7dbf8b89d3ea-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w1-943d8f-7556597485-mznq2\" (UID: \"1cd4d179-c02d-439e-afea-7dbf8b89d3ea\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w1-943d8f-7556597485-mznq2"
Oct 30 22:22:54.755992 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:54.755962    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jz6p4\" (UniqueName: \"kubernetes.io/projected/1cd4d179-c02d-439e-afea-7dbf8b89d3ea-kube-api-access-jz6p4\") pod \"payload-processing-e2e-shared-b-w1-943d8f-7556597485-mznq2\" (UID: \"1cd4d179-c02d-439e-afea-7dbf8b89d3ea\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w1-943d8f-7556597485-mznq2"
Oct 30 22:22:54.884612 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:54.884580    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-943d8f-7556597485-mznq2"
Oct 30 22:22:54.906598 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:54.906568    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w1-943d8f-7b5555d7f9-v5c6w"]
Oct 30 22:22:54.910266 ip-10-0-136-210 kubenswrapper[5510]: W1030 22:22:54.910241    5510 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd303451d_0d81_4a9b_8bc6_91f019cfc4a0.slice/crio-b4cb23f118ad5b95c16cc5659aac0b73847635679d9aa578c6369ac341572115 WatchSource:0}: Error finding container b4cb23f118ad5b95c16cc5659aac0b73847635679d9aa578c6369ac341572115: Status 404 returned error can't find the container with id b4cb23f118ad5b95c16cc5659aac0b73847635679d9aa578c6369ac341572115
Oct 30 22:22:55.264459 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:55.264427    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w1-943d8f-7556597485-mznq2"]
Oct 30 22:22:55.264923 ip-10-0-136-210 kubenswrapper[5510]: W1030 22:22:55.264896    5510 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1cd4d179_c02d_439e_afea_7dbf8b89d3ea.slice/crio-cfdcd4a6eaab7985da997e65bfbd31a03ca0d8d8198e46f4bcddc9c9b213d00e WatchSource:0}: Error finding container cfdcd4a6eaab7985da997e65bfbd31a03ca0d8d8198e46f4bcddc9c9b213d00e: Status 404 returned error can't find the container with id cfdcd4a6eaab7985da997e65bfbd31a03ca0d8d8198e46f4bcddc9c9b213d00e
Oct 30 22:22:55.509150 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:55.509096    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-943d8f-7556597485-mznq2" event={"ID":"1cd4d179-c02d-439e-afea-7dbf8b89d3ea","Type":"ContainerStarted","Data":"38ab9efb9db90ac7407b447c51b01274bbe9aed79c4195cec5c448ad4024e61f"}
Oct 30 22:22:55.509351 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:55.509166    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-943d8f-7556597485-mznq2" event={"ID":"1cd4d179-c02d-439e-afea-7dbf8b89d3ea","Type":"ContainerStarted","Data":"cfdcd4a6eaab7985da997e65bfbd31a03ca0d8d8198e46f4bcddc9c9b213d00e"}
Oct 30 22:22:55.509351 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:55.509305    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-943d8f-7556597485-mznq2"
Oct 30 22:22:55.510891 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:55.510857    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-943d8f-7b5555d7f9-v5c6w" event={"ID":"d303451d-0d81-4a9b-8bc6-91f019cfc4a0","Type":"ContainerStarted","Data":"d26b719290619fc74c172ef711ab53a39c5792ed79d4a6920aca02c769635075"}
Oct 30 22:22:55.510999 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:55.510899    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-943d8f-7b5555d7f9-v5c6w" event={"ID":"d303451d-0d81-4a9b-8bc6-91f019cfc4a0","Type":"ContainerStarted","Data":"b4cb23f118ad5b95c16cc5659aac0b73847635679d9aa578c6369ac341572115"}
Oct 30 22:22:55.510999 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:55.510919    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-943d8f-7b5555d7f9-v5c6w"
Oct 30 22:22:55.621102 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:55.621040    5510 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-943d8f-7556597485-mznq2" podStartSLOduration=1.621020621 podStartE2EDuration="1.621020621s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:22:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:22:55.61860522 +0000 UTC m=+1956.179415224" watchObservedRunningTime="2026-10-30 22:22:55.621020621 +0000 UTC m=+1956.181830627"
Oct 30 22:22:55.731075 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:55.730998    5510 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-943d8f-7b5555d7f9-v5c6w" podStartSLOduration=1.730978333 podStartE2EDuration="1.730978333s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:22:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:22:55.729546872 +0000 UTC m=+1956.290356878" watchObservedRunningTime="2026-10-30 22:22:55.730978333 +0000 UTC m=+1956.291788341"
Oct 30 22:22:57.458095 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:57.458069    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-cf1826-54f5f6c8c4-d972s"
Oct 30 22:22:58.746632 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:22:58.746591    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:23:03.315682 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:23:03.306928    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:23:06.522316 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:23:06.522285    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-943d8f-7556597485-mznq2"
Oct 30 22:23:06.522769 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:23:06.522355    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-943d8f-7b5555d7f9-v5c6w"
Oct 30 22:23:07.769243 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:23:07.765859    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:23:07.972678 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:23:07.969992    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:23:08.421804 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:23:08.421766    5510 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6gjbf" containerName="extractor"
Oct 30 22:23:12.534756 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:23:12.528288    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:23:13.883958 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:23:13.883919    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:23:14.091677 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:23:14.091622    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:23:17.122679 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:23:17.121879    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:23:21.996868 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:23:21.996810    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:23:22.277288 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:23:22.277239    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:23:22.913861 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:23:22.913819    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-ffkrj"]
Oct 30 22:23:22.914071 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:23:22.914050    5510 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-ffkrj" podUID="2937b77d-4ec3-4b52-86aa-b2e115927327" containerName="istio-proxy" containerID="cri-o://349968883e13a9093e2802f7c1bee63828f29f8f3eafc55d5d8d85f21478c610" gracePeriod=30
Oct 30 22:23:24.872757 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:23:24.872715    5510 patch_prober.go:29] interesting pod/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-ffkrj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 22:23:24.873157 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:23:24.872770    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-ffkrj" podUID="2937b77d-4ec3-4b52-86aa-b2e115927327" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 22:23:26.069302 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:23:26.069258    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:23:26.938070 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:23:26.938028    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:23:27.294109 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:23:27.294057    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:23:28.112716 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:23:28.112677    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:23:28.148136 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:23:28.148101    5510 generic.go:356] "Generic (PLEG): container finished" podID="2937b77d-4ec3-4b52-86aa-b2e115927327" containerID="349968883e13a9093e2802f7c1bee63828f29f8f3eafc55d5d8d85f21478c610" exitCode=0
Oct 30 22:23:28.148347 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:23:28.148180    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-ffkrj" event={"ID":"2937b77d-4ec3-4b52-86aa-b2e115927327","Type":"ContainerDied","Data":"349968883e13a9093e2802f7c1bee63828f29f8f3eafc55d5d8d85f21478c610"}
Oct 30 22:23:28.321728 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:23:28.321702    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-ffkrj"
Oct 30 22:23:28.450938 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:23:28.450831    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2937b77d-4ec3-4b52-86aa-b2e115927327-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2937b77d-4ec3-4b52-86aa-b2e115927327-istio-envoy\") pod \"2937b77d-4ec3-4b52-86aa-b2e115927327\" (UID: \"2937b77d-4ec3-4b52-86aa-b2e115927327\") "
Oct 30 22:23:28.450938 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:23:28.450898    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/2937b77d-4ec3-4b52-86aa-b2e115927327-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2937b77d-4ec3-4b52-86aa-b2e115927327-istio-podinfo\") pod \"2937b77d-4ec3-4b52-86aa-b2e115927327\" (UID: \"2937b77d-4ec3-4b52-86aa-b2e115927327\") "
Oct 30 22:23:28.451132 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:23:28.451010    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2937b77d-4ec3-4b52-86aa-b2e115927327-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2937b77d-4ec3-4b52-86aa-b2e115927327-workload-certs\") pod \"2937b77d-4ec3-4b52-86aa-b2e115927327\" (UID: \"2937b77d-4ec3-4b52-86aa-b2e115927327\") "
Oct 30 22:23:28.451132 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:23:28.451088    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2937b77d-4ec3-4b52-86aa-b2e115927327-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2937b77d-4ec3-4b52-86aa-b2e115927327-workload-socket\") pod \"2937b77d-4ec3-4b52-86aa-b2e115927327\" (UID: \"2937b77d-4ec3-4b52-86aa-b2e115927327\") "
Oct 30 22:23:28.451208 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:23:28.451196    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2937b77d-4ec3-4b52-86aa-b2e115927327-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2937b77d-4ec3-4b52-86aa-b2e115927327-credential-socket\") pod \"2937b77d-4ec3-4b52-86aa-b2e115927327\" (UID: \"2937b77d-4ec3-4b52-86aa-b2e115927327\") "
Oct 30 22:23:28.451247 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:23:28.451227    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2937b77d-4ec3-4b52-86aa-b2e115927327-istio-token\" (UniqueName: \"kubernetes.io/projected/2937b77d-4ec3-4b52-86aa-b2e115927327-istio-token\") pod \"2937b77d-4ec3-4b52-86aa-b2e115927327\" (UID: \"2937b77d-4ec3-4b52-86aa-b2e115927327\") "
Oct 30 22:23:28.451336 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:23:28.451323    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2937b77d-4ec3-4b52-86aa-b2e115927327-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2937b77d-4ec3-4b52-86aa-b2e115927327-istiod-ca-cert\") pod \"2937b77d-4ec3-4b52-86aa-b2e115927327\" (UID: \"2937b77d-4ec3-4b52-86aa-b2e115927327\") "
Oct 30 22:23:28.451449 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:23:28.451433    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2937b77d-4ec3-4b52-86aa-b2e115927327-kube-api-access-vw7gz\" (UniqueName: \"kubernetes.io/projected/2937b77d-4ec3-4b52-86aa-b2e115927327-kube-api-access-vw7gz\") pod \"2937b77d-4ec3-4b52-86aa-b2e115927327\" (UID: \"2937b77d-4ec3-4b52-86aa-b2e115927327\") "
Oct 30 22:23:28.451516 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:23:28.451493    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2937b77d-4ec3-4b52-86aa-b2e115927327-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2937b77d-4ec3-4b52-86aa-b2e115927327-istio-data\") pod \"2937b77d-4ec3-4b52-86aa-b2e115927327\" (UID: \"2937b77d-4ec3-4b52-86aa-b2e115927327\") "
Oct 30 22:23:28.451900 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:23:28.451873    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2937b77d-4ec3-4b52-86aa-b2e115927327-credential-socket" pod "2937b77d-4ec3-4b52-86aa-b2e115927327" (UID: "2937b77d-4ec3-4b52-86aa-b2e115927327"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:23:28.452321 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:23:28.452298    5510 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2937b77d-4ec3-4b52-86aa-b2e115927327-credential-socket\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:23:28.452321 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:23:28.452296    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2937b77d-4ec3-4b52-86aa-b2e115927327-istiod-ca-cert" pod "2937b77d-4ec3-4b52-86aa-b2e115927327" (UID: "2937b77d-4ec3-4b52-86aa-b2e115927327"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:23:28.452580 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:23:28.452555    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2937b77d-4ec3-4b52-86aa-b2e115927327-workload-certs" pod "2937b77d-4ec3-4b52-86aa-b2e115927327" (UID: "2937b77d-4ec3-4b52-86aa-b2e115927327"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:23:28.452777 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:23:28.452755    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2937b77d-4ec3-4b52-86aa-b2e115927327-workload-socket" pod "2937b77d-4ec3-4b52-86aa-b2e115927327" (UID: "2937b77d-4ec3-4b52-86aa-b2e115927327"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:23:28.452891 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:23:28.452874    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2937b77d-4ec3-4b52-86aa-b2e115927327-istio-data" pod "2937b77d-4ec3-4b52-86aa-b2e115927327" (UID: "2937b77d-4ec3-4b52-86aa-b2e115927327"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:23:28.453386 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:23:28.453365    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2937b77d-4ec3-4b52-86aa-b2e115927327-istio-envoy" pod "2937b77d-4ec3-4b52-86aa-b2e115927327" (UID: "2937b77d-4ec3-4b52-86aa-b2e115927327"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:23:28.454437 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:23:28.454411    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2937b77d-4ec3-4b52-86aa-b2e115927327-istio-podinfo" pod "2937b77d-4ec3-4b52-86aa-b2e115927327" (UID: "2937b77d-4ec3-4b52-86aa-b2e115927327"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:23:28.457536 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:23:28.457495    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2937b77d-4ec3-4b52-86aa-b2e115927327-istio-token" pod "2937b77d-4ec3-4b52-86aa-b2e115927327" (UID: "2937b77d-4ec3-4b52-86aa-b2e115927327"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:23:28.458950 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:23:28.458924    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2937b77d-4ec3-4b52-86aa-b2e115927327-kube-api-access-vw7gz" pod "2937b77d-4ec3-4b52-86aa-b2e115927327" (UID: "2937b77d-4ec3-4b52-86aa-b2e115927327"). InnerVolumeSpecName "kube-api-access-vw7gz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:23:28.553987 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:23:28.553947    5510 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2937b77d-4ec3-4b52-86aa-b2e115927327-workload-certs\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:23:28.553987 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:23:28.553989    5510 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2937b77d-4ec3-4b52-86aa-b2e115927327-workload-socket\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:23:28.554249 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:23:28.554004    5510 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2937b77d-4ec3-4b52-86aa-b2e115927327-istio-token\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:23:28.554249 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:23:28.554018    5510 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2937b77d-4ec3-4b52-86aa-b2e115927327-istiod-ca-cert\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:23:28.554249 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:23:28.554033    5510 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vw7gz\" (UniqueName: \"kubernetes.io/projected/2937b77d-4ec3-4b52-86aa-b2e115927327-kube-api-access-vw7gz\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:23:28.554249 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:23:28.554155    5510 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2937b77d-4ec3-4b52-86aa-b2e115927327-istio-data\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:23:28.554249 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:23:28.554176    5510 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2937b77d-4ec3-4b52-86aa-b2e115927327-istio-envoy\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:23:28.554249 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:23:28.554189    5510 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2937b77d-4ec3-4b52-86aa-b2e115927327-istio-podinfo\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:23:28.895074 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:23:28.895037    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-4da07739-bf6645659-p8vvn"]
Oct 30 22:23:29.156797 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:23:29.156706    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-ffkrj" event={"ID":"2937b77d-4ec3-4b52-86aa-b2e115927327","Type":"ContainerDied","Data":"1f2ae9c334561f4f2997a0b0be0410a89a9df83567746ca14a214ce6bdbeb638"}
Oct 30 22:23:29.156797 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:23:29.156773    5510 scope.go:122] "RemoveContainer" containerID="349968883e13a9093e2802f7c1bee63828f29f8f3eafc55d5d8d85f21478c610"
Oct 30 22:23:29.156797 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:23:29.156721    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-ffkrj"
Oct 30 22:23:29.210630 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:23:29.210594    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:23:29.331621 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:23:29.331575    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-ffkrj"]
Oct 30 22:23:29.383690 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:23:29.383016    5510 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-ffkrj"]
Oct 30 22:23:29.453909 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:23:29.453802    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:23:29.556089 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:23:29.556050    5510 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2937b77d-4ec3-4b52-86aa-b2e115927327" path="/var/lib/kubelet/pods/2937b77d-4ec3-4b52-86aa-b2e115927327/volumes"
Oct 30 22:23:29.667944 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:23:29.667902    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:23:31.649481 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:23:31.649375    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:23:31.897900 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:23:31.897864    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:23:33.196519 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:23:33.196427    5510 generic.go:356] "Generic (PLEG): container finished" podID="13760c22-8ba0-45ed-b2e4-4e23d5273901" containerID="f572ac7fae6b82a9bf28a0c16eee7c0515f93747676b59f2afe6e114fda3a496" exitCode=0
Oct 30 22:23:33.197381 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:23:33.196508    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-cf1826-openshift-default-cf9479bdc-jgkhc" event={"ID":"13760c22-8ba0-45ed-b2e4-4e23d5273901","Type":"ContainerDied","Data":"f572ac7fae6b82a9bf28a0c16eee7c0515f93747676b59f2afe6e114fda3a496"}
Oct 30 22:23:33.197381 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:23:33.197142    5510 scope.go:122] "RemoveContainer" containerID="f572ac7fae6b82a9bf28a0c16eee7c0515f93747676b59f2afe6e114fda3a496"
Oct 30 22:23:33.199114 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:23:33.199084    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:23:33.199179 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:23:33.199156    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:23:33.199215 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:23:33.199188    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:23:33.358679 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:23:33.354364    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:23:33.899183 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:23:33.899120    5510 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-4da07739-bf6645659-p8vvn" podUID="628be936-435a-4e97-9aaa-98924732fb01" containerName="payload-pre-processing" containerID="cri-o://282e85b9efc1a787703d885bb0cfb4a5260dff41039625bdf4d03942d02f3597" gracePeriod=25
Oct 30 22:23:34.212455 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:23:34.212419    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-cf1826-openshift-default-cf9479bdc-jgkhc" event={"ID":"13760c22-8ba0-45ed-b2e4-4e23d5273901","Type":"ContainerStarted","Data":"70eff1dd984d70819eb2acafb12e2d5504714cd7fb22aff55584bd04524f915b"}
Oct 30 22:23:34.214110 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:23:34.214083    5510 generic.go:356] "Generic (PLEG): container finished" podID="628be936-435a-4e97-9aaa-98924732fb01" containerID="282e85b9efc1a787703d885bb0cfb4a5260dff41039625bdf4d03942d02f3597" exitCode=0
Oct 30 22:23:34.214243 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:23:34.214159    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-4da07739-bf6645659-p8vvn" event={"ID":"628be936-435a-4e97-9aaa-98924732fb01","Type":"ContainerDied","Data":"282e85b9efc1a787703d885bb0cfb4a5260dff41039625bdf4d03942d02f3597"}
Oct 30 22:23:34.214243 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:23:34.214215    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-4da07739-bf6645659-p8vvn" event={"ID":"628be936-435a-4e97-9aaa-98924732fb01","Type":"ContainerDied","Data":"0e494a9a0f687fb1fbf5db7e78f086b161f5f67bcb71306913a0a9cf9f0712bc"}
Oct 30 22:23:34.214243 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:23:34.214238    5510 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0e494a9a0f687fb1fbf5db7e78f086b161f5f67bcb71306913a0a9cf9f0712bc"
Oct 30 22:23:34.227122 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:23:34.227098    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-4da07739-bf6645659-p8vvn"
Oct 30 22:23:34.227274 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:23:34.227257    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w2-cf1826-openshift-default-cf9479bdc-jgkhc"
Oct 30 22:23:34.227483 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:23:34.227460    5510 patch_prober.go:29] interesting pod/e2e-shared-a-w2-cf1826-openshift-default-cf9479bdc-jgkhc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.118:15021/healthz/ready\": dial tcp 10.133.0.118:15021: connect: connection refused" start-of-body=
Oct 30 22:23:34.227590 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:23:34.227504    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-cf1826-openshift-default-cf9479bdc-jgkhc" podUID="13760c22-8ba0-45ed-b2e4-4e23d5273901" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.118:15021/healthz/ready\": dial tcp 10.133.0.118:15021: connect: connection refused"
Oct 30 22:23:34.383165 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:23:34.383076    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/628be936-435a-4e97-9aaa-98924732fb01-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/628be936-435a-4e97-9aaa-98924732fb01-plugins-config-volume\") pod \"628be936-435a-4e97-9aaa-98924732fb01\" (UID: \"628be936-435a-4e97-9aaa-98924732fb01\") "
Oct 30 22:23:34.383468 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:23:34.383419    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/628be936-435a-4e97-9aaa-98924732fb01-kube-api-access-7btwt\" (UniqueName: \"kubernetes.io/projected/628be936-435a-4e97-9aaa-98924732fb01-kube-api-access-7btwt\") pod \"628be936-435a-4e97-9aaa-98924732fb01\" (UID: \"628be936-435a-4e97-9aaa-98924732fb01\") "
Oct 30 22:23:34.385556 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:23:34.385513    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/628be936-435a-4e97-9aaa-98924732fb01-plugins-config-volume" pod "628be936-435a-4e97-9aaa-98924732fb01" (UID: "628be936-435a-4e97-9aaa-98924732fb01"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:23:34.387839 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:23:34.387809    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/628be936-435a-4e97-9aaa-98924732fb01-kube-api-access-7btwt" pod "628be936-435a-4e97-9aaa-98924732fb01" (UID: "628be936-435a-4e97-9aaa-98924732fb01"). InnerVolumeSpecName "kube-api-access-7btwt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:23:34.486183 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:23:34.486098    5510 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7btwt\" (UniqueName: \"kubernetes.io/projected/628be936-435a-4e97-9aaa-98924732fb01-kube-api-access-7btwt\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:23:34.486183 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:23:34.486127    5510 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/628be936-435a-4e97-9aaa-98924732fb01-plugins-config-volume\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:23:34.573713 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:23:34.573389    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:23:34.882682 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:23:34.866979    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:23:35.133740 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:23:35.096215    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:23:35.231939 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:23:35.227862    5510 patch_prober.go:29] interesting pod/e2e-shared-a-w2-cf1826-openshift-default-cf9479bdc-jgkhc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.118:15021/healthz/ready\": dial tcp 10.133.0.118:15021: connect: connection refused" start-of-body=
Oct 30 22:23:35.231939 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:23:35.227923    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-cf1826-openshift-default-cf9479bdc-jgkhc" podUID="13760c22-8ba0-45ed-b2e4-4e23d5273901" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.118:15021/healthz/ready\": dial tcp 10.133.0.118:15021: connect: connection refused"
Oct 30 22:23:35.359695 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:23:35.348314    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-4da07739-bf6645659-p8vvn"
Oct 30 22:23:35.786640 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:23:35.786607    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-4da07739-bf6645659-p8vvn"]
Oct 30 22:23:35.832780 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:23:35.832736    5510 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-4da07739-bf6645659-p8vvn"]
Oct 30 22:23:36.228307 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:23:36.228270    5510 patch_prober.go:29] interesting pod/e2e-shared-a-w2-cf1826-openshift-default-cf9479bdc-jgkhc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.118:15021/healthz/ready\": dial tcp 10.133.0.118:15021: connect: connection refused" start-of-body=
Oct 30 22:23:36.228505 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:23:36.228329    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-cf1826-openshift-default-cf9479bdc-jgkhc" podUID="13760c22-8ba0-45ed-b2e4-4e23d5273901" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.118:15021/healthz/ready\": dial tcp 10.133.0.118:15021: connect: connection refused"
Oct 30 22:23:37.227399 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:23:37.227357    5510 patch_prober.go:29] interesting pod/e2e-shared-a-w2-cf1826-openshift-default-cf9479bdc-jgkhc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.118:15021/healthz/ready\": dial tcp 10.133.0.118:15021: connect: connection refused" start-of-body=
Oct 30 22:23:37.227863 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:23:37.227415    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-cf1826-openshift-default-cf9479bdc-jgkhc" podUID="13760c22-8ba0-45ed-b2e4-4e23d5273901" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.118:15021/healthz/ready\": dial tcp 10.133.0.118:15021: connect: connection refused"
Oct 30 22:23:37.577787 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:23:37.577753    5510 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="628be936-435a-4e97-9aaa-98924732fb01" path="/var/lib/kubelet/pods/628be936-435a-4e97-9aaa-98924732fb01/volumes"
Oct 30 22:23:38.228022 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:23:38.227984    5510 patch_prober.go:29] interesting pod/e2e-shared-a-w2-cf1826-openshift-default-cf9479bdc-jgkhc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.118:15021/healthz/ready\": dial tcp 10.133.0.118:15021: connect: connection refused" start-of-body=
Oct 30 22:23:38.228425 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:23:38.228038    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-cf1826-openshift-default-cf9479bdc-jgkhc" podUID="13760c22-8ba0-45ed-b2e4-4e23d5273901" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.118:15021/healthz/ready\": dial tcp 10.133.0.118:15021: connect: connection refused"
Oct 30 22:23:38.422517 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:23:38.422443    5510 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6gjbf" containerName="extractor"
Oct 30 22:23:38.583692 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:23:38.574498    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:23:38.775678 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:23:38.767339    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:23:39.227487 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:23:39.227454    5510 patch_prober.go:29] interesting pod/e2e-shared-a-w2-cf1826-openshift-default-cf9479bdc-jgkhc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.118:15021/healthz/ready\": dial tcp 10.133.0.118:15021: connect: connection refused" start-of-body=
Oct 30 22:23:39.227680 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:23:39.227504    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-cf1826-openshift-default-cf9479bdc-jgkhc" podUID="13760c22-8ba0-45ed-b2e4-4e23d5273901" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.118:15021/healthz/ready\": dial tcp 10.133.0.118:15021: connect: connection refused"
Oct 30 22:23:40.233790 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:23:40.233752    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w2-cf1826-openshift-default-cf9479bdc-jgkhc"
Oct 30 22:23:40.234265 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:23:40.234247    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w2-cf1826-openshift-default-cf9479bdc-jgkhc"
Oct 30 22:23:40.234899 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:23:40.234879    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w2-cf1826-openshift-default-cf9479bdc-jgkhc"
Oct 30 22:23:43.407683 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:23:43.402458    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:23:47.963710 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:23:47.963638    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:23:51.983784 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:23:51.983740    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:23:52.878366 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:23:52.878324    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:23:57.454963 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:23:57.454921    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:23:58.203946 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:23:58.203890    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:23:59.084897 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:23:59.084855    5510 patch_prober.go:29] interesting pod/e2e-api-a-7f76c4-openshift-default-5969946659-5dwws container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.121:15021/healthz/ready\": read tcp 10.133.0.2:60480->10.133.0.121:15021: read: connection reset by peer" start-of-body=
Oct 30 22:23:59.086472 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:23:59.084925    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-7f76c4-openshift-default-5969946659-5dwws" podUID="11bc04c5-986f-47f9-93d1-0f803b775eb5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.121:15021/healthz/ready\": read tcp 10.133.0.2:60480->10.133.0.121:15021: read: connection reset by peer"
Oct 30 22:23:59.625162 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:23:59.625128    5510 generic.go:356] "Generic (PLEG): container finished" podID="11bc04c5-986f-47f9-93d1-0f803b775eb5" containerID="128a7a0a8c823b9a180f97a1b06a35c9fe3cc8c144345631ac24025deac1a841" exitCode=0
Oct 30 22:23:59.625366 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:23:59.625207    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-7f76c4-openshift-default-5969946659-5dwws" event={"ID":"11bc04c5-986f-47f9-93d1-0f803b775eb5","Type":"ContainerDied","Data":"128a7a0a8c823b9a180f97a1b06a35c9fe3cc8c144345631ac24025deac1a841"}
Oct 30 22:23:59.625755 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:23:59.625734    5510 scope.go:122] "RemoveContainer" containerID="128a7a0a8c823b9a180f97a1b06a35c9fe3cc8c144345631ac24025deac1a841"
Oct 30 22:23:59.627464 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:23:59.627433    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:23:59.627536 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:23:59.627506    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:23:59.627583 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:23:59.627537    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:24:00.634688 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:00.634624    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-7f76c4-openshift-default-5969946659-5dwws" event={"ID":"11bc04c5-986f-47f9-93d1-0f803b775eb5","Type":"ContainerStarted","Data":"0f5feb6ac3de99eb05c29e507b3479c1cbb6551a68e5b1b7b816e2ca5ff6a31c"}
Oct 30 22:24:01.211277 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:01.211236    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-4da07739-5dcb995df-m8gbm"]
Oct 30 22:24:01.211513 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:01.211485    5510 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4da07739-5dcb995df-m8gbm" podUID="2f58edd6-b3f3-4074-af7f-04970eddc81e" containerName="maas-api" containerID="cri-o://28c5136904803d3c2f3bd41644b3c04948751011dd8446dc8f706f95b48d570d" gracePeriod=30
Oct 30 22:24:01.372007 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:01.371966    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:24:01.569224 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:01.569143    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-7f76c4-openshift-default-5969946659-5dwws"
Oct 30 22:24:01.600683 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:01.580791    5510 patch_prober.go:29] interesting pod/e2e-api-a-7f76c4-openshift-default-5969946659-5dwws container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.121:15021/healthz/ready\": dial tcp 10.133.0.121:15021: connect: connection refused" start-of-body=
Oct 30 22:24:01.600683 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:01.580850    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-7f76c4-openshift-default-5969946659-5dwws" podUID="11bc04c5-986f-47f9-93d1-0f803b775eb5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.121:15021/healthz/ready\": dial tcp 10.133.0.121:15021: connect: connection refused"
Oct 30 22:24:01.862680 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:01.850212    5510 generic.go:356] "Generic (PLEG): container finished" podID="2f58edd6-b3f3-4074-af7f-04970eddc81e" containerID="28c5136904803d3c2f3bd41644b3c04948751011dd8446dc8f706f95b48d570d" exitCode=0
Oct 30 22:24:01.871722 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:01.866321    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4da07739-5dcb995df-m8gbm" event={"ID":"2f58edd6-b3f3-4074-af7f-04970eddc81e","Type":"ContainerDied","Data":"28c5136904803d3c2f3bd41644b3c04948751011dd8446dc8f706f95b48d570d"}
Oct 30 22:24:01.924422 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:01.924392    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4da07739-5dcb995df-m8gbm"
Oct 30 22:24:02.072681 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:02.070761    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2f58edd6-b3f3-4074-af7f-04970eddc81e-kube-api-access-8qgf7\" (UniqueName: \"kubernetes.io/projected/2f58edd6-b3f3-4074-af7f-04970eddc81e-kube-api-access-8qgf7\") pod \"2f58edd6-b3f3-4074-af7f-04970eddc81e\" (UID: \"2f58edd6-b3f3-4074-af7f-04970eddc81e\") "
Oct 30 22:24:02.072681 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:02.070852    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2f58edd6-b3f3-4074-af7f-04970eddc81e-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2f58edd6-b3f3-4074-af7f-04970eddc81e-openshift-service-ca\") pod \"2f58edd6-b3f3-4074-af7f-04970eddc81e\" (UID: \"2f58edd6-b3f3-4074-af7f-04970eddc81e\") "
Oct 30 22:24:02.072681 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:02.070995    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2f58edd6-b3f3-4074-af7f-04970eddc81e-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2f58edd6-b3f3-4074-af7f-04970eddc81e-maas-api-tls\") pod \"2f58edd6-b3f3-4074-af7f-04970eddc81e\" (UID: \"2f58edd6-b3f3-4074-af7f-04970eddc81e\") "
Oct 30 22:24:02.072681 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:02.071084    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2f58edd6-b3f3-4074-af7f-04970eddc81e-metrics-certs\" (UniqueName: \"kubernetes.io/secret/2f58edd6-b3f3-4074-af7f-04970eddc81e-metrics-certs\") pod \"2f58edd6-b3f3-4074-af7f-04970eddc81e\" (UID: \"2f58edd6-b3f3-4074-af7f-04970eddc81e\") "
Oct 30 22:24:02.073318 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:02.073285    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2f58edd6-b3f3-4074-af7f-04970eddc81e-openshift-service-ca" pod "2f58edd6-b3f3-4074-af7f-04970eddc81e" (UID: "2f58edd6-b3f3-4074-af7f-04970eddc81e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:24:02.074981 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:02.074951    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2f58edd6-b3f3-4074-af7f-04970eddc81e-metrics-certs" pod "2f58edd6-b3f3-4074-af7f-04970eddc81e" (UID: "2f58edd6-b3f3-4074-af7f-04970eddc81e"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:24:02.076013 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:02.075988    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2f58edd6-b3f3-4074-af7f-04970eddc81e-kube-api-access-8qgf7" pod "2f58edd6-b3f3-4074-af7f-04970eddc81e" (UID: "2f58edd6-b3f3-4074-af7f-04970eddc81e"). InnerVolumeSpecName "kube-api-access-8qgf7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:24:02.076157 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:02.076042    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2f58edd6-b3f3-4074-af7f-04970eddc81e-maas-api-tls" pod "2f58edd6-b3f3-4074-af7f-04970eddc81e" (UID: "2f58edd6-b3f3-4074-af7f-04970eddc81e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:24:02.181671 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:02.181155    5510 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8qgf7\" (UniqueName: \"kubernetes.io/projected/2f58edd6-b3f3-4074-af7f-04970eddc81e-kube-api-access-8qgf7\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:24:02.181671 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:02.181195    5510 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2f58edd6-b3f3-4074-af7f-04970eddc81e-openshift-service-ca\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:24:02.181671 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:02.181213    5510 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2f58edd6-b3f3-4074-af7f-04970eddc81e-maas-api-tls\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:24:02.181671 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:02.181227    5510 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2f58edd6-b3f3-4074-af7f-04970eddc81e-metrics-certs\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:24:02.567680 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:02.566785    5510 patch_prober.go:29] interesting pod/e2e-api-a-7f76c4-openshift-default-5969946659-5dwws container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.121:15021/healthz/ready\": dial tcp 10.133.0.121:15021: connect: connection refused" start-of-body=
Oct 30 22:24:02.567680 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:02.566851    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-7f76c4-openshift-default-5969946659-5dwws" podUID="11bc04c5-986f-47f9-93d1-0f803b775eb5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.121:15021/healthz/ready\": dial tcp 10.133.0.121:15021: connect: connection refused"
Oct 30 22:24:02.876678 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:02.876573    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4da07739-5dcb995df-m8gbm"
Oct 30 22:24:02.876678 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:02.876613    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4da07739-5dcb995df-m8gbm" event={"ID":"2f58edd6-b3f3-4074-af7f-04970eddc81e","Type":"ContainerDied","Data":"026f761b5cb5681be3cd6b471cacec5cdc5f94d8ef3a278fc9a6e17898871ce9"}
Oct 30 22:24:02.877240 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:02.876691    5510 scope.go:122] "RemoveContainer" containerID="28c5136904803d3c2f3bd41644b3c04948751011dd8446dc8f706f95b48d570d"
Oct 30 22:24:02.888396 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:02.888364    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:24:03.199854 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:03.199725    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-4da07739-5dcb995df-m8gbm"]
Oct 30 22:24:03.221127 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:03.221016    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:24:03.243701 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:03.241001    5510 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-4da07739-5dcb995df-m8gbm"]
Oct 30 22:24:03.432814 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:03.431837    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:24:03.555066 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:03.555028    5510 patch_prober.go:29] interesting pod/e2e-api-a-7f76c4-openshift-default-5969946659-5dwws container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.121:15021/healthz/ready\": dial tcp 10.133.0.121:15021: connect: connection refused" start-of-body=
Oct 30 22:24:03.555378 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:03.555343    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-7f76c4-openshift-default-5969946659-5dwws" podUID="11bc04c5-986f-47f9-93d1-0f803b775eb5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.121:15021/healthz/ready\": dial tcp 10.133.0.121:15021: connect: connection refused"
Oct 30 22:24:03.618801 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:03.618767    5510 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2f58edd6-b3f3-4074-af7f-04970eddc81e" path="/var/lib/kubelet/pods/2f58edd6-b3f3-4074-af7f-04970eddc81e/volumes"
Oct 30 22:24:03.869754 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:03.869631    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:24:04.569143 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:04.569096    5510 patch_prober.go:29] interesting pod/e2e-api-a-7f76c4-openshift-default-5969946659-5dwws container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.121:15021/healthz/ready\": dial tcp 10.133.0.121:15021: connect: connection refused" start-of-body=
Oct 30 22:24:04.569696 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:04.569161    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-7f76c4-openshift-default-5969946659-5dwws" podUID="11bc04c5-986f-47f9-93d1-0f803b775eb5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.121:15021/healthz/ready\": dial tcp 10.133.0.121:15021: connect: connection refused"
Oct 30 22:24:05.561136 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:05.561096    5510 patch_prober.go:29] interesting pod/e2e-api-a-7f76c4-openshift-default-5969946659-5dwws container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.121:15021/healthz/ready\": dial tcp 10.133.0.121:15021: connect: connection refused" start-of-body=
Oct 30 22:24:05.568761 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:05.568721    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-7f76c4-openshift-default-5969946659-5dwws" podUID="11bc04c5-986f-47f9-93d1-0f803b775eb5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.121:15021/healthz/ready\": dial tcp 10.133.0.121:15021: connect: connection refused"
Oct 30 22:24:06.559481 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:06.559442    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-7f76c4-openshift-default-5969946659-5dwws"
Oct 30 22:24:06.560031 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:06.559877    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-7f76c4-openshift-default-5969946659-5dwws"
Oct 30 22:24:06.560718 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:06.560691    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-7f76c4-openshift-default-5969946659-5dwws"
Oct 30 22:24:08.388572 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:08.388511    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:24:08.480180 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:24:08.480138    5510 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6gjbf" containerName="extractor"
Oct 30 22:24:10.809332 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:10.809291    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:24:11.077235 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:11.077133    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:24:11.141006 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:11.140973    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-4da07739-openshift-default-6bd77c67d9-7wknn"]
Oct 30 22:24:11.141226 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:11.141202    5510 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-4da07739-openshift-default-6bd77c67d9-7wknn" podUID="f4d4ae01-a1b3-41ea-9e54-88edbe45642c" containerName="istio-proxy" containerID="cri-o://7ce8dd60f4818547de0c6d5dacae278101c713a7c255b98121432465fb8c4545" gracePeriod=30
Oct 30 22:24:11.321685 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:11.316149    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:24:12.220518 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:12.220463    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:24:13.022741 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:12.996483    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:24:13.294097 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:13.293987    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:24:13.638011 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:13.637913    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:24:16.441749 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:16.441719    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-4da07739-openshift-default-6bd77c67d9-7wknn"
Oct 30 22:24:16.519669 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:16.519631    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f4d4ae01-a1b3-41ea-9e54-88edbe45642c-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f4d4ae01-a1b3-41ea-9e54-88edbe45642c-istiod-ca-cert\") pod \"f4d4ae01-a1b3-41ea-9e54-88edbe45642c\" (UID: \"f4d4ae01-a1b3-41ea-9e54-88edbe45642c\") "
Oct 30 22:24:16.519888 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:16.519783    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f4d4ae01-a1b3-41ea-9e54-88edbe45642c-istio-token\" (UniqueName: \"kubernetes.io/projected/f4d4ae01-a1b3-41ea-9e54-88edbe45642c-istio-token\") pod \"f4d4ae01-a1b3-41ea-9e54-88edbe45642c\" (UID: \"f4d4ae01-a1b3-41ea-9e54-88edbe45642c\") "
Oct 30 22:24:16.519979 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:16.519964    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f4d4ae01-a1b3-41ea-9e54-88edbe45642c-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4d4ae01-a1b3-41ea-9e54-88edbe45642c-workload-socket\") pod \"f4d4ae01-a1b3-41ea-9e54-88edbe45642c\" (UID: \"f4d4ae01-a1b3-41ea-9e54-88edbe45642c\") "
Oct 30 22:24:16.520038 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:16.520017    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f4d4ae01-a1b3-41ea-9e54-88edbe45642c-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f4d4ae01-a1b3-41ea-9e54-88edbe45642c-workload-certs\") pod \"f4d4ae01-a1b3-41ea-9e54-88edbe45642c\" (UID: \"f4d4ae01-a1b3-41ea-9e54-88edbe45642c\") "
Oct 30 22:24:16.520107 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:16.520060    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f4d4ae01-a1b3-41ea-9e54-88edbe45642c-kube-api-access-w5stm\" (UniqueName: \"kubernetes.io/projected/f4d4ae01-a1b3-41ea-9e54-88edbe45642c-kube-api-access-w5stm\") pod \"f4d4ae01-a1b3-41ea-9e54-88edbe45642c\" (UID: \"f4d4ae01-a1b3-41ea-9e54-88edbe45642c\") "
Oct 30 22:24:16.520107 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:16.520076    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f4d4ae01-a1b3-41ea-9e54-88edbe45642c-istiod-ca-cert" pod "f4d4ae01-a1b3-41ea-9e54-88edbe45642c" (UID: "f4d4ae01-a1b3-41ea-9e54-88edbe45642c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:24:16.520215 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:16.520132    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f4d4ae01-a1b3-41ea-9e54-88edbe45642c-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f4d4ae01-a1b3-41ea-9e54-88edbe45642c-istio-envoy\") pod \"f4d4ae01-a1b3-41ea-9e54-88edbe45642c\" (UID: \"f4d4ae01-a1b3-41ea-9e54-88edbe45642c\") "
Oct 30 22:24:16.520215 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:16.520209    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/f4d4ae01-a1b3-41ea-9e54-88edbe45642c-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f4d4ae01-a1b3-41ea-9e54-88edbe45642c-istio-podinfo\") pod \"f4d4ae01-a1b3-41ea-9e54-88edbe45642c\" (UID: \"f4d4ae01-a1b3-41ea-9e54-88edbe45642c\") "
Oct 30 22:24:16.520331 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:16.520259    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f4d4ae01-a1b3-41ea-9e54-88edbe45642c-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4d4ae01-a1b3-41ea-9e54-88edbe45642c-credential-socket\") pod \"f4d4ae01-a1b3-41ea-9e54-88edbe45642c\" (UID: \"f4d4ae01-a1b3-41ea-9e54-88edbe45642c\") "
Oct 30 22:24:16.520331 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:16.520291    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f4d4ae01-a1b3-41ea-9e54-88edbe45642c-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f4d4ae01-a1b3-41ea-9e54-88edbe45642c-istio-data\") pod \"f4d4ae01-a1b3-41ea-9e54-88edbe45642c\" (UID: \"f4d4ae01-a1b3-41ea-9e54-88edbe45642c\") "
Oct 30 22:24:16.520331 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:16.520300    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f4d4ae01-a1b3-41ea-9e54-88edbe45642c-workload-certs" pod "f4d4ae01-a1b3-41ea-9e54-88edbe45642c" (UID: "f4d4ae01-a1b3-41ea-9e54-88edbe45642c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:24:16.520331 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:16.520288    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f4d4ae01-a1b3-41ea-9e54-88edbe45642c-workload-socket" pod "f4d4ae01-a1b3-41ea-9e54-88edbe45642c" (UID: "f4d4ae01-a1b3-41ea-9e54-88edbe45642c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:24:16.520544 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:16.520519    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f4d4ae01-a1b3-41ea-9e54-88edbe45642c-credential-socket" pod "f4d4ae01-a1b3-41ea-9e54-88edbe45642c" (UID: "f4d4ae01-a1b3-41ea-9e54-88edbe45642c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:24:16.520960 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:16.520936    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f4d4ae01-a1b3-41ea-9e54-88edbe45642c-istio-data" pod "f4d4ae01-a1b3-41ea-9e54-88edbe45642c" (UID: "f4d4ae01-a1b3-41ea-9e54-88edbe45642c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:24:16.521202 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:16.521181    5510 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4d4ae01-a1b3-41ea-9e54-88edbe45642c-workload-socket\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:24:16.521254 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:16.521210    5510 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f4d4ae01-a1b3-41ea-9e54-88edbe45642c-workload-certs\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:24:16.521254 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:16.521226    5510 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4d4ae01-a1b3-41ea-9e54-88edbe45642c-credential-socket\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:24:16.521254 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:16.521242    5510 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f4d4ae01-a1b3-41ea-9e54-88edbe45642c-istio-data\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:24:16.521351 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:16.521256    5510 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f4d4ae01-a1b3-41ea-9e54-88edbe45642c-istiod-ca-cert\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:24:16.522481 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:16.522459    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f4d4ae01-a1b3-41ea-9e54-88edbe45642c-istio-envoy" pod "f4d4ae01-a1b3-41ea-9e54-88edbe45642c" (UID: "f4d4ae01-a1b3-41ea-9e54-88edbe45642c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:24:16.522570 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:16.522548    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f4d4ae01-a1b3-41ea-9e54-88edbe45642c-kube-api-access-w5stm" pod "f4d4ae01-a1b3-41ea-9e54-88edbe45642c" (UID: "f4d4ae01-a1b3-41ea-9e54-88edbe45642c"). InnerVolumeSpecName "kube-api-access-w5stm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:24:16.522749 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:16.522720    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f4d4ae01-a1b3-41ea-9e54-88edbe45642c-istio-token" pod "f4d4ae01-a1b3-41ea-9e54-88edbe45642c" (UID: "f4d4ae01-a1b3-41ea-9e54-88edbe45642c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:24:16.522814 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:16.522791    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f4d4ae01-a1b3-41ea-9e54-88edbe45642c-istio-podinfo" pod "f4d4ae01-a1b3-41ea-9e54-88edbe45642c" (UID: "f4d4ae01-a1b3-41ea-9e54-88edbe45642c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:24:16.622905 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:16.622873    5510 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w5stm\" (UniqueName: \"kubernetes.io/projected/f4d4ae01-a1b3-41ea-9e54-88edbe45642c-kube-api-access-w5stm\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:24:16.622905 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:16.622902    5510 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f4d4ae01-a1b3-41ea-9e54-88edbe45642c-istio-envoy\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:24:16.622905 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:16.622912    5510 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f4d4ae01-a1b3-41ea-9e54-88edbe45642c-istio-podinfo\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:24:16.623150 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:16.622922    5510 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f4d4ae01-a1b3-41ea-9e54-88edbe45642c-istio-token\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:24:16.721166 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:16.720907    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:24:17.187946 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:17.187902    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:24:17.200146 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:17.200116    5510 generic.go:356] "Generic (PLEG): container finished" podID="f4d4ae01-a1b3-41ea-9e54-88edbe45642c" containerID="7ce8dd60f4818547de0c6d5dacae278101c713a7c255b98121432465fb8c4545" exitCode=0
Oct 30 22:24:17.200293 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:17.200162    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-4da07739-openshift-default-6bd77c67d9-7wknn"
Oct 30 22:24:17.200293 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:17.200200    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-4da07739-openshift-default-6bd77c67d9-7wknn" event={"ID":"f4d4ae01-a1b3-41ea-9e54-88edbe45642c","Type":"ContainerDied","Data":"7ce8dd60f4818547de0c6d5dacae278101c713a7c255b98121432465fb8c4545"}
Oct 30 22:24:17.200293 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:17.200252    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-4da07739-openshift-default-6bd77c67d9-7wknn" event={"ID":"f4d4ae01-a1b3-41ea-9e54-88edbe45642c","Type":"ContainerDied","Data":"54e372f78382471142d59580ad3a6d62d5ac5d65df657ec9f9f6dcaa443d8f23"}
Oct 30 22:24:17.200293 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:17.200282    5510 scope.go:122] "RemoveContainer" containerID="7ce8dd60f4818547de0c6d5dacae278101c713a7c255b98121432465fb8c4545"
Oct 30 22:24:17.212347 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:17.212321    5510 scope.go:122] "RemoveContainer" containerID="0266b6992b04d91261c6136773c8e9e08aa1dbfe4c76a25afc7fb6a7a7eb3f49"
Oct 30 22:24:17.223227 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:17.223203    5510 scope.go:122] "RemoveContainer" containerID="7ce8dd60f4818547de0c6d5dacae278101c713a7c255b98121432465fb8c4545"
Oct 30 22:24:17.223521 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:24:17.223500    5510 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7ce8dd60f4818547de0c6d5dacae278101c713a7c255b98121432465fb8c4545\": container with ID starting with 7ce8dd60f4818547de0c6d5dacae278101c713a7c255b98121432465fb8c4545 not found: ID does not exist" containerID="7ce8dd60f4818547de0c6d5dacae278101c713a7c255b98121432465fb8c4545"
Oct 30 22:24:17.223579 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:17.223530    5510 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7ce8dd60f4818547de0c6d5dacae278101c713a7c255b98121432465fb8c4545"} err="failed to get container status \"7ce8dd60f4818547de0c6d5dacae278101c713a7c255b98121432465fb8c4545\": rpc error: code = NotFound desc = could not find container \"7ce8dd60f4818547de0c6d5dacae278101c713a7c255b98121432465fb8c4545\": container with ID starting with 7ce8dd60f4818547de0c6d5dacae278101c713a7c255b98121432465fb8c4545 not found: ID does not exist"
Oct 30 22:24:17.223579 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:17.223550    5510 scope.go:122] "RemoveContainer" containerID="0266b6992b04d91261c6136773c8e9e08aa1dbfe4c76a25afc7fb6a7a7eb3f49"
Oct 30 22:24:17.223799 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:24:17.223779    5510 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0266b6992b04d91261c6136773c8e9e08aa1dbfe4c76a25afc7fb6a7a7eb3f49\": container with ID starting with 0266b6992b04d91261c6136773c8e9e08aa1dbfe4c76a25afc7fb6a7a7eb3f49 not found: ID does not exist" containerID="0266b6992b04d91261c6136773c8e9e08aa1dbfe4c76a25afc7fb6a7a7eb3f49"
Oct 30 22:24:17.223853 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:17.223809    5510 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0266b6992b04d91261c6136773c8e9e08aa1dbfe4c76a25afc7fb6a7a7eb3f49"} err="failed to get container status \"0266b6992b04d91261c6136773c8e9e08aa1dbfe4c76a25afc7fb6a7a7eb3f49\": rpc error: code = NotFound desc = could not find container \"0266b6992b04d91261c6136773c8e9e08aa1dbfe4c76a25afc7fb6a7a7eb3f49\": container with ID starting with 0266b6992b04d91261c6136773c8e9e08aa1dbfe4c76a25afc7fb6a7a7eb3f49 not found: ID does not exist"
Oct 30 22:24:17.407568 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:17.407527    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-4da07739-openshift-default-6bd77c67d9-7wknn"]
Oct 30 22:24:17.454848 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:17.454770    5510 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-4da07739-openshift-default-6bd77c67d9-7wknn"]
Oct 30 22:24:17.596737 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:17.596703    5510 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f4d4ae01-a1b3-41ea-9e54-88edbe45642c" path="/var/lib/kubelet/pods/f4d4ae01-a1b3-41ea-9e54-88edbe45642c/volumes"
Oct 30 22:24:17.840046 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:17.839980    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:24:18.047203 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:18.047164    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:24:18.589684 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:18.586755    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:24:18.912877 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:18.912788    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:24:19.121933 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:19.121898    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:24:21.064499 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:21.064463    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:24:21.343179 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:21.343095    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:24:21.383288 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:21.383247    5510 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-5637a7-6dfc655fbc-ms5sb"]
Oct 30 22:24:21.384681 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:21.384627    5510 cpu_manager.go:414] "containerMap: removing container" podUID="628be936-435a-4e97-9aaa-98924732fb01" containerName="payload-pre-processing"
Oct 30 22:24:21.384848 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:21.384831    5510 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="628be936-435a-4e97-9aaa-98924732fb01" containerName="payload-pre-processing"
Oct 30 22:24:21.384948 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:21.384938    5510 cpu_manager.go:414] "containerMap: removing container" podUID="2f58edd6-b3f3-4074-af7f-04970eddc81e" containerName="maas-api"
Oct 30 22:24:21.385024 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:21.385015    5510 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2f58edd6-b3f3-4074-af7f-04970eddc81e" containerName="maas-api"
Oct 30 22:24:21.385123 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:21.385114    5510 cpu_manager.go:414] "containerMap: removing container" podUID="2937b77d-4ec3-4b52-86aa-b2e115927327" containerName="istio-proxy"
Oct 30 22:24:21.385193 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:21.385183    5510 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2937b77d-4ec3-4b52-86aa-b2e115927327" containerName="istio-proxy"
Oct 30 22:24:21.385281 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:21.385272    5510 cpu_manager.go:414] "containerMap: removing container" podUID="f4d4ae01-a1b3-41ea-9e54-88edbe45642c" containerName="istio-proxy"
Oct 30 22:24:21.385344 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:21.385336    5510 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f4d4ae01-a1b3-41ea-9e54-88edbe45642c" containerName="istio-proxy"
Oct 30 22:24:21.385423 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:21.385414    5510 cpu_manager.go:414] "containerMap: removing container" podUID="f4d4ae01-a1b3-41ea-9e54-88edbe45642c" containerName="istio-proxy"
Oct 30 22:24:21.385495 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:21.385486    5510 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f4d4ae01-a1b3-41ea-9e54-88edbe45642c" containerName="istio-proxy"
Oct 30 22:24:21.385799 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:21.385785    5510 memory_manager.go:365] "RemoveStaleState removing state" podUID="628be936-435a-4e97-9aaa-98924732fb01" containerName="payload-pre-processing"
Oct 30 22:24:21.385902 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:21.385892    5510 memory_manager.go:365] "RemoveStaleState removing state" podUID="2f58edd6-b3f3-4074-af7f-04970eddc81e" containerName="maas-api"
Oct 30 22:24:21.385981 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:21.385973    5510 memory_manager.go:365] "RemoveStaleState removing state" podUID="2937b77d-4ec3-4b52-86aa-b2e115927327" containerName="istio-proxy"
Oct 30 22:24:21.386055 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:21.386047    5510 memory_manager.go:365] "RemoveStaleState removing state" podUID="f4d4ae01-a1b3-41ea-9e54-88edbe45642c" containerName="istio-proxy"
Oct 30 22:24:21.386123 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:21.386114    5510 memory_manager.go:365] "RemoveStaleState removing state" podUID="f4d4ae01-a1b3-41ea-9e54-88edbe45642c" containerName="istio-proxy"
Oct 30 22:24:21.391936 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:21.390916    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-5637a7-6dfc655fbc-ms5sb"
Oct 30 22:24:21.422489 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:21.422402    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4009806a-5d15-4224-9b63-e3bc115f35ad-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-5637a7-6dfc655fbc-ms5sb\" (UID: \"4009806a-5d15-4224-9b63-e3bc115f35ad\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-5637a7-6dfc655fbc-ms5sb"
Oct 30 22:24:21.422903 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:21.422879    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6mdh4\" (UniqueName: \"kubernetes.io/projected/4009806a-5d15-4224-9b63-e3bc115f35ad-kube-api-access-6mdh4\") pod \"payload-pre-processing-e2e-inf-a-5637a7-6dfc655fbc-ms5sb\" (UID: \"4009806a-5d15-4224-9b63-e3bc115f35ad\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-5637a7-6dfc655fbc-ms5sb"
Oct 30 22:24:21.455957 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:21.454433    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-5637a7-6dfc655fbc-ms5sb"]
Oct 30 22:24:21.489352 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:21.489291    5510 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-5637a7\""
Oct 30 22:24:21.489640 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:21.489617    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-5637a7-dockercfg-6mrww\""
Oct 30 22:24:21.539793 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:21.537168    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4009806a-5d15-4224-9b63-e3bc115f35ad-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-5637a7-6dfc655fbc-ms5sb\" (UID: \"4009806a-5d15-4224-9b63-e3bc115f35ad\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-5637a7-6dfc655fbc-ms5sb"
Oct 30 22:24:21.541552 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:21.541433    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6mdh4\" (UniqueName: \"kubernetes.io/projected/4009806a-5d15-4224-9b63-e3bc115f35ad-kube-api-access-6mdh4\") pod \"payload-pre-processing-e2e-inf-a-5637a7-6dfc655fbc-ms5sb\" (UID: \"4009806a-5d15-4224-9b63-e3bc115f35ad\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-5637a7-6dfc655fbc-ms5sb"
Oct 30 22:24:21.542492 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:21.541289    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4009806a-5d15-4224-9b63-e3bc115f35ad-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-5637a7-6dfc655fbc-ms5sb\" (UID: \"4009806a-5d15-4224-9b63-e3bc115f35ad\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-5637a7-6dfc655fbc-ms5sb"
Oct 30 22:24:21.715728 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:21.707418    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6mdh4\" (UniqueName: \"kubernetes.io/projected/4009806a-5d15-4224-9b63-e3bc115f35ad-kube-api-access-6mdh4\") pod \"payload-pre-processing-e2e-inf-a-5637a7-6dfc655fbc-ms5sb\" (UID: \"4009806a-5d15-4224-9b63-e3bc115f35ad\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-5637a7-6dfc655fbc-ms5sb"
Oct 30 22:24:21.721825 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:21.721789    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-5637a7-6dfc655fbc-ms5sb"
Oct 30 22:24:21.765726 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:21.764828    5510 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-5637a7-6db64b9ddc-lqdjr"]
Oct 30 22:24:21.770736 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:21.770131    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-5637a7-6db64b9ddc-lqdjr"
Oct 30 22:24:21.770736 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:21.770466    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-5637a7-6db64b9ddc-lqdjr"]
Oct 30 22:24:21.925171 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:21.925132    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2c98b4d1-9cc0-4482-af1e-1038eaeb1e61-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-5637a7-6db64b9ddc-lqdjr\" (UID: \"2c98b4d1-9cc0-4482-af1e-1038eaeb1e61\") " pod="openshift-ingress/payload-processing-e2e-inf-a-5637a7-6db64b9ddc-lqdjr"
Oct 30 22:24:21.925378 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:21.925350    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cngq7\" (UniqueName: \"kubernetes.io/projected/2c98b4d1-9cc0-4482-af1e-1038eaeb1e61-kube-api-access-cngq7\") pod \"payload-processing-e2e-inf-a-5637a7-6db64b9ddc-lqdjr\" (UID: \"2c98b4d1-9cc0-4482-af1e-1038eaeb1e61\") " pod="openshift-ingress/payload-processing-e2e-inf-a-5637a7-6db64b9ddc-lqdjr"
Oct 30 22:24:22.027453 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:22.027418    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2c98b4d1-9cc0-4482-af1e-1038eaeb1e61-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-5637a7-6db64b9ddc-lqdjr\" (UID: \"2c98b4d1-9cc0-4482-af1e-1038eaeb1e61\") " pod="openshift-ingress/payload-processing-e2e-inf-a-5637a7-6db64b9ddc-lqdjr"
Oct 30 22:24:22.027634 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:22.027589    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cngq7\" (UniqueName: \"kubernetes.io/projected/2c98b4d1-9cc0-4482-af1e-1038eaeb1e61-kube-api-access-cngq7\") pod \"payload-processing-e2e-inf-a-5637a7-6db64b9ddc-lqdjr\" (UID: \"2c98b4d1-9cc0-4482-af1e-1038eaeb1e61\") " pod="openshift-ingress/payload-processing-e2e-inf-a-5637a7-6db64b9ddc-lqdjr"
Oct 30 22:24:22.028218 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:22.028194    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2c98b4d1-9cc0-4482-af1e-1038eaeb1e61-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-5637a7-6db64b9ddc-lqdjr\" (UID: \"2c98b4d1-9cc0-4482-af1e-1038eaeb1e61\") " pod="openshift-ingress/payload-processing-e2e-inf-a-5637a7-6db64b9ddc-lqdjr"
Oct 30 22:24:22.086525 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:22.086492    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cngq7\" (UniqueName: \"kubernetes.io/projected/2c98b4d1-9cc0-4482-af1e-1038eaeb1e61-kube-api-access-cngq7\") pod \"payload-processing-e2e-inf-a-5637a7-6db64b9ddc-lqdjr\" (UID: \"2c98b4d1-9cc0-4482-af1e-1038eaeb1e61\") " pod="openshift-ingress/payload-processing-e2e-inf-a-5637a7-6db64b9ddc-lqdjr"
Oct 30 22:24:22.109496 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:22.109463    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-5637a7-6db64b9ddc-lqdjr"
Oct 30 22:24:22.150019 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:22.149993    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-5637a7-6dfc655fbc-ms5sb"]
Oct 30 22:24:22.152398 ip-10-0-136-210 kubenswrapper[5510]: W1030 22:24:22.152370    5510 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4009806a_5d15_4224_9b63_e3bc115f35ad.slice/crio-5fe8665ad3c776e037a95fba04166e8175c2a339369438ad18433cf63074424e WatchSource:0}: Error finding container 5fe8665ad3c776e037a95fba04166e8175c2a339369438ad18433cf63074424e: Status 404 returned error can't find the container with id 5fe8665ad3c776e037a95fba04166e8175c2a339369438ad18433cf63074424e
Oct 30 22:24:22.241276 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:22.241239    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-5637a7-6dfc655fbc-ms5sb" event={"ID":"4009806a-5d15-4224-9b63-e3bc115f35ad","Type":"ContainerStarted","Data":"5fe8665ad3c776e037a95fba04166e8175c2a339369438ad18433cf63074424e"}
Oct 30 22:24:22.493602 ip-10-0-136-210 kubenswrapper[5510]: W1030 22:24:22.493566    5510 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c98b4d1_9cc0_4482_af1e_1038eaeb1e61.slice/crio-bd976e5f80650973cd3f87e38aca07798de3d8a5c2abf6198dd3a4731effac03 WatchSource:0}: Error finding container bd976e5f80650973cd3f87e38aca07798de3d8a5c2abf6198dd3a4731effac03: Status 404 returned error can't find the container with id bd976e5f80650973cd3f87e38aca07798de3d8a5c2abf6198dd3a4731effac03
Oct 30 22:24:22.494141 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:22.494116    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-5637a7-6db64b9ddc-lqdjr"]
Oct 30 22:24:22.891704 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:22.891635    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:24:23.139606 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:23.139566    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:24:23.250368 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:23.250334    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-5637a7-6dfc655fbc-ms5sb" event={"ID":"4009806a-5d15-4224-9b63-e3bc115f35ad","Type":"ContainerStarted","Data":"7df82eb9f5f8346aaf90acc553463487049f56e37957a84fdf69d2a6fc69ba9d"}
Oct 30 22:24:23.250584 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:23.250566    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-5637a7-6dfc655fbc-ms5sb"
Oct 30 22:24:23.252114 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:23.252086    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-5637a7-6db64b9ddc-lqdjr" event={"ID":"2c98b4d1-9cc0-4482-af1e-1038eaeb1e61","Type":"ContainerStarted","Data":"82f0cc65e4a9cfe93229b93978d1daad54feaea2e2ff58a2e7e97a25977284bc"}
Oct 30 22:24:23.252240 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:23.252141    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-5637a7-6db64b9ddc-lqdjr" event={"ID":"2c98b4d1-9cc0-4482-af1e-1038eaeb1e61","Type":"ContainerStarted","Data":"bd976e5f80650973cd3f87e38aca07798de3d8a5c2abf6198dd3a4731effac03"}
Oct 30 22:24:23.252240 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:23.252173    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-a-5637a7-6db64b9ddc-lqdjr"
Oct 30 22:24:23.563705 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:23.553941    5510 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-a-5637a7-6db64b9ddc-lqdjr" podStartSLOduration=2.553919049 podStartE2EDuration="2.553919049s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:24:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:24:23.547131994 +0000 UTC m=+2044.107942059" watchObservedRunningTime="2026-10-30 22:24:23.553919049 +0000 UTC m=+2044.114729054"
Oct 30 22:24:23.574763 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:23.572824    5510 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-5637a7-6dfc655fbc-ms5sb" podStartSLOduration=2.572802645 podStartE2EDuration="2.572802645s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:24:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:24:23.414939206 +0000 UTC m=+2043.975749211" watchObservedRunningTime="2026-10-30 22:24:23.572802645 +0000 UTC m=+2044.133612651"
Oct 30 22:24:24.438955 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:24.438915    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:24:26.079205 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:26.079173    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:24:26.322699 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:26.292093    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:24:26.483753 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:26.482968    5510 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-9f3ea1-6cf88f97f5-24dql"]
Oct 30 22:24:26.509846 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:26.497817    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-9f3ea1-6cf88f97f5-24dql"
Oct 30 22:24:26.551679 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:26.548851    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-9f3ea1-6cf88f97f5-24dql"]
Oct 30 22:24:26.579591 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:26.577385    5510 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w2-9f3ea1\""
Oct 30 22:24:26.579591 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:26.577439    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w2-9f3ea1-dockercfg-pxdf8\""
Oct 30 22:24:26.601678 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:26.601201    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k26cf\" (UniqueName: \"kubernetes.io/projected/0796eecd-5044-4c1b-981b-14e8a16c4a9c-kube-api-access-k26cf\") pod \"payload-pre-processing-e2e-shared-b-w2-9f3ea1-6cf88f97f5-24dql\" (UID: \"0796eecd-5044-4c1b-981b-14e8a16c4a9c\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-9f3ea1-6cf88f97f5-24dql"
Oct 30 22:24:26.601678 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:26.601334    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0796eecd-5044-4c1b-981b-14e8a16c4a9c-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-9f3ea1-6cf88f97f5-24dql\" (UID: \"0796eecd-5044-4c1b-981b-14e8a16c4a9c\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-9f3ea1-6cf88f97f5-24dql"
Oct 30 22:24:26.723681 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:26.721758    5510 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-9f3ea1-67cc67f7bf-sw4km"]
Oct 30 22:24:26.727627 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:26.727584    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k26cf\" (UniqueName: \"kubernetes.io/projected/0796eecd-5044-4c1b-981b-14e8a16c4a9c-kube-api-access-k26cf\") pod \"payload-pre-processing-e2e-shared-b-w2-9f3ea1-6cf88f97f5-24dql\" (UID: \"0796eecd-5044-4c1b-981b-14e8a16c4a9c\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-9f3ea1-6cf88f97f5-24dql"
Oct 30 22:24:26.728042 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:26.728018    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0796eecd-5044-4c1b-981b-14e8a16c4a9c-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-9f3ea1-6cf88f97f5-24dql\" (UID: \"0796eecd-5044-4c1b-981b-14e8a16c4a9c\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-9f3ea1-6cf88f97f5-24dql"
Oct 30 22:24:26.746037 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:26.743580    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0796eecd-5044-4c1b-981b-14e8a16c4a9c-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-9f3ea1-6cf88f97f5-24dql\" (UID: \"0796eecd-5044-4c1b-981b-14e8a16c4a9c\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-9f3ea1-6cf88f97f5-24dql"
Oct 30 22:24:26.850909 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:26.850371    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k26cf\" (UniqueName: \"kubernetes.io/projected/0796eecd-5044-4c1b-981b-14e8a16c4a9c-kube-api-access-k26cf\") pod \"payload-pre-processing-e2e-shared-b-w2-9f3ea1-6cf88f97f5-24dql\" (UID: \"0796eecd-5044-4c1b-981b-14e8a16c4a9c\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-9f3ea1-6cf88f97f5-24dql"
Oct 30 22:24:26.852432 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:26.852401    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-9f3ea1-67cc67f7bf-sw4km"
Oct 30 22:24:26.858344 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:26.858311    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-9f3ea1-67cc67f7bf-sw4km"]
Oct 30 22:24:26.950088 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:26.950055    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ecfca1f1-d4b8-421b-ac73-3b726a8afdf4-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-9f3ea1-67cc67f7bf-sw4km\" (UID: \"ecfca1f1-d4b8-421b-ac73-3b726a8afdf4\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-9f3ea1-67cc67f7bf-sw4km"
Oct 30 22:24:26.950287 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:26.950103    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mb42j\" (UniqueName: \"kubernetes.io/projected/ecfca1f1-d4b8-421b-ac73-3b726a8afdf4-kube-api-access-mb42j\") pod \"payload-processing-e2e-shared-b-w2-9f3ea1-67cc67f7bf-sw4km\" (UID: \"ecfca1f1-d4b8-421b-ac73-3b726a8afdf4\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-9f3ea1-67cc67f7bf-sw4km"
Oct 30 22:24:27.051910 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:27.051875    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ecfca1f1-d4b8-421b-ac73-3b726a8afdf4-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-9f3ea1-67cc67f7bf-sw4km\" (UID: \"ecfca1f1-d4b8-421b-ac73-3b726a8afdf4\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-9f3ea1-67cc67f7bf-sw4km"
Oct 30 22:24:27.052093 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:27.051939    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mb42j\" (UniqueName: \"kubernetes.io/projected/ecfca1f1-d4b8-421b-ac73-3b726a8afdf4-kube-api-access-mb42j\") pod \"payload-processing-e2e-shared-b-w2-9f3ea1-67cc67f7bf-sw4km\" (UID: \"ecfca1f1-d4b8-421b-ac73-3b726a8afdf4\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-9f3ea1-67cc67f7bf-sw4km"
Oct 30 22:24:27.052800 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:27.052776    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ecfca1f1-d4b8-421b-ac73-3b726a8afdf4-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-9f3ea1-67cc67f7bf-sw4km\" (UID: \"ecfca1f1-d4b8-421b-ac73-3b726a8afdf4\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-9f3ea1-67cc67f7bf-sw4km"
Oct 30 22:24:27.110495 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:27.110452    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mb42j\" (UniqueName: \"kubernetes.io/projected/ecfca1f1-d4b8-421b-ac73-3b726a8afdf4-kube-api-access-mb42j\") pod \"payload-processing-e2e-shared-b-w2-9f3ea1-67cc67f7bf-sw4km\" (UID: \"ecfca1f1-d4b8-421b-ac73-3b726a8afdf4\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-9f3ea1-67cc67f7bf-sw4km"
Oct 30 22:24:27.145393 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:27.145351    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-9f3ea1-6cf88f97f5-24dql"
Oct 30 22:24:27.187695 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:27.182475    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-9f3ea1-67cc67f7bf-sw4km"
Oct 30 22:24:27.529421 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:27.529390    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-9f3ea1-6cf88f97f5-24dql"]
Oct 30 22:24:27.529692 ip-10-0-136-210 kubenswrapper[5510]: W1030 22:24:27.529650    5510 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0796eecd_5044_4c1b_981b_14e8a16c4a9c.slice/crio-fbd1f491530f559d5b2c0890b1cb0ce9266866b924a73655923b242a8b0956d4 WatchSource:0}: Error finding container fbd1f491530f559d5b2c0890b1cb0ce9266866b924a73655923b242a8b0956d4: Status 404 returned error can't find the container with id fbd1f491530f559d5b2c0890b1cb0ce9266866b924a73655923b242a8b0956d4
Oct 30 22:24:27.589517 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:27.588764    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-9f3ea1-67cc67f7bf-sw4km"]
Oct 30 22:24:27.593824 ip-10-0-136-210 kubenswrapper[5510]: W1030 22:24:27.593797    5510 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podecfca1f1_d4b8_421b_ac73_3b726a8afdf4.slice/crio-b997fe48c474ab3b126bdb8895958ada492565cdf93170c75e6a57b51c91795e WatchSource:0}: Error finding container b997fe48c474ab3b126bdb8895958ada492565cdf93170c75e6a57b51c91795e: Status 404 returned error can't find the container with id b997fe48c474ab3b126bdb8895958ada492565cdf93170c75e6a57b51c91795e
Oct 30 22:24:28.383967 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:28.383930    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-9f3ea1-6cf88f97f5-24dql" event={"ID":"0796eecd-5044-4c1b-981b-14e8a16c4a9c","Type":"ContainerStarted","Data":"2ec718b390c6bdc61ef3cb7ff35497233b820dc4fd94553ce7a84f1524c5a5d6"}
Oct 30 22:24:28.384454 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:28.383984    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-9f3ea1-6cf88f97f5-24dql" event={"ID":"0796eecd-5044-4c1b-981b-14e8a16c4a9c","Type":"ContainerStarted","Data":"fbd1f491530f559d5b2c0890b1cb0ce9266866b924a73655923b242a8b0956d4"}
Oct 30 22:24:28.384454 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:28.384033    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-9f3ea1-6cf88f97f5-24dql"
Oct 30 22:24:28.385702 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:28.385649    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-9f3ea1-67cc67f7bf-sw4km" event={"ID":"ecfca1f1-d4b8-421b-ac73-3b726a8afdf4","Type":"ContainerStarted","Data":"248a30a993fbee3b65eb019dcb18e4f644a9dac99ea09596d223a69f700d415d"}
Oct 30 22:24:28.385826 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:28.385723    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-9f3ea1-67cc67f7bf-sw4km" event={"ID":"ecfca1f1-d4b8-421b-ac73-3b726a8afdf4","Type":"ContainerStarted","Data":"b997fe48c474ab3b126bdb8895958ada492565cdf93170c75e6a57b51c91795e"}
Oct 30 22:24:28.385826 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:28.385746    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-9f3ea1-67cc67f7bf-sw4km"
Oct 30 22:24:28.497745 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:28.497686    5510 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-9f3ea1-6cf88f97f5-24dql" podStartSLOduration=2.497635287 podStartE2EDuration="2.497635287s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:24:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:24:28.494724318 +0000 UTC m=+2049.055534343" watchObservedRunningTime="2026-10-30 22:24:28.497635287 +0000 UTC m=+2049.058445292"
Oct 30 22:24:28.610680 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:28.607313    5510 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-9f3ea1-67cc67f7bf-sw4km" podStartSLOduration=2.607291873 podStartE2EDuration="2.607291873s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:24:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:24:28.606110958 +0000 UTC m=+2049.166920992" watchObservedRunningTime="2026-10-30 22:24:28.607291873 +0000 UTC m=+2049.168101879"
Oct 30 22:24:29.383633 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:29.379308    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:24:29.955528 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:29.955494    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:24:34.287680 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:34.286119    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-5637a7-6dfc655fbc-ms5sb"
Oct 30 22:24:34.293680 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:34.290092    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-a-5637a7-6db64b9ddc-lqdjr"
Oct 30 22:24:34.340679 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:34.340190    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-fb0b93/distinct-w6-fb0b93-kserve-58db6b6757-6mww8" podUID="8ee6579b-acea-4492-862b-9883f70d00c9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:24:34.451876 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:34.448814    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-7bb1c2/distinct-2-w3-7bb1c2-kserve-c549df844-zt595" podUID="1916e87c-a1a4-4967-8640-8bbcd9d12903" containerName="main" probeResult="failure" output="Get \"https://10.133.0.73:8000/ready\": context deadline exceeded"
Oct 30 22:24:34.692809 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:34.690244    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-779f32/distinct-w0-779f32-kserve-6dcbc948c7-2vzdf" podUID="e17cedd4-7d48-42b3-b127-b6192c1b3795" containerName="main" probeResult="failure" output="Get \"https://10.133.0.65:8000/ready\": context deadline exceeded"
Oct 30 22:24:34.819806 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:34.816549    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:24:35.079693 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:35.077843    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-779f32/distinct-2-w0-779f32-kserve-857ffdf778-gk2h9" podUID="ef500cdd-cc52-45b0-8dd7-1896897acc46" containerName="main" probeResult="failure" output="Get \"https://10.133.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:24:35.264680 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:35.258781    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-bc714d/distinct-w2-bc714d-kserve-66cfb968dd-jvgqj" podUID="9e9b11b8-1cc9-4fc8-8097-1afcffa8c422" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:24:35.361741 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:35.361207    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-fb0b93/distinct-w6-fb0b93-kserve-58db6b6757-6mww8" podUID="8ee6579b-acea-4492-862b-9883f70d00c9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:24:35.362245 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:35.361765    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-fb0b93/distinct-w6-fb0b93-kserve-58db6b6757-6mww8"
Oct 30 22:24:35.467692 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:35.465780    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-7bb1c2/distinct-2-w3-7bb1c2-kserve-c549df844-zt595" podUID="1916e87c-a1a4-4967-8640-8bbcd9d12903" containerName="main" probeResult="failure" output="Get \"https://10.133.0.73:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:24:35.467692 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:35.467165    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-7bb1c2/distinct-2-w3-7bb1c2-kserve-c549df844-zt595"
Oct 30 22:24:35.701740 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:35.700821    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-c9dbe6/distinct-w5-c9dbe6-kserve-7599fd86d9-bdhxd" podUID="e903db90-f2cd-4244-b00b-f99b21873f6e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.67:8000/ready\": context deadline exceeded"
Oct 30 22:24:35.726682 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:35.706823    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-779f32/distinct-w0-779f32-kserve-6dcbc948c7-2vzdf" podUID="e17cedd4-7d48-42b3-b127-b6192c1b3795" containerName="main" probeResult="failure" output="Get \"https://10.133.0.65:8000/ready\": context deadline exceeded"
Oct 30 22:24:35.754677 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:35.754049    5510 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-bc714d/distinct-w2-bc714d-kserve-66cfb968dd-jvgqj" podUID="9e9b11b8-1cc9-4fc8-8097-1afcffa8c422" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:24:36.197858 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:36.197801    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-779f32/distinct-2-w0-779f32-kserve-857ffdf778-gk2h9" podUID="ef500cdd-cc52-45b0-8dd7-1896897acc46" containerName="main" probeResult="failure" output="Get \"https://10.133.0.74:8000/ready\": context deadline exceeded"
Oct 30 22:24:36.204054 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:36.204014    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-fb0b93/premium-w6-fb0b93-kserve-5cb9948c54-w7pnk" podUID="290ae7a2-de37-4af3-82ae-a8e855b83bb9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": context deadline exceeded"
Oct 30 22:24:36.341682 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:36.314485    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-bc714d/distinct-w2-bc714d-kserve-66cfb968dd-jvgqj" podUID="9e9b11b8-1cc9-4fc8-8097-1afcffa8c422" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:24:36.370717 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:36.366778    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-fb0b93/distinct-w6-fb0b93-kserve-58db6b6757-6mww8" podUID="8ee6579b-acea-4492-862b-9883f70d00c9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:24:36.474948 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:36.474848    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-7bb1c2/distinct-2-w3-7bb1c2-kserve-c549df844-zt595" podUID="1916e87c-a1a4-4967-8640-8bbcd9d12903" containerName="main" probeResult="failure" output="Get \"https://10.133.0.73:8000/ready\": context deadline exceeded"
Oct 30 22:24:36.563687 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:36.561780    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-779f32/distinct-w0-779f32-kserve-6dcbc948c7-2vzdf"
Oct 30 22:24:36.563687 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:36.561926    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-779f32/distinct-2-w0-779f32-kserve-857ffdf778-gk2h9"
Oct 30 22:24:36.563687 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:36.561972    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-bc714d/distinct-w2-bc714d-kserve-66cfb968dd-jvgqj"
Oct 30 22:24:36.727736 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:36.721759    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-c9dbe6/distinct-w5-c9dbe6-kserve-7599fd86d9-bdhxd" podUID="e903db90-f2cd-4244-b00b-f99b21873f6e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:24:36.733945 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:36.733917    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-c9dbe6/distinct-w5-c9dbe6-kserve-7599fd86d9-bdhxd"
Oct 30 22:24:36.735152 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:36.735047    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-779f32/distinct-w0-779f32-kserve-6dcbc948c7-2vzdf" podUID="e17cedd4-7d48-42b3-b127-b6192c1b3795" containerName="main" probeResult="failure" output="Get \"https://10.133.0.65:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:24:37.235682 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:37.234043    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-779f32/distinct-2-w0-779f32-kserve-857ffdf778-gk2h9" podUID="ef500cdd-cc52-45b0-8dd7-1896897acc46" containerName="main" probeResult="failure" output="Get \"https://10.133.0.74:8000/ready\": context deadline exceeded"
Oct 30 22:24:37.278678 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:37.277765    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-fb0b93/premium-w6-fb0b93-kserve-5cb9948c54-w7pnk" podUID="290ae7a2-de37-4af3-82ae-a8e855b83bb9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:24:37.286438 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:37.286410    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-fb0b93/premium-w6-fb0b93-kserve-5cb9948c54-w7pnk"
Oct 30 22:24:37.367702 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:37.341069    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-bc714d/distinct-w2-bc714d-kserve-66cfb968dd-jvgqj" podUID="9e9b11b8-1cc9-4fc8-8097-1afcffa8c422" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:24:37.367925 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:37.367888    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-fb0b93/distinct-w6-fb0b93-kserve-58db6b6757-6mww8" podUID="8ee6579b-acea-4492-862b-9883f70d00c9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.59:8000/ready\": context deadline exceeded"
Oct 30 22:24:37.497737 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:37.479010    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-7bb1c2/distinct-2-w3-7bb1c2-kserve-c549df844-zt595" podUID="1916e87c-a1a4-4967-8640-8bbcd9d12903" containerName="main" probeResult="failure" output="Get \"https://10.133.0.73:8000/ready\": context deadline exceeded"
Oct 30 22:24:37.721294 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:37.721256    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-fb0b93/distinct-w6-fb0b93-kserve-58db6b6757-6mww8"
Oct 30 22:24:37.721507 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:37.721313    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-c9dbe6/distinct-w5-c9dbe6-kserve-7599fd86d9-bdhxd"
Oct 30 22:24:37.721507 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:37.721347    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-779f32/distinct-w0-779f32-kserve-6dcbc948c7-2vzdf"
Oct 30 22:24:37.721645 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:37.721547    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-779f32/distinct-2-w0-779f32-kserve-857ffdf778-gk2h9"
Oct 30 22:24:37.721645 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:37.721580    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-bc714d/distinct-w2-bc714d-kserve-66cfb968dd-jvgqj"
Oct 30 22:24:37.721645 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:37.721601    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-7bb1c2/distinct-2-w3-7bb1c2-kserve-c549df844-zt595"
Oct 30 22:24:37.721645 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:37.721623    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-fb0b93/premium-w6-fb0b93-kserve-5cb9948c54-w7pnk"
Oct 30 22:24:38.296735 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:38.296698    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:24:38.651735 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:24:38.644554    5510 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6gjbf" containerName="extractor"
Oct 30 22:24:39.198955 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:39.198922    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:24:39.447471 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:39.447437    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-9f3ea1-67cc67f7bf-sw4km"
Oct 30 22:24:39.448578 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:39.448554    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-9f3ea1-6cf88f97f5-24dql"
Oct 30 22:24:40.526681 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:40.515816    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:24:40.722049 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:40.721974    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:24:40.932456 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:40.932377    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:24:43.760530 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:24:43.760490    5510 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/system.slice/system-systemd\\\\x2dcoredump.slice/systemd-coredump@11-28397-0.service\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4152cef_f9cf_46c1_a9c4_78484ab58efe.slice/crio-e97bc2ef207378d6ceb74ad4321a4a3e7a692e93e16ed58c50e1d900ac569c18.scope\": RecentStats: unable to find data in memory cache]"
Oct 30 22:24:44.173488 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:44.173398    5510 generic.go:356] "Generic (PLEG): container finished" podID="dcbe788e-cc5c-4807-af27-b98a507ea810" containerID="d7f8bddcd52f0750ee71642815306a33677a593752edf6eb4fdc0e6bdbe20a33" exitCode=0
Oct 30 22:24:44.173488 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:44.173471    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w1-943d8f-openshift-default-6c57fb7747-z6x89" event={"ID":"dcbe788e-cc5c-4807-af27-b98a507ea810","Type":"ContainerDied","Data":"d7f8bddcd52f0750ee71642815306a33677a593752edf6eb4fdc0e6bdbe20a33"}
Oct 30 22:24:44.174059 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:44.174041    5510 scope.go:122] "RemoveContainer" containerID="d7f8bddcd52f0750ee71642815306a33677a593752edf6eb4fdc0e6bdbe20a33"
Oct 30 22:24:44.175530 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:44.175507    5510 generic.go:356] "Generic (PLEG): container finished" podID="c4152cef-f9cf-46c1-a9c4-78484ab58efe" containerID="e97bc2ef207378d6ceb74ad4321a4a3e7a692e93e16ed58c50e1d900ac569c18" exitCode=0
Oct 30 22:24:44.175623 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:44.175565    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w5-99304b-openshift-default-665c757b9-6t2xl" event={"ID":"c4152cef-f9cf-46c1-a9c4-78484ab58efe","Type":"ContainerDied","Data":"e97bc2ef207378d6ceb74ad4321a4a3e7a692e93e16ed58c50e1d900ac569c18"}
Oct 30 22:24:44.175961 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:44.175948    5510 scope.go:122] "RemoveContainer" containerID="e97bc2ef207378d6ceb74ad4321a4a3e7a692e93e16ed58c50e1d900ac569c18"
Oct 30 22:24:44.176092 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:44.176062    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:24:44.176152 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:44.176138    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:24:44.176194 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:44.176178    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:24:44.177522 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:44.177493    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:24:44.177615 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:44.177554    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:24:44.177615 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:44.177588    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:24:45.078680 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:45.078036    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:24:45.201214 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:45.201146    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w1-943d8f-openshift-default-6c57fb7747-z6x89" event={"ID":"dcbe788e-cc5c-4807-af27-b98a507ea810","Type":"ContainerStarted","Data":"eb7566c34b76578a2c3664993b4ec0c90a0b953ce9f17a85614c79f8ee92657b"}
Oct 30 22:24:45.204548 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:45.204513    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w5-99304b-openshift-default-665c757b9-6t2xl" event={"ID":"c4152cef-f9cf-46c1-a9c4-78484ab58efe","Type":"ContainerStarted","Data":"f4221738b50279b211037184bfdeb5fb11332f527e5d64ebfb3deb31bc033520"}
Oct 30 22:24:45.362743 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:45.354564    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:24:45.606903 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:45.606868    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w1-943d8f-openshift-default-6c57fb7747-z6x89"
Oct 30 22:24:45.608291 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:45.608267    5510 patch_prober.go:29] interesting pod/e2e-shared-b-w1-943d8f-openshift-default-6c57fb7747-z6x89 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.127:15021/healthz/ready\": dial tcp 10.133.0.127:15021: connect: connection refused" start-of-body=
Oct 30 22:24:45.608351 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:45.608318    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w1-943d8f-openshift-default-6c57fb7747-z6x89" podUID="dcbe788e-cc5c-4807-af27-b98a507ea810" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.127:15021/healthz/ready\": dial tcp 10.133.0.127:15021: connect: connection refused"
Oct 30 22:24:45.923682 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:45.923618    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w5-99304b-openshift-default-665c757b9-6t2xl"
Oct 30 22:24:45.923935 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:45.923795    5510 patch_prober.go:29] interesting pod/e2e-shared-b-w5-99304b-openshift-default-665c757b9-6t2xl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.124:15021/healthz/ready\": dial tcp 10.133.0.124:15021: connect: connection refused" start-of-body=
Oct 30 22:24:45.923935 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:45.923841    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w5-99304b-openshift-default-665c757b9-6t2xl" podUID="c4152cef-f9cf-46c1-a9c4-78484ab58efe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.124:15021/healthz/ready\": dial tcp 10.133.0.124:15021: connect: connection refused"
Oct 30 22:24:45.979680 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:45.978770    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:24:46.607890 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:46.607852    5510 patch_prober.go:29] interesting pod/e2e-shared-b-w1-943d8f-openshift-default-6c57fb7747-z6x89 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.127:15021/healthz/ready\": dial tcp 10.133.0.127:15021: connect: connection refused" start-of-body=
Oct 30 22:24:46.608399 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:46.607912    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w1-943d8f-openshift-default-6c57fb7747-z6x89" podUID="dcbe788e-cc5c-4807-af27-b98a507ea810" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.127:15021/healthz/ready\": dial tcp 10.133.0.127:15021: connect: connection refused"
Oct 30 22:24:46.920273 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:46.920196    5510 patch_prober.go:29] interesting pod/e2e-shared-b-w5-99304b-openshift-default-665c757b9-6t2xl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.124:15021/healthz/ready\": dial tcp 10.133.0.124:15021: connect: connection refused" start-of-body=
Oct 30 22:24:46.920273 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:46.920247    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w5-99304b-openshift-default-665c757b9-6t2xl" podUID="c4152cef-f9cf-46c1-a9c4-78484ab58efe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.124:15021/healthz/ready\": dial tcp 10.133.0.124:15021: connect: connection refused"
Oct 30 22:24:47.608131 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:47.608093    5510 patch_prober.go:29] interesting pod/e2e-shared-b-w1-943d8f-openshift-default-6c57fb7747-z6x89 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.127:15021/healthz/ready\": dial tcp 10.133.0.127:15021: connect: connection refused" start-of-body=
Oct 30 22:24:47.608531 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:47.608150    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w1-943d8f-openshift-default-6c57fb7747-z6x89" podUID="dcbe788e-cc5c-4807-af27-b98a507ea810" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.127:15021/healthz/ready\": dial tcp 10.133.0.127:15021: connect: connection refused"
Oct 30 22:24:47.919967 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:47.919882    5510 patch_prober.go:29] interesting pod/e2e-shared-b-w5-99304b-openshift-default-665c757b9-6t2xl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.124:15021/healthz/ready\": dial tcp 10.133.0.124:15021: connect: connection refused" start-of-body=
Oct 30 22:24:47.919967 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:47.919932    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w5-99304b-openshift-default-665c757b9-6t2xl" podUID="c4152cef-f9cf-46c1-a9c4-78484ab58efe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.124:15021/healthz/ready\": dial tcp 10.133.0.124:15021: connect: connection refused"
Oct 30 22:24:48.572697 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:48.571361    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:24:48.621692 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:48.620120    5510 patch_prober.go:29] interesting pod/e2e-shared-b-w1-943d8f-openshift-default-6c57fb7747-z6x89 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.127:15021/healthz/ready\": dial tcp 10.133.0.127:15021: connect: connection refused" start-of-body=
Oct 30 22:24:48.621692 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:48.620185    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w1-943d8f-openshift-default-6c57fb7747-z6x89" podUID="dcbe788e-cc5c-4807-af27-b98a507ea810" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.127:15021/healthz/ready\": dial tcp 10.133.0.127:15021: connect: connection refused"
Oct 30 22:24:48.931229 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:48.931129    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:24:48.931936 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:48.931907    5510 patch_prober.go:29] interesting pod/e2e-shared-b-w5-99304b-openshift-default-665c757b9-6t2xl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.124:15021/healthz/ready\": dial tcp 10.133.0.124:15021: connect: connection refused" start-of-body=
Oct 30 22:24:48.932164 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:48.932139    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w5-99304b-openshift-default-665c757b9-6t2xl" podUID="c4152cef-f9cf-46c1-a9c4-78484ab58efe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.124:15021/healthz/ready\": dial tcp 10.133.0.124:15021: connect: connection refused"
Oct 30 22:24:49.024757 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:49.024714    5510 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-5686cb-6c9b4b9b7f-pt4gf"]
Oct 30 22:24:49.631227 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:49.631179    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-779f32/distinct-2-w0-779f32-kserve-857ffdf778-gk2h9" podUID="ef500cdd-cc52-45b0-8dd7-1896897acc46" containerName="main" probeResult="failure" output="Get \"https://10.133.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:24:49.647692 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:49.631966    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-c9dbe6/distinct-w5-c9dbe6-kserve-7599fd86d9-bdhxd" podUID="e903db90-f2cd-4244-b00b-f99b21873f6e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:24:49.647692 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:49.639243    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-5686cb-6c9b4b9b7f-pt4gf"
Oct 30 22:24:49.672681 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:49.663910    5510 patch_prober.go:29] interesting pod/e2e-shared-b-w1-943d8f-openshift-default-6c57fb7747-z6x89 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.127:15021/healthz/ready\": dial tcp 10.133.0.127:15021: connect: connection refused" start-of-body=
Oct 30 22:24:49.672681 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:49.663990    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w1-943d8f-openshift-default-6c57fb7747-z6x89" podUID="dcbe788e-cc5c-4807-af27-b98a507ea810" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.127:15021/healthz/ready\": dial tcp 10.133.0.127:15021: connect: connection refused"
Oct 30 22:24:49.707229 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:49.706824    5510 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-5686cb\""
Oct 30 22:24:49.714555 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:49.714528    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-5686cb-6c9b4b9b7f-pt4gf"]
Oct 30 22:24:49.714705 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:49.714582    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w1-943d8f-openshift-default-6c57fb7747-z6x89"
Oct 30 22:24:49.714705 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:49.714606    5510 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-5686cb-7b8659769-g92j9"]
Oct 30 22:24:49.718161 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:49.718145    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-5686cb-7b8659769-g92j9"]
Oct 30 22:24:49.718228 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:49.718218    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-5686cb-7b8659769-g92j9"
Oct 30 22:24:49.737463 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:49.737442    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-5686cb-dockercfg-tqt4k\""
Oct 30 22:24:49.781462 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:49.754001    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/32e3490e-853c-4d7b-8c67-4423a2b89be3-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-5686cb-6c9b4b9b7f-pt4gf\" (UID: \"32e3490e-853c-4d7b-8c67-4423a2b89be3\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-5686cb-6c9b4b9b7f-pt4gf"
Oct 30 22:24:49.781462 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:49.766287    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7j49g\" (UniqueName: \"kubernetes.io/projected/32e3490e-853c-4d7b-8c67-4423a2b89be3-kube-api-access-7j49g\") pod \"payload-pre-processing-e2e-api-b-5686cb-6c9b4b9b7f-pt4gf\" (UID: \"32e3490e-853c-4d7b-8c67-4423a2b89be3\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-5686cb-6c9b4b9b7f-pt4gf"
Oct 30 22:24:49.867986 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:49.867936    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8zpzv\" (UniqueName: \"kubernetes.io/projected/bc1e8851-bb44-4e8d-a9da-387642b1c23c-kube-api-access-8zpzv\") pod \"payload-processing-e2e-api-b-5686cb-7b8659769-g92j9\" (UID: \"bc1e8851-bb44-4e8d-a9da-387642b1c23c\") " pod="openshift-ingress/payload-processing-e2e-api-b-5686cb-7b8659769-g92j9"
Oct 30 22:24:49.868285 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:49.868264    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/32e3490e-853c-4d7b-8c67-4423a2b89be3-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-5686cb-6c9b4b9b7f-pt4gf\" (UID: \"32e3490e-853c-4d7b-8c67-4423a2b89be3\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-5686cb-6c9b4b9b7f-pt4gf"
Oct 30 22:24:49.868537 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:49.868515    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bc1e8851-bb44-4e8d-a9da-387642b1c23c-plugins-config-volume\") pod \"payload-processing-e2e-api-b-5686cb-7b8659769-g92j9\" (UID: \"bc1e8851-bb44-4e8d-a9da-387642b1c23c\") " pod="openshift-ingress/payload-processing-e2e-api-b-5686cb-7b8659769-g92j9"
Oct 30 22:24:49.888391 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:49.888309    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7j49g\" (UniqueName: \"kubernetes.io/projected/32e3490e-853c-4d7b-8c67-4423a2b89be3-kube-api-access-7j49g\") pod \"payload-pre-processing-e2e-api-b-5686cb-6c9b4b9b7f-pt4gf\" (UID: \"32e3490e-853c-4d7b-8c67-4423a2b89be3\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-5686cb-6c9b4b9b7f-pt4gf"
Oct 30 22:24:49.889127 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:49.885385    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/32e3490e-853c-4d7b-8c67-4423a2b89be3-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-5686cb-6c9b4b9b7f-pt4gf\" (UID: \"32e3490e-853c-4d7b-8c67-4423a2b89be3\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-5686cb-6c9b4b9b7f-pt4gf"
Oct 30 22:24:49.920714 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:49.920680    5510 patch_prober.go:29] interesting pod/e2e-shared-b-w5-99304b-openshift-default-665c757b9-6t2xl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.124:15021/healthz/ready\": dial tcp 10.133.0.124:15021: connect: connection refused" start-of-body=
Oct 30 22:24:49.920882 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:49.920735    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w5-99304b-openshift-default-665c757b9-6t2xl" podUID="c4152cef-f9cf-46c1-a9c4-78484ab58efe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.124:15021/healthz/ready\": dial tcp 10.133.0.124:15021: connect: connection refused"
Oct 30 22:24:49.976454 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:49.976418    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7j49g\" (UniqueName: \"kubernetes.io/projected/32e3490e-853c-4d7b-8c67-4423a2b89be3-kube-api-access-7j49g\") pod \"payload-pre-processing-e2e-api-b-5686cb-6c9b4b9b7f-pt4gf\" (UID: \"32e3490e-853c-4d7b-8c67-4423a2b89be3\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-5686cb-6c9b4b9b7f-pt4gf"
Oct 30 22:24:49.990987 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:49.990940    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8zpzv\" (UniqueName: \"kubernetes.io/projected/bc1e8851-bb44-4e8d-a9da-387642b1c23c-kube-api-access-8zpzv\") pod \"payload-processing-e2e-api-b-5686cb-7b8659769-g92j9\" (UID: \"bc1e8851-bb44-4e8d-a9da-387642b1c23c\") " pod="openshift-ingress/payload-processing-e2e-api-b-5686cb-7b8659769-g92j9"
Oct 30 22:24:49.991907 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:49.991849    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bc1e8851-bb44-4e8d-a9da-387642b1c23c-plugins-config-volume\") pod \"payload-processing-e2e-api-b-5686cb-7b8659769-g92j9\" (UID: \"bc1e8851-bb44-4e8d-a9da-387642b1c23c\") " pod="openshift-ingress/payload-processing-e2e-api-b-5686cb-7b8659769-g92j9"
Oct 30 22:24:49.994696 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:49.992986    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bc1e8851-bb44-4e8d-a9da-387642b1c23c-plugins-config-volume\") pod \"payload-processing-e2e-api-b-5686cb-7b8659769-g92j9\" (UID: \"bc1e8851-bb44-4e8d-a9da-387642b1c23c\") " pod="openshift-ingress/payload-processing-e2e-api-b-5686cb-7b8659769-g92j9"
Oct 30 22:24:50.014819 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:50.014791    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-5686cb-6c9b4b9b7f-pt4gf"
Oct 30 22:24:50.101721 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:50.101686    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8zpzv\" (UniqueName: \"kubernetes.io/projected/bc1e8851-bb44-4e8d-a9da-387642b1c23c-kube-api-access-8zpzv\") pod \"payload-processing-e2e-api-b-5686cb-7b8659769-g92j9\" (UID: \"bc1e8851-bb44-4e8d-a9da-387642b1c23c\") " pod="openshift-ingress/payload-processing-e2e-api-b-5686cb-7b8659769-g92j9"
Oct 30 22:24:50.331394 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:50.331361    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-5686cb-7b8659769-g92j9"
Oct 30 22:24:50.501926 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:50.501774    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-5686cb-6c9b4b9b7f-pt4gf"]
Oct 30 22:24:50.608704 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:50.608215    5510 patch_prober.go:29] interesting pod/e2e-shared-b-w1-943d8f-openshift-default-6c57fb7747-z6x89 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.127:15021/healthz/ready\": dial tcp 10.133.0.127:15021: connect: connection refused" start-of-body=
Oct 30 22:24:50.608704 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:50.608277    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w1-943d8f-openshift-default-6c57fb7747-z6x89" podUID="dcbe788e-cc5c-4807-af27-b98a507ea810" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.127:15021/healthz/ready\": dial tcp 10.133.0.127:15021: connect: connection refused"
Oct 30 22:24:50.617382 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:50.617344    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-5686cb-6c9b4b9b7f-pt4gf" event={"ID":"32e3490e-853c-4d7b-8c67-4423a2b89be3","Type":"ContainerStarted","Data":"af36ef95b9da6ef97da1dba38fb05fca90062ab0f8860d107af7b4dece0d4def"}
Oct 30 22:24:50.806699 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:50.806451    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-5686cb-7b8659769-g92j9"]
Oct 30 22:24:50.807858 ip-10-0-136-210 kubenswrapper[5510]: W1030 22:24:50.807829    5510 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc1e8851_bb44_4e8d_a9da_387642b1c23c.slice/crio-fa2699b4c3c7a3577630579d61816d853e95ccbca05f67588bb702452f7c636f WatchSource:0}: Error finding container fa2699b4c3c7a3577630579d61816d853e95ccbca05f67588bb702452f7c636f: Status 404 returned error can't find the container with id fa2699b4c3c7a3577630579d61816d853e95ccbca05f67588bb702452f7c636f
Oct 30 22:24:50.867691 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:50.865917    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w5-99304b-openshift-default-665c757b9-6t2xl"
Oct 30 22:24:50.920162 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:50.920135    5510 patch_prober.go:29] interesting pod/e2e-shared-b-w5-99304b-openshift-default-665c757b9-6t2xl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.124:15021/healthz/ready\": dial tcp 10.133.0.124:15021: connect: connection refused" start-of-body=
Oct 30 22:24:50.920243 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:50.920190    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w5-99304b-openshift-default-665c757b9-6t2xl" podUID="c4152cef-f9cf-46c1-a9c4-78484ab58efe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.124:15021/healthz/ready\": dial tcp 10.133.0.124:15021: connect: connection refused"
Oct 30 22:24:51.612073 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:51.612044    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w1-943d8f-openshift-default-6c57fb7747-z6x89"
Oct 30 22:24:51.613327 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:51.613302    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w1-943d8f-openshift-default-6c57fb7747-z6x89"
Oct 30 22:24:51.629852 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:51.629818    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-5686cb-6c9b4b9b7f-pt4gf" event={"ID":"32e3490e-853c-4d7b-8c67-4423a2b89be3","Type":"ContainerStarted","Data":"6272f039d4715f19aba5aaf7a13df82dcdc8f52aa8681ded3e7aed98801ba026"}
Oct 30 22:24:51.629987 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:51.629880    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-5686cb-6c9b4b9b7f-pt4gf"
Oct 30 22:24:51.631709 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:51.631676    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-5686cb-7b8659769-g92j9" event={"ID":"bc1e8851-bb44-4e8d-a9da-387642b1c23c","Type":"ContainerStarted","Data":"d06c3b6bf9e2e69f4be3ef8b14048417b6e082f6158eaeeafb57c72f29a354f3"}
Oct 30 22:24:51.631845 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:51.631732    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-5686cb-7b8659769-g92j9" event={"ID":"bc1e8851-bb44-4e8d-a9da-387642b1c23c","Type":"ContainerStarted","Data":"fa2699b4c3c7a3577630579d61816d853e95ccbca05f67588bb702452f7c636f"}
Oct 30 22:24:51.631913 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:51.631888    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-b-5686cb-7b8659769-g92j9"
Oct 30 22:24:51.811543 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:51.811474    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:24:51.899603 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:51.899490    5510 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-b-5686cb-6c9b4b9b7f-pt4gf" podStartSLOduration=3.899475443 podStartE2EDuration="3.899475443s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:24:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:24:51.89641423 +0000 UTC m=+2072.457224235" watchObservedRunningTime="2026-10-30 22:24:51.899475443 +0000 UTC m=+2072.460285448"
Oct 30 22:24:51.925270 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:51.925239    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w5-99304b-openshift-default-665c757b9-6t2xl"
Oct 30 22:24:51.926264 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:51.926242    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w5-99304b-openshift-default-665c757b9-6t2xl"
Oct 30 22:24:52.157515 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:52.157392    5510 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-b-5686cb-7b8659769-g92j9" podStartSLOduration=3.157373862 podStartE2EDuration="3.157373862s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:24:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:24:52.156899909 +0000 UTC m=+2072.717709938" watchObservedRunningTime="2026-10-30 22:24:52.157373862 +0000 UTC m=+2072.718183869"
Oct 30 22:24:56.525684 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:56.511521    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:24:56.740080 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:24:56.740041    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:25:01.276747 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:01.276319    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:25:02.643337 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:02.643306    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-5686cb-6c9b4b9b7f-pt4gf"
Oct 30 22:25:02.643808 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:02.643556    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-b-5686cb-7b8659769-g92j9"
Oct 30 22:25:05.979607 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:05.979566    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:25:07.098638 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:07.098598    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-vtwfv"]
Oct 30 22:25:07.099097 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:07.098870    5510 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-vtwfv" podUID="00517b45-26ad-41d2-b86e-99d9092c2403" containerName="istio-proxy" containerID="cri-o://a0e12056ee7c8d043a262bdc5684d0d0f797a7ac2e31c4f8f3d351f84bc90018" gracePeriod=30
Oct 30 22:25:07.145927 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:07.145892    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-jtsvv"]
Oct 30 22:25:07.146181 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:07.146153    5510 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-jtsvv" podUID="0101f982-8c38-4576-be55-949dfc3fc2de" containerName="istio-proxy" containerID="cri-o://97e7fc3b556f7b3864d94a4ad7254ea0ff9badb4a6e1da68d91af1e3451c4a30" gracePeriod=30
Oct 30 22:25:07.153494 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:07.153462    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-j4m88"]
Oct 30 22:25:07.153992 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:07.153958    5510 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-j4m88" podUID="a089cfee-9de2-4d44-b7d9-dbbc2a66ebd4" containerName="istio-proxy" containerID="cri-o://431729942c9ae9dc292e10d5cdf7329d815661b49d6bdcd85877e365b8488573" gracePeriod=30
Oct 30 22:25:07.296877 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:07.296833    5510 patch_prober.go:29] interesting pod/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-vtwfv container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 22:25:07.297068 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:07.296938    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-vtwfv" podUID="00517b45-26ad-41d2-b86e-99d9092c2403" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 22:25:08.420551 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:25:08.420513    5510 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6gjbf" containerName="extractor"
Oct 30 22:25:08.891432 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:08.891389    5510 patch_prober.go:29] interesting pod/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-jtsvv container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 22:25:08.891693 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:08.891455    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-jtsvv" podUID="0101f982-8c38-4576-be55-949dfc3fc2de" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 22:25:10.793679 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:10.790310    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:25:11.322443 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:11.322406    5510 patch_prober.go:29] interesting pod/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-j4m88 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 22:25:11.322611 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:11.322466    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-j4m88" podUID="a089cfee-9de2-4d44-b7d9-dbbc2a66ebd4" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 22:25:12.483460 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:12.483423    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-vtwfv"
Oct 30 22:25:12.494091 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:12.494057    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-jtsvv"
Oct 30 22:25:12.545636 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:12.545609    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-j4m88"
Oct 30 22:25:12.588431 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:12.588399    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/00517b45-26ad-41d2-b86e-99d9092c2403-kube-api-access-xwkh8\" (UniqueName: \"kubernetes.io/projected/00517b45-26ad-41d2-b86e-99d9092c2403-kube-api-access-xwkh8\") pod \"00517b45-26ad-41d2-b86e-99d9092c2403\" (UID: \"00517b45-26ad-41d2-b86e-99d9092c2403\") "
Oct 30 22:25:12.588636 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:12.588456    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0101f982-8c38-4576-be55-949dfc3fc2de-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0101f982-8c38-4576-be55-949dfc3fc2de-istiod-ca-cert\") pod \"0101f982-8c38-4576-be55-949dfc3fc2de\" (UID: \"0101f982-8c38-4576-be55-949dfc3fc2de\") "
Oct 30 22:25:12.588636 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:12.588486    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/00517b45-26ad-41d2-b86e-99d9092c2403-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/00517b45-26ad-41d2-b86e-99d9092c2403-workload-certs\") pod \"00517b45-26ad-41d2-b86e-99d9092c2403\" (UID: \"00517b45-26ad-41d2-b86e-99d9092c2403\") "
Oct 30 22:25:12.588804 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:12.588694    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0101f982-8c38-4576-be55-949dfc3fc2de-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0101f982-8c38-4576-be55-949dfc3fc2de-istio-data\") pod \"0101f982-8c38-4576-be55-949dfc3fc2de\" (UID: \"0101f982-8c38-4576-be55-949dfc3fc2de\") "
Oct 30 22:25:12.588804 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:12.588726    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/00517b45-26ad-41d2-b86e-99d9092c2403-workload-certs" pod "00517b45-26ad-41d2-b86e-99d9092c2403" (UID: "00517b45-26ad-41d2-b86e-99d9092c2403"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:25:12.588804 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:12.588760    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0101f982-8c38-4576-be55-949dfc3fc2de-kube-api-access-8lhzp\" (UniqueName: \"kubernetes.io/projected/0101f982-8c38-4576-be55-949dfc3fc2de-kube-api-access-8lhzp\") pod \"0101f982-8c38-4576-be55-949dfc3fc2de\" (UID: \"0101f982-8c38-4576-be55-949dfc3fc2de\") "
Oct 30 22:25:12.588959 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:12.588832    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/00517b45-26ad-41d2-b86e-99d9092c2403-istio-token\" (UniqueName: \"kubernetes.io/projected/00517b45-26ad-41d2-b86e-99d9092c2403-istio-token\") pod \"00517b45-26ad-41d2-b86e-99d9092c2403\" (UID: \"00517b45-26ad-41d2-b86e-99d9092c2403\") "
Oct 30 22:25:12.588959 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:12.588845    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0101f982-8c38-4576-be55-949dfc3fc2de-istiod-ca-cert" pod "0101f982-8c38-4576-be55-949dfc3fc2de" (UID: "0101f982-8c38-4576-be55-949dfc3fc2de"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:25:12.588959 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:12.588894    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/00517b45-26ad-41d2-b86e-99d9092c2403-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/00517b45-26ad-41d2-b86e-99d9092c2403-istio-data\") pod \"00517b45-26ad-41d2-b86e-99d9092c2403\" (UID: \"00517b45-26ad-41d2-b86e-99d9092c2403\") "
Oct 30 22:25:12.588959 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:12.588943    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/00517b45-26ad-41d2-b86e-99d9092c2403-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/00517b45-26ad-41d2-b86e-99d9092c2403-workload-socket\") pod \"00517b45-26ad-41d2-b86e-99d9092c2403\" (UID: \"00517b45-26ad-41d2-b86e-99d9092c2403\") "
Oct 30 22:25:12.589140 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:12.588967    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0101f982-8c38-4576-be55-949dfc3fc2de-istio-data" pod "0101f982-8c38-4576-be55-949dfc3fc2de" (UID: "0101f982-8c38-4576-be55-949dfc3fc2de"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:25:12.589140 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:12.588971    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0101f982-8c38-4576-be55-949dfc3fc2de-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0101f982-8c38-4576-be55-949dfc3fc2de-workload-socket\") pod \"0101f982-8c38-4576-be55-949dfc3fc2de\" (UID: \"0101f982-8c38-4576-be55-949dfc3fc2de\") "
Oct 30 22:25:12.589140 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:12.589015    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0101f982-8c38-4576-be55-949dfc3fc2de-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0101f982-8c38-4576-be55-949dfc3fc2de-workload-certs\") pod \"0101f982-8c38-4576-be55-949dfc3fc2de\" (UID: \"0101f982-8c38-4576-be55-949dfc3fc2de\") "
Oct 30 22:25:12.589304 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:12.589169    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0101f982-8c38-4576-be55-949dfc3fc2de-workload-socket" pod "0101f982-8c38-4576-be55-949dfc3fc2de" (UID: "0101f982-8c38-4576-be55-949dfc3fc2de"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:25:12.589304 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:12.589176    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/00517b45-26ad-41d2-b86e-99d9092c2403-istio-data" pod "00517b45-26ad-41d2-b86e-99d9092c2403" (UID: "00517b45-26ad-41d2-b86e-99d9092c2403"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:25:12.589304 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:12.589207    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/00517b45-26ad-41d2-b86e-99d9092c2403-workload-socket" pod "00517b45-26ad-41d2-b86e-99d9092c2403" (UID: "00517b45-26ad-41d2-b86e-99d9092c2403"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:25:12.589304 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:12.589180    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/00517b45-26ad-41d2-b86e-99d9092c2403-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/00517b45-26ad-41d2-b86e-99d9092c2403-credential-socket\") pod \"00517b45-26ad-41d2-b86e-99d9092c2403\" (UID: \"00517b45-26ad-41d2-b86e-99d9092c2403\") "
Oct 30 22:25:12.589304 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:12.589294    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0101f982-8c38-4576-be55-949dfc3fc2de-workload-certs" pod "0101f982-8c38-4576-be55-949dfc3fc2de" (UID: "0101f982-8c38-4576-be55-949dfc3fc2de"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:25:12.589580 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:12.589316    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/00517b45-26ad-41d2-b86e-99d9092c2403-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/00517b45-26ad-41d2-b86e-99d9092c2403-istio-envoy\") pod \"00517b45-26ad-41d2-b86e-99d9092c2403\" (UID: \"00517b45-26ad-41d2-b86e-99d9092c2403\") "
Oct 30 22:25:12.589580 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:12.589362    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/00517b45-26ad-41d2-b86e-99d9092c2403-credential-socket" pod "00517b45-26ad-41d2-b86e-99d9092c2403" (UID: "00517b45-26ad-41d2-b86e-99d9092c2403"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:25:12.589580 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:12.589382    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0101f982-8c38-4576-be55-949dfc3fc2de-istio-token\" (UniqueName: \"kubernetes.io/projected/0101f982-8c38-4576-be55-949dfc3fc2de-istio-token\") pod \"0101f982-8c38-4576-be55-949dfc3fc2de\" (UID: \"0101f982-8c38-4576-be55-949dfc3fc2de\") "
Oct 30 22:25:12.589580 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:12.589495    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/00517b45-26ad-41d2-b86e-99d9092c2403-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/00517b45-26ad-41d2-b86e-99d9092c2403-istio-podinfo\") pod \"00517b45-26ad-41d2-b86e-99d9092c2403\" (UID: \"00517b45-26ad-41d2-b86e-99d9092c2403\") "
Oct 30 22:25:12.589580 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:12.589531    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0101f982-8c38-4576-be55-949dfc3fc2de-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0101f982-8c38-4576-be55-949dfc3fc2de-istio-envoy\") pod \"0101f982-8c38-4576-be55-949dfc3fc2de\" (UID: \"0101f982-8c38-4576-be55-949dfc3fc2de\") "
Oct 30 22:25:12.589922 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:12.589744    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/00517b45-26ad-41d2-b86e-99d9092c2403-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/00517b45-26ad-41d2-b86e-99d9092c2403-istiod-ca-cert\") pod \"00517b45-26ad-41d2-b86e-99d9092c2403\" (UID: \"00517b45-26ad-41d2-b86e-99d9092c2403\") "
Oct 30 22:25:12.589922 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:12.589812    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0101f982-8c38-4576-be55-949dfc3fc2de-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0101f982-8c38-4576-be55-949dfc3fc2de-credential-socket\") pod \"0101f982-8c38-4576-be55-949dfc3fc2de\" (UID: \"0101f982-8c38-4576-be55-949dfc3fc2de\") "
Oct 30 22:25:12.591276 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:12.590073    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/0101f982-8c38-4576-be55-949dfc3fc2de-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0101f982-8c38-4576-be55-949dfc3fc2de-istio-podinfo\") pod \"0101f982-8c38-4576-be55-949dfc3fc2de\" (UID: \"0101f982-8c38-4576-be55-949dfc3fc2de\") "
Oct 30 22:25:12.591276 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:12.590101    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0101f982-8c38-4576-be55-949dfc3fc2de-credential-socket" pod "0101f982-8c38-4576-be55-949dfc3fc2de" (UID: "0101f982-8c38-4576-be55-949dfc3fc2de"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:25:12.591276 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:12.590132    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/00517b45-26ad-41d2-b86e-99d9092c2403-istiod-ca-cert" pod "00517b45-26ad-41d2-b86e-99d9092c2403" (UID: "00517b45-26ad-41d2-b86e-99d9092c2403"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:25:12.591276 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:12.591119    5510 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/00517b45-26ad-41d2-b86e-99d9092c2403-credential-socket\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:25:12.591276 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:12.591144    5510 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/00517b45-26ad-41d2-b86e-99d9092c2403-istiod-ca-cert\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:25:12.591276 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:12.591159    5510 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0101f982-8c38-4576-be55-949dfc3fc2de-credential-socket\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:25:12.591276 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:12.591172    5510 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0101f982-8c38-4576-be55-949dfc3fc2de-istiod-ca-cert\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:25:12.591276 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:12.591185    5510 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/00517b45-26ad-41d2-b86e-99d9092c2403-workload-certs\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:25:12.591276 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:12.591197    5510 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0101f982-8c38-4576-be55-949dfc3fc2de-istio-data\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:25:12.591276 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:12.591208    5510 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/00517b45-26ad-41d2-b86e-99d9092c2403-istio-data\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:25:12.591276 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:12.591221    5510 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/00517b45-26ad-41d2-b86e-99d9092c2403-workload-socket\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:25:12.591276 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:12.591237    5510 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0101f982-8c38-4576-be55-949dfc3fc2de-workload-socket\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:25:12.591276 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:12.591250    5510 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0101f982-8c38-4576-be55-949dfc3fc2de-workload-certs\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:25:12.592087 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:12.591795    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/00517b45-26ad-41d2-b86e-99d9092c2403-istio-token" pod "00517b45-26ad-41d2-b86e-99d9092c2403" (UID: "00517b45-26ad-41d2-b86e-99d9092c2403"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:25:12.592087 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:12.591843    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/00517b45-26ad-41d2-b86e-99d9092c2403-kube-api-access-xwkh8" pod "00517b45-26ad-41d2-b86e-99d9092c2403" (UID: "00517b45-26ad-41d2-b86e-99d9092c2403"). InnerVolumeSpecName "kube-api-access-xwkh8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:25:12.592087 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:12.591880    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0101f982-8c38-4576-be55-949dfc3fc2de-kube-api-access-8lhzp" pod "0101f982-8c38-4576-be55-949dfc3fc2de" (UID: "0101f982-8c38-4576-be55-949dfc3fc2de"). InnerVolumeSpecName "kube-api-access-8lhzp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:25:12.592087 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:12.591922    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0101f982-8c38-4576-be55-949dfc3fc2de-istio-token" pod "0101f982-8c38-4576-be55-949dfc3fc2de" (UID: "0101f982-8c38-4576-be55-949dfc3fc2de"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:25:12.592510 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:12.592492    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/00517b45-26ad-41d2-b86e-99d9092c2403-istio-envoy" pod "00517b45-26ad-41d2-b86e-99d9092c2403" (UID: "00517b45-26ad-41d2-b86e-99d9092c2403"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:25:12.592563 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:12.592542    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/00517b45-26ad-41d2-b86e-99d9092c2403-istio-podinfo" pod "00517b45-26ad-41d2-b86e-99d9092c2403" (UID: "00517b45-26ad-41d2-b86e-99d9092c2403"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:25:12.593208 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:12.593190    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0101f982-8c38-4576-be55-949dfc3fc2de-istio-envoy" pod "0101f982-8c38-4576-be55-949dfc3fc2de" (UID: "0101f982-8c38-4576-be55-949dfc3fc2de"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:25:12.593621 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:12.593606    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0101f982-8c38-4576-be55-949dfc3fc2de-istio-podinfo" pod "0101f982-8c38-4576-be55-949dfc3fc2de" (UID: "0101f982-8c38-4576-be55-949dfc3fc2de"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:25:12.691974 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:12.691878    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a089cfee-9de2-4d44-b7d9-dbbc2a66ebd4-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a089cfee-9de2-4d44-b7d9-dbbc2a66ebd4-istio-data\") pod \"a089cfee-9de2-4d44-b7d9-dbbc2a66ebd4\" (UID: \"a089cfee-9de2-4d44-b7d9-dbbc2a66ebd4\") "
Oct 30 22:25:12.691974 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:12.691922    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a089cfee-9de2-4d44-b7d9-dbbc2a66ebd4-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a089cfee-9de2-4d44-b7d9-dbbc2a66ebd4-workload-socket\") pod \"a089cfee-9de2-4d44-b7d9-dbbc2a66ebd4\" (UID: \"a089cfee-9de2-4d44-b7d9-dbbc2a66ebd4\") "
Oct 30 22:25:12.692209 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:12.692025    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a089cfee-9de2-4d44-b7d9-dbbc2a66ebd4-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a089cfee-9de2-4d44-b7d9-dbbc2a66ebd4-credential-socket\") pod \"a089cfee-9de2-4d44-b7d9-dbbc2a66ebd4\" (UID: \"a089cfee-9de2-4d44-b7d9-dbbc2a66ebd4\") "
Oct 30 22:25:12.692209 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:12.692062    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a089cfee-9de2-4d44-b7d9-dbbc2a66ebd4-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a089cfee-9de2-4d44-b7d9-dbbc2a66ebd4-istiod-ca-cert\") pod \"a089cfee-9de2-4d44-b7d9-dbbc2a66ebd4\" (UID: \"a089cfee-9de2-4d44-b7d9-dbbc2a66ebd4\") "
Oct 30 22:25:12.692209 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:12.692169    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a089cfee-9de2-4d44-b7d9-dbbc2a66ebd4-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a089cfee-9de2-4d44-b7d9-dbbc2a66ebd4-istio-envoy\") pod \"a089cfee-9de2-4d44-b7d9-dbbc2a66ebd4\" (UID: \"a089cfee-9de2-4d44-b7d9-dbbc2a66ebd4\") "
Oct 30 22:25:12.692388 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:12.692240    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a089cfee-9de2-4d44-b7d9-dbbc2a66ebd4-istio-data" pod "a089cfee-9de2-4d44-b7d9-dbbc2a66ebd4" (UID: "a089cfee-9de2-4d44-b7d9-dbbc2a66ebd4"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:25:12.692388 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:12.692274    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a089cfee-9de2-4d44-b7d9-dbbc2a66ebd4-workload-socket" pod "a089cfee-9de2-4d44-b7d9-dbbc2a66ebd4" (UID: "a089cfee-9de2-4d44-b7d9-dbbc2a66ebd4"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:25:12.692388 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:12.692283    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a089cfee-9de2-4d44-b7d9-dbbc2a66ebd4-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a089cfee-9de2-4d44-b7d9-dbbc2a66ebd4-workload-certs\") pod \"a089cfee-9de2-4d44-b7d9-dbbc2a66ebd4\" (UID: \"a089cfee-9de2-4d44-b7d9-dbbc2a66ebd4\") "
Oct 30 22:25:12.692549 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:12.692389    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a089cfee-9de2-4d44-b7d9-dbbc2a66ebd4-credential-socket" pod "a089cfee-9de2-4d44-b7d9-dbbc2a66ebd4" (UID: "a089cfee-9de2-4d44-b7d9-dbbc2a66ebd4"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:25:12.692549 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:12.692458    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a089cfee-9de2-4d44-b7d9-dbbc2a66ebd4-istio-token\" (UniqueName: \"kubernetes.io/projected/a089cfee-9de2-4d44-b7d9-dbbc2a66ebd4-istio-token\") pod \"a089cfee-9de2-4d44-b7d9-dbbc2a66ebd4\" (UID: \"a089cfee-9de2-4d44-b7d9-dbbc2a66ebd4\") "
Oct 30 22:25:12.692549 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:12.692486    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a089cfee-9de2-4d44-b7d9-dbbc2a66ebd4-workload-certs" pod "a089cfee-9de2-4d44-b7d9-dbbc2a66ebd4" (UID: "a089cfee-9de2-4d44-b7d9-dbbc2a66ebd4"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:25:12.692549 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:12.692502    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/a089cfee-9de2-4d44-b7d9-dbbc2a66ebd4-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a089cfee-9de2-4d44-b7d9-dbbc2a66ebd4-istio-podinfo\") pod \"a089cfee-9de2-4d44-b7d9-dbbc2a66ebd4\" (UID: \"a089cfee-9de2-4d44-b7d9-dbbc2a66ebd4\") "
Oct 30 22:25:12.692800 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:12.692553    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a089cfee-9de2-4d44-b7d9-dbbc2a66ebd4-kube-api-access-t29zw\" (UniqueName: \"kubernetes.io/projected/a089cfee-9de2-4d44-b7d9-dbbc2a66ebd4-kube-api-access-t29zw\") pod \"a089cfee-9de2-4d44-b7d9-dbbc2a66ebd4\" (UID: \"a089cfee-9de2-4d44-b7d9-dbbc2a66ebd4\") "
Oct 30 22:25:12.692800 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:12.692559    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a089cfee-9de2-4d44-b7d9-dbbc2a66ebd4-istiod-ca-cert" pod "a089cfee-9de2-4d44-b7d9-dbbc2a66ebd4" (UID: "a089cfee-9de2-4d44-b7d9-dbbc2a66ebd4"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:25:12.693745 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:12.693716    5510 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a089cfee-9de2-4d44-b7d9-dbbc2a66ebd4-workload-certs\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:25:12.693745 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:12.693743    5510 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0101f982-8c38-4576-be55-949dfc3fc2de-istio-podinfo\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:25:12.693940 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:12.693759    5510 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xwkh8\" (UniqueName: \"kubernetes.io/projected/00517b45-26ad-41d2-b86e-99d9092c2403-kube-api-access-xwkh8\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:25:12.693940 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:12.693775    5510 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8lhzp\" (UniqueName: \"kubernetes.io/projected/0101f982-8c38-4576-be55-949dfc3fc2de-kube-api-access-8lhzp\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:25:12.693940 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:12.693791    5510 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a089cfee-9de2-4d44-b7d9-dbbc2a66ebd4-istio-data\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:25:12.693940 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:12.693808    5510 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a089cfee-9de2-4d44-b7d9-dbbc2a66ebd4-workload-socket\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:25:12.693940 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:12.693824    5510 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/00517b45-26ad-41d2-b86e-99d9092c2403-istio-token\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:25:12.693940 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:12.693838    5510 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a089cfee-9de2-4d44-b7d9-dbbc2a66ebd4-credential-socket\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:25:12.693940 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:12.693861    5510 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a089cfee-9de2-4d44-b7d9-dbbc2a66ebd4-istiod-ca-cert\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:25:12.693940 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:12.693871    5510 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/00517b45-26ad-41d2-b86e-99d9092c2403-istio-envoy\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:25:12.693940 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:12.693879    5510 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0101f982-8c38-4576-be55-949dfc3fc2de-istio-token\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:25:12.693940 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:12.693887    5510 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/00517b45-26ad-41d2-b86e-99d9092c2403-istio-podinfo\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:25:12.693940 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:12.693898    5510 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0101f982-8c38-4576-be55-949dfc3fc2de-istio-envoy\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:25:12.694996 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:12.694965    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a089cfee-9de2-4d44-b7d9-dbbc2a66ebd4-istio-envoy" pod "a089cfee-9de2-4d44-b7d9-dbbc2a66ebd4" (UID: "a089cfee-9de2-4d44-b7d9-dbbc2a66ebd4"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:25:12.695123 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:12.695028    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a089cfee-9de2-4d44-b7d9-dbbc2a66ebd4-istio-token" pod "a089cfee-9de2-4d44-b7d9-dbbc2a66ebd4" (UID: "a089cfee-9de2-4d44-b7d9-dbbc2a66ebd4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:25:12.695123 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:12.695048    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a089cfee-9de2-4d44-b7d9-dbbc2a66ebd4-kube-api-access-t29zw" pod "a089cfee-9de2-4d44-b7d9-dbbc2a66ebd4" (UID: "a089cfee-9de2-4d44-b7d9-dbbc2a66ebd4"). InnerVolumeSpecName "kube-api-access-t29zw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:25:12.695364 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:12.695351    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a089cfee-9de2-4d44-b7d9-dbbc2a66ebd4-istio-podinfo" pod "a089cfee-9de2-4d44-b7d9-dbbc2a66ebd4" (UID: "a089cfee-9de2-4d44-b7d9-dbbc2a66ebd4"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:25:12.795729 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:12.795693    5510 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a089cfee-9de2-4d44-b7d9-dbbc2a66ebd4-istio-token\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:25:12.795729 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:12.795726    5510 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a089cfee-9de2-4d44-b7d9-dbbc2a66ebd4-istio-podinfo\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:25:12.795945 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:12.795741    5510 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t29zw\" (UniqueName: \"kubernetes.io/projected/a089cfee-9de2-4d44-b7d9-dbbc2a66ebd4-kube-api-access-t29zw\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:25:12.795945 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:12.795755    5510 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a089cfee-9de2-4d44-b7d9-dbbc2a66ebd4-istio-envoy\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:25:12.887290 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:12.887043    5510 generic.go:356] "Generic (PLEG): container finished" podID="a089cfee-9de2-4d44-b7d9-dbbc2a66ebd4" containerID="431729942c9ae9dc292e10d5cdf7329d815661b49d6bdcd85877e365b8488573" exitCode=0
Oct 30 22:25:12.887619 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:12.887124    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-j4m88"
Oct 30 22:25:12.887818 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:12.887140    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-j4m88" event={"ID":"a089cfee-9de2-4d44-b7d9-dbbc2a66ebd4","Type":"ContainerDied","Data":"431729942c9ae9dc292e10d5cdf7329d815661b49d6bdcd85877e365b8488573"}
Oct 30 22:25:12.887818 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:12.887746    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-j4m88" event={"ID":"a089cfee-9de2-4d44-b7d9-dbbc2a66ebd4","Type":"ContainerDied","Data":"cd6dea1499f666a7cc7afc0a66e18a52aca824a9aa9167d0fe6df971e9176e13"}
Oct 30 22:25:12.887818 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:12.887784    5510 scope.go:122] "RemoveContainer" containerID="431729942c9ae9dc292e10d5cdf7329d815661b49d6bdcd85877e365b8488573"
Oct 30 22:25:12.889255 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:12.889126    5510 generic.go:356] "Generic (PLEG): container finished" podID="0101f982-8c38-4576-be55-949dfc3fc2de" containerID="97e7fc3b556f7b3864d94a4ad7254ea0ff9badb4a6e1da68d91af1e3451c4a30" exitCode=0
Oct 30 22:25:12.889338 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:12.889273    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-jtsvv" event={"ID":"0101f982-8c38-4576-be55-949dfc3fc2de","Type":"ContainerDied","Data":"97e7fc3b556f7b3864d94a4ad7254ea0ff9badb4a6e1da68d91af1e3451c4a30"}
Oct 30 22:25:12.889338 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:12.889312    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-jtsvv" event={"ID":"0101f982-8c38-4576-be55-949dfc3fc2de","Type":"ContainerDied","Data":"5658a607c7d4d4d8a331699557783c14c93a5b8aac3870bfd4a63ae819a998c4"}
Oct 30 22:25:12.889416 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:12.889373    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-jtsvv"
Oct 30 22:25:12.896583 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:12.896553    5510 generic.go:356] "Generic (PLEG): container finished" podID="00517b45-26ad-41d2-b86e-99d9092c2403" containerID="a0e12056ee7c8d043a262bdc5684d0d0f797a7ac2e31c4f8f3d351f84bc90018" exitCode=0
Oct 30 22:25:12.896724 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:12.896600    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-vtwfv"
Oct 30 22:25:12.896724 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:12.896601    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-vtwfv" event={"ID":"00517b45-26ad-41d2-b86e-99d9092c2403","Type":"ContainerDied","Data":"a0e12056ee7c8d043a262bdc5684d0d0f797a7ac2e31c4f8f3d351f84bc90018"}
Oct 30 22:25:12.896724 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:12.896706    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-vtwfv" event={"ID":"00517b45-26ad-41d2-b86e-99d9092c2403","Type":"ContainerDied","Data":"7a04639681263207122fc121fcfc8ed37ad4cd9c19d36748c6870185bee7705f"}
Oct 30 22:25:12.900273 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:12.900255    5510 scope.go:122] "RemoveContainer" containerID="431729942c9ae9dc292e10d5cdf7329d815661b49d6bdcd85877e365b8488573"
Oct 30 22:25:12.900593 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:25:12.900568    5510 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"431729942c9ae9dc292e10d5cdf7329d815661b49d6bdcd85877e365b8488573\": container with ID starting with 431729942c9ae9dc292e10d5cdf7329d815661b49d6bdcd85877e365b8488573 not found: ID does not exist" containerID="431729942c9ae9dc292e10d5cdf7329d815661b49d6bdcd85877e365b8488573"
Oct 30 22:25:12.900712 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:12.900601    5510 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"431729942c9ae9dc292e10d5cdf7329d815661b49d6bdcd85877e365b8488573"} err="failed to get container status \"431729942c9ae9dc292e10d5cdf7329d815661b49d6bdcd85877e365b8488573\": rpc error: code = NotFound desc = could not find container \"431729942c9ae9dc292e10d5cdf7329d815661b49d6bdcd85877e365b8488573\": container with ID starting with 431729942c9ae9dc292e10d5cdf7329d815661b49d6bdcd85877e365b8488573 not found: ID does not exist"
Oct 30 22:25:12.900712 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:12.900620    5510 scope.go:122] "RemoveContainer" containerID="97e7fc3b556f7b3864d94a4ad7254ea0ff9badb4a6e1da68d91af1e3451c4a30"
Oct 30 22:25:12.916475 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:12.916450    5510 scope.go:122] "RemoveContainer" containerID="97e7fc3b556f7b3864d94a4ad7254ea0ff9badb4a6e1da68d91af1e3451c4a30"
Oct 30 22:25:12.916873 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:25:12.916846    5510 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"97e7fc3b556f7b3864d94a4ad7254ea0ff9badb4a6e1da68d91af1e3451c4a30\": container with ID starting with 97e7fc3b556f7b3864d94a4ad7254ea0ff9badb4a6e1da68d91af1e3451c4a30 not found: ID does not exist" containerID="97e7fc3b556f7b3864d94a4ad7254ea0ff9badb4a6e1da68d91af1e3451c4a30"
Oct 30 22:25:12.916973 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:12.916884    5510 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"97e7fc3b556f7b3864d94a4ad7254ea0ff9badb4a6e1da68d91af1e3451c4a30"} err="failed to get container status \"97e7fc3b556f7b3864d94a4ad7254ea0ff9badb4a6e1da68d91af1e3451c4a30\": rpc error: code = NotFound desc = could not find container \"97e7fc3b556f7b3864d94a4ad7254ea0ff9badb4a6e1da68d91af1e3451c4a30\": container with ID starting with 97e7fc3b556f7b3864d94a4ad7254ea0ff9badb4a6e1da68d91af1e3451c4a30 not found: ID does not exist"
Oct 30 22:25:12.916973 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:12.916908    5510 scope.go:122] "RemoveContainer" containerID="a0e12056ee7c8d043a262bdc5684d0d0f797a7ac2e31c4f8f3d351f84bc90018"
Oct 30 22:25:12.927826 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:12.927795    5510 scope.go:122] "RemoveContainer" containerID="a0e12056ee7c8d043a262bdc5684d0d0f797a7ac2e31c4f8f3d351f84bc90018"
Oct 30 22:25:12.928163 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:25:12.928140    5510 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a0e12056ee7c8d043a262bdc5684d0d0f797a7ac2e31c4f8f3d351f84bc90018\": container with ID starting with a0e12056ee7c8d043a262bdc5684d0d0f797a7ac2e31c4f8f3d351f84bc90018 not found: ID does not exist" containerID="a0e12056ee7c8d043a262bdc5684d0d0f797a7ac2e31c4f8f3d351f84bc90018"
Oct 30 22:25:12.928235 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:12.928172    5510 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a0e12056ee7c8d043a262bdc5684d0d0f797a7ac2e31c4f8f3d351f84bc90018"} err="failed to get container status \"a0e12056ee7c8d043a262bdc5684d0d0f797a7ac2e31c4f8f3d351f84bc90018\": rpc error: code = NotFound desc = could not find container \"a0e12056ee7c8d043a262bdc5684d0d0f797a7ac2e31c4f8f3d351f84bc90018\": container with ID starting with a0e12056ee7c8d043a262bdc5684d0d0f797a7ac2e31c4f8f3d351f84bc90018 not found: ID does not exist"
Oct 30 22:25:13.046650 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:13.046611    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-jtsvv"]
Oct 30 22:25:13.054257 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:13.054228    5510 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-jtsvv"]
Oct 30 22:25:13.159043 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:13.159003    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-j4m88"]
Oct 30 22:25:13.167712 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:13.167684    5510 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-j4m88"]
Oct 30 22:25:13.301735 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:13.299708    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-vtwfv"]
Oct 30 22:25:13.348679 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:13.348318    5510 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-vtwfv"]
Oct 30 22:25:13.555547 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:13.555470    5510 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="00517b45-26ad-41d2-b86e-99d9092c2403" path="/var/lib/kubelet/pods/00517b45-26ad-41d2-b86e-99d9092c2403/volumes"
Oct 30 22:25:13.556075 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:13.556059    5510 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0101f982-8c38-4576-be55-949dfc3fc2de" path="/var/lib/kubelet/pods/0101f982-8c38-4576-be55-949dfc3fc2de/volumes"
Oct 30 22:25:13.556555 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:13.556541    5510 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a089cfee-9de2-4d44-b7d9-dbbc2a66ebd4" path="/var/lib/kubelet/pods/a089cfee-9de2-4d44-b7d9-dbbc2a66ebd4/volumes"
Oct 30 22:25:15.909679 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:15.907727    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:25:16.335323 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:16.335291    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:25:17.629891 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:17.629842    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:25:18.898819 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:18.898720    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:25:19.164051 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:19.163964    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:25:21.013497 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:21.013461    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:25:22.502905 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:22.502867    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:25:23.324915 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:23.324876    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:25:23.691948 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:23.691859    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:25:23.996205 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:23.996127    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:25:24.014104 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:24.014074    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-xbr94"]
Oct 30 22:25:24.014310 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:24.014282    5510 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-xbr94" podUID="af4ee930-6db5-4c5c-9727-1ca855360b34" containerName="istio-proxy" containerID="cri-o://445c0c4f1af25dc592c4a25f8f5a08758b91b16fb928dd8ae7996c4bc64f0544" gracePeriod=30
Oct 30 22:25:24.269979 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:24.269940    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:25:26.271331 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:26.271279    5510 patch_prober.go:29] interesting pod/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-xbr94 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 22:25:26.271758 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:26.271346    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-xbr94" podUID="af4ee930-6db5-4c5c-9727-1ca855360b34" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 22:25:26.385438 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:26.385401    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:25:27.117470 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:27.117432    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:25:27.333387 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:27.333207    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:25:28.688182 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:28.688149    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:25:28.933516 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:28.933481    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:25:29.301832 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:29.301756    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-xbr94"
Oct 30 22:25:29.406540 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:29.406496    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/af4ee930-6db5-4c5c-9727-1ca855360b34-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/af4ee930-6db5-4c5c-9727-1ca855360b34-credential-socket\") pod \"af4ee930-6db5-4c5c-9727-1ca855360b34\" (UID: \"af4ee930-6db5-4c5c-9727-1ca855360b34\") "
Oct 30 22:25:29.406773 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:29.406552    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/af4ee930-6db5-4c5c-9727-1ca855360b34-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/af4ee930-6db5-4c5c-9727-1ca855360b34-istio-podinfo\") pod \"af4ee930-6db5-4c5c-9727-1ca855360b34\" (UID: \"af4ee930-6db5-4c5c-9727-1ca855360b34\") "
Oct 30 22:25:29.406773 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:29.406601    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/af4ee930-6db5-4c5c-9727-1ca855360b34-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/af4ee930-6db5-4c5c-9727-1ca855360b34-istio-data\") pod \"af4ee930-6db5-4c5c-9727-1ca855360b34\" (UID: \"af4ee930-6db5-4c5c-9727-1ca855360b34\") "
Oct 30 22:25:29.406773 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:29.406769    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/af4ee930-6db5-4c5c-9727-1ca855360b34-istio-token\" (UniqueName: \"kubernetes.io/projected/af4ee930-6db5-4c5c-9727-1ca855360b34-istio-token\") pod \"af4ee930-6db5-4c5c-9727-1ca855360b34\" (UID: \"af4ee930-6db5-4c5c-9727-1ca855360b34\") "
Oct 30 22:25:29.406955 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:29.406809    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/af4ee930-6db5-4c5c-9727-1ca855360b34-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/af4ee930-6db5-4c5c-9727-1ca855360b34-workload-socket\") pod \"af4ee930-6db5-4c5c-9727-1ca855360b34\" (UID: \"af4ee930-6db5-4c5c-9727-1ca855360b34\") "
Oct 30 22:25:29.406955 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:29.406849    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/af4ee930-6db5-4c5c-9727-1ca855360b34-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/af4ee930-6db5-4c5c-9727-1ca855360b34-istio-envoy\") pod \"af4ee930-6db5-4c5c-9727-1ca855360b34\" (UID: \"af4ee930-6db5-4c5c-9727-1ca855360b34\") "
Oct 30 22:25:29.406955 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:29.406802    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/af4ee930-6db5-4c5c-9727-1ca855360b34-credential-socket" pod "af4ee930-6db5-4c5c-9727-1ca855360b34" (UID: "af4ee930-6db5-4c5c-9727-1ca855360b34"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:25:29.406955 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:29.406902    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/af4ee930-6db5-4c5c-9727-1ca855360b34-istio-data" pod "af4ee930-6db5-4c5c-9727-1ca855360b34" (UID: "af4ee930-6db5-4c5c-9727-1ca855360b34"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:25:29.406955 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:29.406923    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/af4ee930-6db5-4c5c-9727-1ca855360b34-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/af4ee930-6db5-4c5c-9727-1ca855360b34-workload-certs\") pod \"af4ee930-6db5-4c5c-9727-1ca855360b34\" (UID: \"af4ee930-6db5-4c5c-9727-1ca855360b34\") "
Oct 30 22:25:29.407193 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:29.406999    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/af4ee930-6db5-4c5c-9727-1ca855360b34-workload-socket" pod "af4ee930-6db5-4c5c-9727-1ca855360b34" (UID: "af4ee930-6db5-4c5c-9727-1ca855360b34"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:25:29.407193 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:29.407026    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/af4ee930-6db5-4c5c-9727-1ca855360b34-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/af4ee930-6db5-4c5c-9727-1ca855360b34-istiod-ca-cert\") pod \"af4ee930-6db5-4c5c-9727-1ca855360b34\" (UID: \"af4ee930-6db5-4c5c-9727-1ca855360b34\") "
Oct 30 22:25:29.407193 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:29.407078    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/af4ee930-6db5-4c5c-9727-1ca855360b34-kube-api-access-9g7vz\" (UniqueName: \"kubernetes.io/projected/af4ee930-6db5-4c5c-9727-1ca855360b34-kube-api-access-9g7vz\") pod \"af4ee930-6db5-4c5c-9727-1ca855360b34\" (UID: \"af4ee930-6db5-4c5c-9727-1ca855360b34\") "
Oct 30 22:25:29.407349 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:29.407191    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/af4ee930-6db5-4c5c-9727-1ca855360b34-workload-certs" pod "af4ee930-6db5-4c5c-9727-1ca855360b34" (UID: "af4ee930-6db5-4c5c-9727-1ca855360b34"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:25:29.407437 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:29.407415    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/af4ee930-6db5-4c5c-9727-1ca855360b34-istiod-ca-cert" pod "af4ee930-6db5-4c5c-9727-1ca855360b34" (UID: "af4ee930-6db5-4c5c-9727-1ca855360b34"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:25:29.407970 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:29.407949    5510 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/af4ee930-6db5-4c5c-9727-1ca855360b34-workload-certs\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:25:29.408040 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:29.407977    5510 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/af4ee930-6db5-4c5c-9727-1ca855360b34-istiod-ca-cert\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:25:29.408040 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:29.407992    5510 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/af4ee930-6db5-4c5c-9727-1ca855360b34-credential-socket\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:25:29.408040 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:29.408006    5510 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/af4ee930-6db5-4c5c-9727-1ca855360b34-istio-data\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:25:29.408040 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:29.408020    5510 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/af4ee930-6db5-4c5c-9727-1ca855360b34-workload-socket\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:25:29.409142 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:29.409118    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/af4ee930-6db5-4c5c-9727-1ca855360b34-istio-token" pod "af4ee930-6db5-4c5c-9727-1ca855360b34" (UID: "af4ee930-6db5-4c5c-9727-1ca855360b34"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:25:29.409240 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:29.409169    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/af4ee930-6db5-4c5c-9727-1ca855360b34-kube-api-access-9g7vz" pod "af4ee930-6db5-4c5c-9727-1ca855360b34" (UID: "af4ee930-6db5-4c5c-9727-1ca855360b34"). InnerVolumeSpecName "kube-api-access-9g7vz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:25:29.409370 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:29.409345    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/af4ee930-6db5-4c5c-9727-1ca855360b34-istio-podinfo" pod "af4ee930-6db5-4c5c-9727-1ca855360b34" (UID: "af4ee930-6db5-4c5c-9727-1ca855360b34"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:25:29.409370 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:29.409350    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/af4ee930-6db5-4c5c-9727-1ca855360b34-istio-envoy" pod "af4ee930-6db5-4c5c-9727-1ca855360b34" (UID: "af4ee930-6db5-4c5c-9727-1ca855360b34"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:25:29.510253 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:29.510197    5510 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/af4ee930-6db5-4c5c-9727-1ca855360b34-istio-token\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:25:29.510253 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:29.510253    5510 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/af4ee930-6db5-4c5c-9727-1ca855360b34-istio-envoy\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:25:29.510531 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:29.510271    5510 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9g7vz\" (UniqueName: \"kubernetes.io/projected/af4ee930-6db5-4c5c-9727-1ca855360b34-kube-api-access-9g7vz\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:25:29.510531 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:29.510287    5510 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/af4ee930-6db5-4c5c-9727-1ca855360b34-istio-podinfo\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:25:30.107789 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:30.107750    5510 generic.go:356] "Generic (PLEG): container finished" podID="af4ee930-6db5-4c5c-9727-1ca855360b34" containerID="445c0c4f1af25dc592c4a25f8f5a08758b91b16fb928dd8ae7996c4bc64f0544" exitCode=0
Oct 30 22:25:30.108259 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:30.107803    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-xbr94"
Oct 30 22:25:30.108259 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:30.107831    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-xbr94" event={"ID":"af4ee930-6db5-4c5c-9727-1ca855360b34","Type":"ContainerDied","Data":"445c0c4f1af25dc592c4a25f8f5a08758b91b16fb928dd8ae7996c4bc64f0544"}
Oct 30 22:25:30.108259 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:30.107889    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-xbr94" event={"ID":"af4ee930-6db5-4c5c-9727-1ca855360b34","Type":"ContainerDied","Data":"a1edd121ab99acf681dd5503aca8b5edffc0e491c9c0a77c11a0a89f84469978"}
Oct 30 22:25:30.108259 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:30.107911    5510 scope.go:122] "RemoveContainer" containerID="445c0c4f1af25dc592c4a25f8f5a08758b91b16fb928dd8ae7996c4bc64f0544"
Oct 30 22:25:30.118957 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:30.118936    5510 scope.go:122] "RemoveContainer" containerID="445c0c4f1af25dc592c4a25f8f5a08758b91b16fb928dd8ae7996c4bc64f0544"
Oct 30 22:25:30.119258 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:25:30.119234    5510 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"445c0c4f1af25dc592c4a25f8f5a08758b91b16fb928dd8ae7996c4bc64f0544\": container with ID starting with 445c0c4f1af25dc592c4a25f8f5a08758b91b16fb928dd8ae7996c4bc64f0544 not found: ID does not exist" containerID="445c0c4f1af25dc592c4a25f8f5a08758b91b16fb928dd8ae7996c4bc64f0544"
Oct 30 22:25:30.119323 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:30.119267    5510 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"445c0c4f1af25dc592c4a25f8f5a08758b91b16fb928dd8ae7996c4bc64f0544"} err="failed to get container status \"445c0c4f1af25dc592c4a25f8f5a08758b91b16fb928dd8ae7996c4bc64f0544\": rpc error: code = NotFound desc = could not find container \"445c0c4f1af25dc592c4a25f8f5a08758b91b16fb928dd8ae7996c4bc64f0544\": container with ID starting with 445c0c4f1af25dc592c4a25f8f5a08758b91b16fb928dd8ae7996c4bc64f0544 not found: ID does not exist"
Oct 30 22:25:30.337208 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:30.337172    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-xbr94"]
Oct 30 22:25:30.384138 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:30.384055    5510 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-xbr94"]
Oct 30 22:25:30.410982 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:30.410948    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:25:31.555567 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:31.555536    5510 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="af4ee930-6db5-4c5c-9727-1ca855360b34" path="/var/lib/kubelet/pods/af4ee930-6db5-4c5c-9727-1ca855360b34/volumes"
Oct 30 22:25:31.605000 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:31.604966    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:25:31.979383 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:31.979262    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:25:32.197679 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:32.193251    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:25:32.603311 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:32.603277    5510 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-0b6a3f-f4bfcfb9f-fvtq8"]
Oct 30 22:25:32.604126 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:32.604109    5510 cpu_manager.go:414] "containerMap: removing container" podUID="af4ee930-6db5-4c5c-9727-1ca855360b34" containerName="istio-proxy"
Oct 30 22:25:32.604176 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:32.604128    5510 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="af4ee930-6db5-4c5c-9727-1ca855360b34" containerName="istio-proxy"
Oct 30 22:25:32.604176 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:32.604141    5510 cpu_manager.go:414] "containerMap: removing container" podUID="a089cfee-9de2-4d44-b7d9-dbbc2a66ebd4" containerName="istio-proxy"
Oct 30 22:25:32.604176 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:32.604148    5510 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a089cfee-9de2-4d44-b7d9-dbbc2a66ebd4" containerName="istio-proxy"
Oct 30 22:25:32.604176 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:32.604165    5510 cpu_manager.go:414] "containerMap: removing container" podUID="00517b45-26ad-41d2-b86e-99d9092c2403" containerName="istio-proxy"
Oct 30 22:25:32.604176 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:32.604170    5510 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="00517b45-26ad-41d2-b86e-99d9092c2403" containerName="istio-proxy"
Oct 30 22:25:32.604176 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:32.604176    5510 cpu_manager.go:414] "containerMap: removing container" podUID="0101f982-8c38-4576-be55-949dfc3fc2de" containerName="istio-proxy"
Oct 30 22:25:32.604368 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:32.604182    5510 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0101f982-8c38-4576-be55-949dfc3fc2de" containerName="istio-proxy"
Oct 30 22:25:32.604368 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:32.604331    5510 memory_manager.go:365] "RemoveStaleState removing state" podUID="00517b45-26ad-41d2-b86e-99d9092c2403" containerName="istio-proxy"
Oct 30 22:25:32.604368 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:32.604340    5510 memory_manager.go:365] "RemoveStaleState removing state" podUID="0101f982-8c38-4576-be55-949dfc3fc2de" containerName="istio-proxy"
Oct 30 22:25:32.604368 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:32.604357    5510 memory_manager.go:365] "RemoveStaleState removing state" podUID="af4ee930-6db5-4c5c-9727-1ca855360b34" containerName="istio-proxy"
Oct 30 22:25:32.604368 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:32.604364    5510 memory_manager.go:365] "RemoveStaleState removing state" podUID="a089cfee-9de2-4d44-b7d9-dbbc2a66ebd4" containerName="istio-proxy"
Oct 30 22:25:32.609920 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:32.609894    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-0b6a3f-f4bfcfb9f-fvtq8"
Oct 30 22:25:32.632403 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:32.632366    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xgv5f\" (UniqueName: \"kubernetes.io/projected/8023fe04-37eb-4458-bfee-ef8170d34ab6-kube-api-access-xgv5f\") pod \"payload-pre-processing-e2e-api-a-0b6a3f-f4bfcfb9f-fvtq8\" (UID: \"8023fe04-37eb-4458-bfee-ef8170d34ab6\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-0b6a3f-f4bfcfb9f-fvtq8"
Oct 30 22:25:32.632639 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:32.632615    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8023fe04-37eb-4458-bfee-ef8170d34ab6-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-0b6a3f-f4bfcfb9f-fvtq8\" (UID: \"8023fe04-37eb-4458-bfee-ef8170d34ab6\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-0b6a3f-f4bfcfb9f-fvtq8"
Oct 30 22:25:32.666321 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:32.666292    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-0b6a3f-dockercfg-cns8g\""
Oct 30 22:25:32.670181 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:32.670155    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-0b6a3f-f4bfcfb9f-fvtq8"]
Oct 30 22:25:32.711863 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:32.711826    5510 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-0b6a3f\""
Oct 30 22:25:32.734159 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:32.734128    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xgv5f\" (UniqueName: \"kubernetes.io/projected/8023fe04-37eb-4458-bfee-ef8170d34ab6-kube-api-access-xgv5f\") pod \"payload-pre-processing-e2e-api-a-0b6a3f-f4bfcfb9f-fvtq8\" (UID: \"8023fe04-37eb-4458-bfee-ef8170d34ab6\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-0b6a3f-f4bfcfb9f-fvtq8"
Oct 30 22:25:32.734315 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:32.734248    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8023fe04-37eb-4458-bfee-ef8170d34ab6-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-0b6a3f-f4bfcfb9f-fvtq8\" (UID: \"8023fe04-37eb-4458-bfee-ef8170d34ab6\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-0b6a3f-f4bfcfb9f-fvtq8"
Oct 30 22:25:32.734969 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:32.734946    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8023fe04-37eb-4458-bfee-ef8170d34ab6-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-0b6a3f-f4bfcfb9f-fvtq8\" (UID: \"8023fe04-37eb-4458-bfee-ef8170d34ab6\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-0b6a3f-f4bfcfb9f-fvtq8"
Oct 30 22:25:32.792282 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:32.792244    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xgv5f\" (UniqueName: \"kubernetes.io/projected/8023fe04-37eb-4458-bfee-ef8170d34ab6-kube-api-access-xgv5f\") pod \"payload-pre-processing-e2e-api-a-0b6a3f-f4bfcfb9f-fvtq8\" (UID: \"8023fe04-37eb-4458-bfee-ef8170d34ab6\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-0b6a3f-f4bfcfb9f-fvtq8"
Oct 30 22:25:32.926252 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:32.926158    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-0b6a3f-f4bfcfb9f-fvtq8"
Oct 30 22:25:33.305756 ip-10-0-136-210 kubenswrapper[5510]: W1030 22:25:33.305726    5510 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8023fe04_37eb_4458_bfee_ef8170d34ab6.slice/crio-2c8a8fefe43b82d55b9ae8fd69eb330b43c6305ffdad82f480d7cef4cebbbf59 WatchSource:0}: Error finding container 2c8a8fefe43b82d55b9ae8fd69eb330b43c6305ffdad82f480d7cef4cebbbf59: Status 404 returned error can't find the container with id 2c8a8fefe43b82d55b9ae8fd69eb330b43c6305ffdad82f480d7cef4cebbbf59
Oct 30 22:25:33.306214 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:33.306191    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-0b6a3f-f4bfcfb9f-fvtq8"]
Oct 30 22:25:34.143495 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:34.143451    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-0b6a3f-f4bfcfb9f-fvtq8" event={"ID":"8023fe04-37eb-4458-bfee-ef8170d34ab6","Type":"ContainerStarted","Data":"df4370ed198e7b1f06806d9c2917ae30be569e48db0e52c0cc8dc673d96a67b7"}
Oct 30 22:25:34.143960 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:34.143506    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-0b6a3f-f4bfcfb9f-fvtq8" event={"ID":"8023fe04-37eb-4458-bfee-ef8170d34ab6","Type":"ContainerStarted","Data":"2c8a8fefe43b82d55b9ae8fd69eb330b43c6305ffdad82f480d7cef4cebbbf59"}
Oct 30 22:25:34.143960 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:34.143532    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-0b6a3f-f4bfcfb9f-fvtq8"
Oct 30 22:25:34.257999 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:34.257947    5510 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-0b6a3f-f4bfcfb9f-fvtq8" podStartSLOduration=2.257933724 podStartE2EDuration="2.257933724s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:25:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:25:34.254416825 +0000 UTC m=+2114.815226833" watchObservedRunningTime="2026-10-30 22:25:34.257933724 +0000 UTC m=+2114.818743728"
Oct 30 22:25:36.990428 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:36.990390    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:25:38.422300 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:25:38.422255    5510 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6gjbf" containerName="extractor"
Oct 30 22:25:38.520321 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:38.520278    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:25:38.723129 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:38.723036    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:25:39.763844 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:39.763804    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:25:40.582650 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:40.582615    5510 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7403-3cbdb7"]
Oct 30 22:25:40.587509 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:40.587490    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7403-3cbdb7"
Oct 30 22:25:40.639026 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:40.638971    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7403-3cbdb7"]
Oct 30 22:25:40.679802 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:40.679766    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-rx2kz\""
Oct 30 22:25:40.692025 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:40.691990    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jqph9\" (UniqueName: \"kubernetes.io/projected/b439bfca-283c-4bf4-8431-44c7da159a15-kube-api-access-jqph9\") pod \"test-curl-7403-3cbdb7\" (UID: \"b439bfca-283c-4bf4-8431-44c7da159a15\") " pod="opendatahub/test-curl-7403-3cbdb7"
Oct 30 22:25:40.794320 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:40.794283    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jqph9\" (UniqueName: \"kubernetes.io/projected/b439bfca-283c-4bf4-8431-44c7da159a15-kube-api-access-jqph9\") pod \"test-curl-7403-3cbdb7\" (UID: \"b439bfca-283c-4bf4-8431-44c7da159a15\") " pod="opendatahub/test-curl-7403-3cbdb7"
Oct 30 22:25:40.863229 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:40.863155    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jqph9\" (UniqueName: \"kubernetes.io/projected/b439bfca-283c-4bf4-8431-44c7da159a15-kube-api-access-jqph9\") pod \"test-curl-7403-3cbdb7\" (UID: \"b439bfca-283c-4bf4-8431-44c7da159a15\") " pod="opendatahub/test-curl-7403-3cbdb7"
Oct 30 22:25:40.900114 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:40.900085    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7403-3cbdb7"
Oct 30 22:25:41.290697 ip-10-0-136-210 kubenswrapper[5510]: W1030 22:25:41.290630    5510 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb439bfca_283c_4bf4_8431_44c7da159a15.slice/crio-b8983faff7c27c38b3586420b6d9c2281ead1263b7f425acb7f0937bcd508aba WatchSource:0}: Error finding container b8983faff7c27c38b3586420b6d9c2281ead1263b7f425acb7f0937bcd508aba: Status 404 returned error can't find the container with id b8983faff7c27c38b3586420b6d9c2281ead1263b7f425acb7f0937bcd508aba
Oct 30 22:25:41.292182 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:41.292100    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7403-3cbdb7"]
Oct 30 22:25:42.073679 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:42.069977    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:25:42.246764 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:42.246725    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7403-3cbdb7" event={"ID":"b439bfca-283c-4bf4-8431-44c7da159a15","Type":"ContainerStarted","Data":"2e93b5aba87b7aef14c23af6b52809ac8e86695776ea59c1e64c7e16ff4e80f1"}
Oct 30 22:25:42.246923 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:42.246780    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7403-3cbdb7" event={"ID":"b439bfca-283c-4bf4-8431-44c7da159a15","Type":"ContainerStarted","Data":"b8983faff7c27c38b3586420b6d9c2281ead1263b7f425acb7f0937bcd508aba"}
Oct 30 22:25:42.356554 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:42.356436    5510 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/test-curl-7403-3cbdb7" podStartSLOduration=2.356420315 podStartE2EDuration="2.356420315s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:25:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:25:42.353892525 +0000 UTC m=+2122.914702529" watchObservedRunningTime="2026-10-30 22:25:42.356420315 +0000 UTC m=+2122.917230319"
Oct 30 22:25:43.888512 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:43.888476    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7403-3cbdb7"]
Oct 30 22:25:43.893674 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:43.893630    5510 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7403-3cbdb7"]
Oct 30 22:25:44.054404 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:44.054370    5510 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7403-b38b41"]
Oct 30 22:25:44.055215 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:44.055198    5510 cpu_manager.go:414] "containerMap: removing container" podUID="b439bfca-283c-4bf4-8431-44c7da159a15" containerName="test-curl-7403-3cbdb7"
Oct 30 22:25:44.055264 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:44.055217    5510 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b439bfca-283c-4bf4-8431-44c7da159a15" containerName="test-curl-7403-3cbdb7"
Oct 30 22:25:44.055360 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:44.055350    5510 memory_manager.go:365] "RemoveStaleState removing state" podUID="b439bfca-283c-4bf4-8431-44c7da159a15" containerName="test-curl-7403-3cbdb7"
Oct 30 22:25:44.059746 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:44.059729    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7403-b38b41"
Oct 30 22:25:44.111740 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:44.111708    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7403-b38b41"]
Oct 30 22:25:44.146299 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:44.146219    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sv79g\" (UniqueName: \"kubernetes.io/projected/8697e52a-a2fb-4a10-8c1d-a47c1d85f396-kube-api-access-sv79g\") pod \"test-curl-7403-b38b41\" (UID: \"8697e52a-a2fb-4a10-8c1d-a47c1d85f396\") " pod="opendatahub/test-curl-7403-b38b41"
Oct 30 22:25:44.247536 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:44.247496    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sv79g\" (UniqueName: \"kubernetes.io/projected/8697e52a-a2fb-4a10-8c1d-a47c1d85f396-kube-api-access-sv79g\") pod \"test-curl-7403-b38b41\" (UID: \"8697e52a-a2fb-4a10-8c1d-a47c1d85f396\") " pod="opendatahub/test-curl-7403-b38b41"
Oct 30 22:25:44.262742 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:44.262702    5510 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/test-curl-7403-3cbdb7" podUID="b439bfca-283c-4bf4-8431-44c7da159a15" containerName="test-curl-7403-3cbdb7" containerID="cri-o://2e93b5aba87b7aef14c23af6b52809ac8e86695776ea59c1e64c7e16ff4e80f1" gracePeriod=2
Oct 30 22:25:44.309067 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:44.309040    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sv79g\" (UniqueName: \"kubernetes.io/projected/8697e52a-a2fb-4a10-8c1d-a47c1d85f396-kube-api-access-sv79g\") pod \"test-curl-7403-b38b41\" (UID: \"8697e52a-a2fb-4a10-8c1d-a47c1d85f396\") " pod="opendatahub/test-curl-7403-b38b41"
Oct 30 22:25:44.312673 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:25:44.312634    5510 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7403-3cbdb7\" is forbidden: User \"system:node:ip-10-0-136-210.us-west-2.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-136-210.us-west-2.compute.internal' and this object" podUID="b439bfca-283c-4bf4-8431-44c7da159a15" pod="opendatahub/test-curl-7403-3cbdb7"
Oct 30 22:25:44.372881 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:44.372843    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7403-b38b41"
Oct 30 22:25:44.756721 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:44.756056    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7403-b38b41"]
Oct 30 22:25:44.769442 ip-10-0-136-210 kubenswrapper[5510]: W1030 22:25:44.769412    5510 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8697e52a_a2fb_4a10_8c1d_a47c1d85f396.slice/crio-6f5caa42e85d5642b41eb34192a8f7f918f0a7de684146c3ab068dca02f0329e WatchSource:0}: Error finding container 6f5caa42e85d5642b41eb34192a8f7f918f0a7de684146c3ab068dca02f0329e: Status 404 returned error can't find the container with id 6f5caa42e85d5642b41eb34192a8f7f918f0a7de684146c3ab068dca02f0329e
Oct 30 22:25:45.154860 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:45.154825    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-0b6a3f-f4bfcfb9f-fvtq8"
Oct 30 22:25:45.272840 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:45.272798    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7403-b38b41" event={"ID":"8697e52a-a2fb-4a10-8c1d-a47c1d85f396","Type":"ContainerStarted","Data":"15ee8dcd1285e9209404a5b2993df6df003be47c6d50fd51cfb68bf851526e99"}
Oct 30 22:25:45.273056 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:45.272856    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7403-b38b41" event={"ID":"8697e52a-a2fb-4a10-8c1d-a47c1d85f396","Type":"ContainerStarted","Data":"6f5caa42e85d5642b41eb34192a8f7f918f0a7de684146c3ab068dca02f0329e"}
Oct 30 22:25:45.313949 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:25:45.313915    5510 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7403-3cbdb7\" is forbidden: User \"system:node:ip-10-0-136-210.us-west-2.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-136-210.us-west-2.compute.internal' and this object" podUID="b439bfca-283c-4bf4-8431-44c7da159a15" pod="opendatahub/test-curl-7403-3cbdb7"
Oct 30 22:25:45.362365 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:25:45.362327    5510 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7403-3cbdb7\" is forbidden: User \"system:node:ip-10-0-136-210.us-west-2.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-136-210.us-west-2.compute.internal' and this object" podUID="b439bfca-283c-4bf4-8431-44c7da159a15" pod="opendatahub/test-curl-7403-3cbdb7"
Oct 30 22:25:45.475752 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:45.475615    5510 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/test-curl-7403-b38b41" podStartSLOduration=1.475599843 podStartE2EDuration="1.475599843s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:25:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:25:45.473453113 +0000 UTC m=+2126.034263118" watchObservedRunningTime="2026-10-30 22:25:45.475599843 +0000 UTC m=+2126.036409849"
Oct 30 22:25:46.565529 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:46.565492    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:25:46.575850 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:46.575789    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7403-3cbdb7"
Oct 30 22:25:46.703680 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:46.703404    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b439bfca-283c-4bf4-8431-44c7da159a15-kube-api-access-jqph9\" (UniqueName: \"kubernetes.io/projected/b439bfca-283c-4bf4-8431-44c7da159a15-kube-api-access-jqph9\") pod \"b439bfca-283c-4bf4-8431-44c7da159a15\" (UID: \"b439bfca-283c-4bf4-8431-44c7da159a15\") "
Oct 30 22:25:46.718684 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:46.710884    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b439bfca-283c-4bf4-8431-44c7da159a15-kube-api-access-jqph9" pod "b439bfca-283c-4bf4-8431-44c7da159a15" (UID: "b439bfca-283c-4bf4-8431-44c7da159a15"). InnerVolumeSpecName "kube-api-access-jqph9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:25:46.773465 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:46.773430    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:25:46.806496 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:46.806459    5510 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jqph9\" (UniqueName: \"kubernetes.io/projected/b439bfca-283c-4bf4-8431-44c7da159a15-kube-api-access-jqph9\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:25:46.972768 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:46.972645    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:25:47.115851 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:47.115808    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7403-b38b41"]
Oct 30 22:25:47.130857 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:47.130814    5510 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7403-b38b41"]
Oct 30 22:25:47.289767 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:47.289733    5510 generic.go:356] "Generic (PLEG): container finished" podID="b439bfca-283c-4bf4-8431-44c7da159a15" containerID="2e93b5aba87b7aef14c23af6b52809ac8e86695776ea59c1e64c7e16ff4e80f1" exitCode=137
Oct 30 22:25:47.289939 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:47.289805    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7403-3cbdb7"
Oct 30 22:25:47.289939 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:47.289843    5510 scope.go:122] "RemoveContainer" containerID="2e93b5aba87b7aef14c23af6b52809ac8e86695776ea59c1e64c7e16ff4e80f1"
Oct 30 22:25:47.290118 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:47.290094    5510 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/test-curl-7403-b38b41" podUID="8697e52a-a2fb-4a10-8c1d-a47c1d85f396" containerName="test-curl-7403-b38b41" containerID="cri-o://15ee8dcd1285e9209404a5b2993df6df003be47c6d50fd51cfb68bf851526e99" gracePeriod=2
Oct 30 22:25:47.301315 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:47.301294    5510 scope.go:122] "RemoveContainer" containerID="2e93b5aba87b7aef14c23af6b52809ac8e86695776ea59c1e64c7e16ff4e80f1"
Oct 30 22:25:47.302120 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:25:47.302093    5510 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2e93b5aba87b7aef14c23af6b52809ac8e86695776ea59c1e64c7e16ff4e80f1\": container with ID starting with 2e93b5aba87b7aef14c23af6b52809ac8e86695776ea59c1e64c7e16ff4e80f1 not found: ID does not exist" containerID="2e93b5aba87b7aef14c23af6b52809ac8e86695776ea59c1e64c7e16ff4e80f1"
Oct 30 22:25:47.302221 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:47.302127    5510 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2e93b5aba87b7aef14c23af6b52809ac8e86695776ea59c1e64c7e16ff4e80f1"} err="failed to get container status \"2e93b5aba87b7aef14c23af6b52809ac8e86695776ea59c1e64c7e16ff4e80f1\": rpc error: code = NotFound desc = could not find container \"2e93b5aba87b7aef14c23af6b52809ac8e86695776ea59c1e64c7e16ff4e80f1\": container with ID starting with 2e93b5aba87b7aef14c23af6b52809ac8e86695776ea59c1e64c7e16ff4e80f1 not found: ID does not exist"
Oct 30 22:25:47.305118 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:47.305097    5510 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7403-3f2473"]
Oct 30 22:25:47.305938 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:47.305919    5510 cpu_manager.go:414] "containerMap: removing container" podUID="8697e52a-a2fb-4a10-8c1d-a47c1d85f396" containerName="test-curl-7403-b38b41"
Oct 30 22:25:47.306019 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:47.305940    5510 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8697e52a-a2fb-4a10-8c1d-a47c1d85f396" containerName="test-curl-7403-b38b41"
Oct 30 22:25:47.306095 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:47.306083    5510 memory_manager.go:365] "RemoveStaleState removing state" podUID="8697e52a-a2fb-4a10-8c1d-a47c1d85f396" containerName="test-curl-7403-b38b41"
Oct 30 22:25:47.311432 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:47.311414    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7403-3f2473"
Oct 30 22:25:47.339070 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:25:47.339038    5510 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7403-b38b41\" is forbidden: User \"system:node:ip-10-0-136-210.us-west-2.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-136-210.us-west-2.compute.internal' and this object" podUID="8697e52a-a2fb-4a10-8c1d-a47c1d85f396" pod="opendatahub/test-curl-7403-b38b41"
Oct 30 22:25:47.379278 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:47.379239    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7403-3f2473"]
Oct 30 22:25:47.387236 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:25:47.387192    5510 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7403-3cbdb7\" is forbidden: User \"system:node:ip-10-0-136-210.us-west-2.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-136-210.us-west-2.compute.internal' and this object" podUID="b439bfca-283c-4bf4-8431-44c7da159a15" pod="opendatahub/test-curl-7403-3cbdb7"
Oct 30 22:25:47.422407 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:47.422371    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rcg2n\" (UniqueName: \"kubernetes.io/projected/d08d0fc6-b14b-47f7-8eb0-0fedb3a86feb-kube-api-access-rcg2n\") pod \"test-curl-7403-3f2473\" (UID: \"d08d0fc6-b14b-47f7-8eb0-0fedb3a86feb\") " pod="opendatahub/test-curl-7403-3f2473"
Oct 30 22:25:47.435349 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:25:47.435307    5510 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7403-b38b41\" is forbidden: User \"system:node:ip-10-0-136-210.us-west-2.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-136-210.us-west-2.compute.internal' and this object" podUID="8697e52a-a2fb-4a10-8c1d-a47c1d85f396" pod="opendatahub/test-curl-7403-b38b41"
Oct 30 22:25:47.524530 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:47.524496    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rcg2n\" (UniqueName: \"kubernetes.io/projected/d08d0fc6-b14b-47f7-8eb0-0fedb3a86feb-kube-api-access-rcg2n\") pod \"test-curl-7403-3f2473\" (UID: \"d08d0fc6-b14b-47f7-8eb0-0fedb3a86feb\") " pod="opendatahub/test-curl-7403-3f2473"
Oct 30 22:25:47.555781 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:47.555699    5510 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b439bfca-283c-4bf4-8431-44c7da159a15" path="/var/lib/kubelet/pods/b439bfca-283c-4bf4-8431-44c7da159a15/volumes"
Oct 30 22:25:47.583091 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:47.583054    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rcg2n\" (UniqueName: \"kubernetes.io/projected/d08d0fc6-b14b-47f7-8eb0-0fedb3a86feb-kube-api-access-rcg2n\") pod \"test-curl-7403-3f2473\" (UID: \"d08d0fc6-b14b-47f7-8eb0-0fedb3a86feb\") " pod="opendatahub/test-curl-7403-3f2473"
Oct 30 22:25:47.625060 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:47.625026    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7403-3f2473"
Oct 30 22:25:48.004229 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:48.004187    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7403-3f2473"]
Oct 30 22:25:48.004966 ip-10-0-136-210 kubenswrapper[5510]: W1030 22:25:48.004936    5510 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd08d0fc6_b14b_47f7_8eb0_0fedb3a86feb.slice/crio-9c77aeb5af2468e3eeb67e6aa799c9830011dfcf566fd7b04273cd77abf4589a WatchSource:0}: Error finding container 9c77aeb5af2468e3eeb67e6aa799c9830011dfcf566fd7b04273cd77abf4589a: Status 404 returned error can't find the container with id 9c77aeb5af2468e3eeb67e6aa799c9830011dfcf566fd7b04273cd77abf4589a
Oct 30 22:25:48.299902 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:48.299865    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7403-3f2473" event={"ID":"d08d0fc6-b14b-47f7-8eb0-0fedb3a86feb","Type":"ContainerStarted","Data":"90c0773c860b50998371075335e49bca35ea67442d61ef3890a245e1fb44df52"}
Oct 30 22:25:48.300109 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:48.299918    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7403-3f2473" event={"ID":"d08d0fc6-b14b-47f7-8eb0-0fedb3a86feb","Type":"ContainerStarted","Data":"9c77aeb5af2468e3eeb67e6aa799c9830011dfcf566fd7b04273cd77abf4589a"}
Oct 30 22:25:48.409073 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:48.408915    5510 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/test-curl-7403-3f2473" podStartSLOduration=1.408897452 podStartE2EDuration="1.408897452s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:25:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:25:48.407826318 +0000 UTC m=+2128.968636323" watchObservedRunningTime="2026-10-30 22:25:48.408897452 +0000 UTC m=+2128.969707461"
Oct 30 22:25:49.584635 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:49.584607    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7403-b38b41"
Oct 30 22:25:49.633403 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:25:49.633358    5510 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7403-b38b41\" is forbidden: User \"system:node:ip-10-0-136-210.us-west-2.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-136-210.us-west-2.compute.internal' and this object" podUID="8697e52a-a2fb-4a10-8c1d-a47c1d85f396" pod="opendatahub/test-curl-7403-b38b41"
Oct 30 22:25:49.678840 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:49.678803    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8697e52a-a2fb-4a10-8c1d-a47c1d85f396-kube-api-access-sv79g\" (UniqueName: \"kubernetes.io/projected/8697e52a-a2fb-4a10-8c1d-a47c1d85f396-kube-api-access-sv79g\") pod \"8697e52a-a2fb-4a10-8c1d-a47c1d85f396\" (UID: \"8697e52a-a2fb-4a10-8c1d-a47c1d85f396\") "
Oct 30 22:25:49.681124 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:49.681097    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8697e52a-a2fb-4a10-8c1d-a47c1d85f396-kube-api-access-sv79g" pod "8697e52a-a2fb-4a10-8c1d-a47c1d85f396" (UID: "8697e52a-a2fb-4a10-8c1d-a47c1d85f396"). InnerVolumeSpecName "kube-api-access-sv79g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:25:49.781379 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:49.781335    5510 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sv79g\" (UniqueName: \"kubernetes.io/projected/8697e52a-a2fb-4a10-8c1d-a47c1d85f396-kube-api-access-sv79g\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:25:49.873000 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:49.872900    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7403-3f2473"]
Oct 30 22:25:49.878934 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:49.878897    5510 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7403-3f2473"]
Oct 30 22:25:50.048852 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:50.048819    5510 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7403-0364cf"]
Oct 30 22:25:50.049966 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:50.049946    5510 cpu_manager.go:414] "containerMap: removing container" podUID="d08d0fc6-b14b-47f7-8eb0-0fedb3a86feb" containerName="test-curl-7403-3f2473"
Oct 30 22:25:50.050023 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:50.049967    5510 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d08d0fc6-b14b-47f7-8eb0-0fedb3a86feb" containerName="test-curl-7403-3f2473"
Oct 30 22:25:50.050103 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:50.050094    5510 memory_manager.go:365] "RemoveStaleState removing state" podUID="d08d0fc6-b14b-47f7-8eb0-0fedb3a86feb" containerName="test-curl-7403-3f2473"
Oct 30 22:25:50.053869 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:50.053851    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7403-0364cf"
Oct 30 22:25:50.102410 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:25:50.102368    5510 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7403-b38b41\" is forbidden: User \"system:node:ip-10-0-136-210.us-west-2.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-136-210.us-west-2.compute.internal' and this object" podUID="8697e52a-a2fb-4a10-8c1d-a47c1d85f396" pod="opendatahub/test-curl-7403-b38b41"
Oct 30 22:25:50.115443 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:50.115405    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7403-0364cf"]
Oct 30 22:25:50.187794 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:50.187693    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7ddp5\" (UniqueName: \"kubernetes.io/projected/0cf97990-91b2-469c-be74-929c28af6ad9-kube-api-access-7ddp5\") pod \"test-curl-7403-0364cf\" (UID: \"0cf97990-91b2-469c-be74-929c28af6ad9\") " pod="opendatahub/test-curl-7403-0364cf"
Oct 30 22:25:50.290034 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:50.289987    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7ddp5\" (UniqueName: \"kubernetes.io/projected/0cf97990-91b2-469c-be74-929c28af6ad9-kube-api-access-7ddp5\") pod \"test-curl-7403-0364cf\" (UID: \"0cf97990-91b2-469c-be74-929c28af6ad9\") " pod="opendatahub/test-curl-7403-0364cf"
Oct 30 22:25:50.348838 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:50.348811    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7ddp5\" (UniqueName: \"kubernetes.io/projected/0cf97990-91b2-469c-be74-929c28af6ad9-kube-api-access-7ddp5\") pod \"test-curl-7403-0364cf\" (UID: \"0cf97990-91b2-469c-be74-929c28af6ad9\") " pod="opendatahub/test-curl-7403-0364cf"
Oct 30 22:25:50.366813 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:50.366782    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7403-0364cf"
Oct 30 22:25:50.367038 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:50.367014    5510 generic.go:356] "Generic (PLEG): container finished" podID="8697e52a-a2fb-4a10-8c1d-a47c1d85f396" containerID="15ee8dcd1285e9209404a5b2993df6df003be47c6d50fd51cfb68bf851526e99" exitCode=137
Oct 30 22:25:50.367092 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:50.367069    5510 scope.go:122] "RemoveContainer" containerID="15ee8dcd1285e9209404a5b2993df6df003be47c6d50fd51cfb68bf851526e99"
Oct 30 22:25:50.367141 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:50.367085    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7403-b38b41"
Oct 30 22:25:50.367313 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:50.367288    5510 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/test-curl-7403-3f2473" podUID="d08d0fc6-b14b-47f7-8eb0-0fedb3a86feb" containerName="test-curl-7403-3f2473" containerID="cri-o://90c0773c860b50998371075335e49bca35ea67442d61ef3890a245e1fb44df52" gracePeriod=2
Oct 30 22:25:50.392533 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:50.392508    5510 scope.go:122] "RemoveContainer" containerID="15ee8dcd1285e9209404a5b2993df6df003be47c6d50fd51cfb68bf851526e99"
Oct 30 22:25:50.392923 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:25:50.392896    5510 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"15ee8dcd1285e9209404a5b2993df6df003be47c6d50fd51cfb68bf851526e99\": container with ID starting with 15ee8dcd1285e9209404a5b2993df6df003be47c6d50fd51cfb68bf851526e99 not found: ID does not exist" containerID="15ee8dcd1285e9209404a5b2993df6df003be47c6d50fd51cfb68bf851526e99"
Oct 30 22:25:50.393009 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:50.392933    5510 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"15ee8dcd1285e9209404a5b2993df6df003be47c6d50fd51cfb68bf851526e99"} err="failed to get container status \"15ee8dcd1285e9209404a5b2993df6df003be47c6d50fd51cfb68bf851526e99\": rpc error: code = NotFound desc = could not find container \"15ee8dcd1285e9209404a5b2993df6df003be47c6d50fd51cfb68bf851526e99\": container with ID starting with 15ee8dcd1285e9209404a5b2993df6df003be47c6d50fd51cfb68bf851526e99 not found: ID does not exist"
Oct 30 22:25:50.416200 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:25:50.416154    5510 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7403-b38b41\" is forbidden: User \"system:node:ip-10-0-136-210.us-west-2.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-136-210.us-west-2.compute.internal' and this object" podUID="8697e52a-a2fb-4a10-8c1d-a47c1d85f396" pod="opendatahub/test-curl-7403-b38b41"
Oct 30 22:25:50.464333 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:25:50.464279    5510 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7403-3f2473\" is forbidden: User \"system:node:ip-10-0-136-210.us-west-2.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-136-210.us-west-2.compute.internal' and this object" podUID="d08d0fc6-b14b-47f7-8eb0-0fedb3a86feb" pod="opendatahub/test-curl-7403-3f2473"
Oct 30 22:25:50.512569 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:25:50.512526    5510 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7403-b38b41\" is forbidden: User \"system:node:ip-10-0-136-210.us-west-2.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-136-210.us-west-2.compute.internal' and this object" podUID="8697e52a-a2fb-4a10-8c1d-a47c1d85f396" pod="opendatahub/test-curl-7403-b38b41"
Oct 30 22:25:50.560305 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:25:50.560264    5510 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7403-3f2473\" is forbidden: User \"system:node:ip-10-0-136-210.us-west-2.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-136-210.us-west-2.compute.internal' and this object" podUID="d08d0fc6-b14b-47f7-8eb0-0fedb3a86feb" pod="opendatahub/test-curl-7403-3f2473"
Oct 30 22:25:50.761161 ip-10-0-136-210 kubenswrapper[5510]: W1030 22:25:50.761128    5510 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0cf97990_91b2_469c_be74_929c28af6ad9.slice/crio-083fef1763c9e7b4e5d2545bde44ef19e3eb165074a6a63635d95f02c15e02a7 WatchSource:0}: Error finding container 083fef1763c9e7b4e5d2545bde44ef19e3eb165074a6a63635d95f02c15e02a7: Status 404 returned error can't find the container with id 083fef1763c9e7b4e5d2545bde44ef19e3eb165074a6a63635d95f02c15e02a7
Oct 30 22:25:50.761809 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:50.761783    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7403-0364cf"]
Oct 30 22:25:51.362556 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:51.361583    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:25:51.386310 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:51.386274    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7403-0364cf" event={"ID":"0cf97990-91b2-469c-be74-929c28af6ad9","Type":"ContainerStarted","Data":"da94fd76e781d1aa33e15d725d7e7bef42a183b9ab2696b552621f20ec9e16d5"}
Oct 30 22:25:51.386470 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:51.386321    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7403-0364cf" event={"ID":"0cf97990-91b2-469c-be74-929c28af6ad9","Type":"ContainerStarted","Data":"083fef1763c9e7b4e5d2545bde44ef19e3eb165074a6a63635d95f02c15e02a7"}
Oct 30 22:25:51.434521 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:25:51.434477    5510 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7403-b38b41\" is forbidden: User \"system:node:ip-10-0-136-210.us-west-2.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-136-210.us-west-2.compute.internal' and this object" podUID="8697e52a-a2fb-4a10-8c1d-a47c1d85f396" pod="opendatahub/test-curl-7403-b38b41"
Oct 30 22:25:51.482803 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:25:51.482766    5510 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7403-3f2473\" is forbidden: User \"system:node:ip-10-0-136-210.us-west-2.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-136-210.us-west-2.compute.internal' and this object" podUID="d08d0fc6-b14b-47f7-8eb0-0fedb3a86feb" pod="opendatahub/test-curl-7403-3f2473"
Oct 30 22:25:51.555217 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:51.555182    5510 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8697e52a-a2fb-4a10-8c1d-a47c1d85f396" path="/var/lib/kubelet/pods/8697e52a-a2fb-4a10-8c1d-a47c1d85f396/volumes"
Oct 30 22:25:51.596092 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:51.596035    5510 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/test-curl-7403-0364cf" podStartSLOduration=2.59602061 podStartE2EDuration="2.59602061s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:25:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:25:51.592197355 +0000 UTC m=+2132.153007361" watchObservedRunningTime="2026-10-30 22:25:51.59602061 +0000 UTC m=+2132.156830615"
Oct 30 22:25:52.592634 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:52.592607    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7403-3f2473"
Oct 30 22:25:52.725929 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:52.725836    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d08d0fc6-b14b-47f7-8eb0-0fedb3a86feb-kube-api-access-rcg2n\" (UniqueName: \"kubernetes.io/projected/d08d0fc6-b14b-47f7-8eb0-0fedb3a86feb-kube-api-access-rcg2n\") pod \"d08d0fc6-b14b-47f7-8eb0-0fedb3a86feb\" (UID: \"d08d0fc6-b14b-47f7-8eb0-0fedb3a86feb\") "
Oct 30 22:25:52.727972 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:52.727951    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d08d0fc6-b14b-47f7-8eb0-0fedb3a86feb-kube-api-access-rcg2n" pod "d08d0fc6-b14b-47f7-8eb0-0fedb3a86feb" (UID: "d08d0fc6-b14b-47f7-8eb0-0fedb3a86feb"). InnerVolumeSpecName "kube-api-access-rcg2n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:25:52.828227 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:52.828189    5510 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rcg2n\" (UniqueName: \"kubernetes.io/projected/d08d0fc6-b14b-47f7-8eb0-0fedb3a86feb-kube-api-access-rcg2n\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:25:53.209903 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:53.209858    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7403-0364cf"]
Oct 30 22:25:53.215864 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:53.215832    5510 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7403-0364cf"]
Oct 30 22:25:53.402899 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:53.402858    5510 generic.go:356] "Generic (PLEG): container finished" podID="d08d0fc6-b14b-47f7-8eb0-0fedb3a86feb" containerID="90c0773c860b50998371075335e49bca35ea67442d61ef3890a245e1fb44df52" exitCode=137
Oct 30 22:25:53.403051 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:53.402945    5510 scope.go:122] "RemoveContainer" containerID="90c0773c860b50998371075335e49bca35ea67442d61ef3890a245e1fb44df52"
Oct 30 22:25:53.403051 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:53.402949    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7403-3f2473"
Oct 30 22:25:53.403051 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:53.403024    5510 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/test-curl-7403-0364cf" podUID="0cf97990-91b2-469c-be74-929c28af6ad9" containerName="test-curl-7403-0364cf" containerID="cri-o://da94fd76e781d1aa33e15d725d7e7bef42a183b9ab2696b552621f20ec9e16d5" gracePeriod=2
Oct 30 22:25:53.415046 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:53.415023    5510 scope.go:122] "RemoveContainer" containerID="90c0773c860b50998371075335e49bca35ea67442d61ef3890a245e1fb44df52"
Oct 30 22:25:53.415322 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:25:53.415302    5510 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"90c0773c860b50998371075335e49bca35ea67442d61ef3890a245e1fb44df52\": container with ID starting with 90c0773c860b50998371075335e49bca35ea67442d61ef3890a245e1fb44df52 not found: ID does not exist" containerID="90c0773c860b50998371075335e49bca35ea67442d61ef3890a245e1fb44df52"
Oct 30 22:25:53.415382 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:53.415329    5510 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"90c0773c860b50998371075335e49bca35ea67442d61ef3890a245e1fb44df52"} err="failed to get container status \"90c0773c860b50998371075335e49bca35ea67442d61ef3890a245e1fb44df52\": rpc error: code = NotFound desc = could not find container \"90c0773c860b50998371075335e49bca35ea67442d61ef3890a245e1fb44df52\": container with ID starting with 90c0773c860b50998371075335e49bca35ea67442d61ef3890a245e1fb44df52 not found: ID does not exist"
Oct 30 22:25:53.453482 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:25:53.453448    5510 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7403-0364cf\" is forbidden: User \"system:node:ip-10-0-136-210.us-west-2.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-136-210.us-west-2.compute.internal' and this object" podUID="0cf97990-91b2-469c-be74-929c28af6ad9" pod="opendatahub/test-curl-7403-0364cf"
Oct 30 22:25:53.532043 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:25:53.532001    5510 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7403-3f2473\" is forbidden: User \"system:node:ip-10-0-136-210.us-west-2.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-136-210.us-west-2.compute.internal' and this object" podUID="d08d0fc6-b14b-47f7-8eb0-0fedb3a86feb" pod="opendatahub/test-curl-7403-3f2473"
Oct 30 22:25:53.556846 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:53.556817    5510 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d08d0fc6-b14b-47f7-8eb0-0fedb3a86feb" path="/var/lib/kubelet/pods/d08d0fc6-b14b-47f7-8eb0-0fedb3a86feb/volumes"
Oct 30 22:25:53.580475 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:25:53.580431    5510 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7403-0364cf\" is forbidden: User \"system:node:ip-10-0-136-210.us-west-2.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-136-210.us-west-2.compute.internal' and this object" podUID="0cf97990-91b2-469c-be74-929c28af6ad9" pod="opendatahub/test-curl-7403-0364cf"
Oct 30 22:25:54.166883 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:54.166848    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:25:55.553456 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:55.553281    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:25:55.831924 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:55.831886    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:25:55.832213 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:55.832195    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7403-0364cf"
Oct 30 22:25:55.862593 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:55.862559    5510 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-9f3ea1-65cfd5854b-lcwjw"]
Oct 30 22:25:55.863442 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:55.863422    5510 cpu_manager.go:414] "containerMap: removing container" podUID="0cf97990-91b2-469c-be74-929c28af6ad9" containerName="test-curl-7403-0364cf"
Oct 30 22:25:55.863498 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:55.863445    5510 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0cf97990-91b2-469c-be74-929c28af6ad9" containerName="test-curl-7403-0364cf"
Oct 30 22:25:55.863604 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:55.863595    5510 memory_manager.go:365] "RemoveStaleState removing state" podUID="0cf97990-91b2-469c-be74-929c28af6ad9" containerName="test-curl-7403-0364cf"
Oct 30 22:25:55.867449 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:55.867429    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-9f3ea1-65cfd5854b-lcwjw"
Oct 30 22:25:55.919506 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:55.919410    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w2-9f3ea1-serving-cert\""
Oct 30 22:25:55.924477 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:55.924449    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-9f3ea1-65cfd5854b-lcwjw"]
Oct 30 22:25:55.984584 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:55.984546    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0cf97990-91b2-469c-be74-929c28af6ad9-kube-api-access-7ddp5\" (UniqueName: \"kubernetes.io/projected/0cf97990-91b2-469c-be74-929c28af6ad9-kube-api-access-7ddp5\") pod \"0cf97990-91b2-469c-be74-929c28af6ad9\" (UID: \"0cf97990-91b2-469c-be74-929c28af6ad9\") "
Oct 30 22:25:55.984999 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:55.984975    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/743f9a1e-c695-454d-823d-7939bfbb6aff-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-9f3ea1-65cfd5854b-lcwjw\" (UID: \"743f9a1e-c695-454d-823d-7939bfbb6aff\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-9f3ea1-65cfd5854b-lcwjw"
Oct 30 22:25:55.985263 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:55.985245    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v2gzl\" (UniqueName: \"kubernetes.io/projected/743f9a1e-c695-454d-823d-7939bfbb6aff-kube-api-access-v2gzl\") pod \"maas-api-e2e-shared-b-w2-9f3ea1-65cfd5854b-lcwjw\" (UID: \"743f9a1e-c695-454d-823d-7939bfbb6aff\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-9f3ea1-65cfd5854b-lcwjw"
Oct 30 22:25:55.985371 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:55.985278    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/743f9a1e-c695-454d-823d-7939bfbb6aff-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-9f3ea1-65cfd5854b-lcwjw\" (UID: \"743f9a1e-c695-454d-823d-7939bfbb6aff\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-9f3ea1-65cfd5854b-lcwjw"
Oct 30 22:25:55.985428 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:55.985395    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/743f9a1e-c695-454d-823d-7939bfbb6aff-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-9f3ea1-65cfd5854b-lcwjw\" (UID: \"743f9a1e-c695-454d-823d-7939bfbb6aff\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-9f3ea1-65cfd5854b-lcwjw"
Oct 30 22:25:55.986903 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:55.986876    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0cf97990-91b2-469c-be74-929c28af6ad9-kube-api-access-7ddp5" pod "0cf97990-91b2-469c-be74-929c28af6ad9" (UID: "0cf97990-91b2-469c-be74-929c28af6ad9"). InnerVolumeSpecName "kube-api-access-7ddp5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:25:56.087014 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:56.086979    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/743f9a1e-c695-454d-823d-7939bfbb6aff-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-9f3ea1-65cfd5854b-lcwjw\" (UID: \"743f9a1e-c695-454d-823d-7939bfbb6aff\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-9f3ea1-65cfd5854b-lcwjw"
Oct 30 22:25:56.087182 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:56.087098    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/743f9a1e-c695-454d-823d-7939bfbb6aff-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-9f3ea1-65cfd5854b-lcwjw\" (UID: \"743f9a1e-c695-454d-823d-7939bfbb6aff\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-9f3ea1-65cfd5854b-lcwjw"
Oct 30 22:25:56.087353 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:56.087331    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v2gzl\" (UniqueName: \"kubernetes.io/projected/743f9a1e-c695-454d-823d-7939bfbb6aff-kube-api-access-v2gzl\") pod \"maas-api-e2e-shared-b-w2-9f3ea1-65cfd5854b-lcwjw\" (UID: \"743f9a1e-c695-454d-823d-7939bfbb6aff\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-9f3ea1-65cfd5854b-lcwjw"
Oct 30 22:25:56.087398 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:56.087370    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/743f9a1e-c695-454d-823d-7939bfbb6aff-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-9f3ea1-65cfd5854b-lcwjw\" (UID: \"743f9a1e-c695-454d-823d-7939bfbb6aff\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-9f3ea1-65cfd5854b-lcwjw"
Oct 30 22:25:56.087509 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:56.087493    5510 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7ddp5\" (UniqueName: \"kubernetes.io/projected/0cf97990-91b2-469c-be74-929c28af6ad9-kube-api-access-7ddp5\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:25:56.088061 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:56.088035    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/743f9a1e-c695-454d-823d-7939bfbb6aff-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-9f3ea1-65cfd5854b-lcwjw\" (UID: \"743f9a1e-c695-454d-823d-7939bfbb6aff\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-9f3ea1-65cfd5854b-lcwjw"
Oct 30 22:25:56.089649 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:56.089630    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/743f9a1e-c695-454d-823d-7939bfbb6aff-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-9f3ea1-65cfd5854b-lcwjw\" (UID: \"743f9a1e-c695-454d-823d-7939bfbb6aff\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-9f3ea1-65cfd5854b-lcwjw"
Oct 30 22:25:56.089649 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:56.089635    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/743f9a1e-c695-454d-823d-7939bfbb6aff-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-9f3ea1-65cfd5854b-lcwjw\" (UID: \"743f9a1e-c695-454d-823d-7939bfbb6aff\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-9f3ea1-65cfd5854b-lcwjw"
Oct 30 22:25:56.095179 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:56.095146    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:25:56.148704 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:56.148675    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v2gzl\" (UniqueName: \"kubernetes.io/projected/743f9a1e-c695-454d-823d-7939bfbb6aff-kube-api-access-v2gzl\") pod \"maas-api-e2e-shared-b-w2-9f3ea1-65cfd5854b-lcwjw\" (UID: \"743f9a1e-c695-454d-823d-7939bfbb6aff\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-9f3ea1-65cfd5854b-lcwjw"
Oct 30 22:25:56.183401 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:56.183324    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-9f3ea1-65cfd5854b-lcwjw"
Oct 30 22:25:56.327024 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:56.326952    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:25:56.436763 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:56.436634    5510 generic.go:356] "Generic (PLEG): container finished" podID="0cf97990-91b2-469c-be74-929c28af6ad9" containerID="da94fd76e781d1aa33e15d725d7e7bef42a183b9ab2696b552621f20ec9e16d5" exitCode=137
Oct 30 22:25:56.436908 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:56.436769    5510 scope.go:122] "RemoveContainer" containerID="da94fd76e781d1aa33e15d725d7e7bef42a183b9ab2696b552621f20ec9e16d5"
Oct 30 22:25:56.436950 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:56.436920    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7403-0364cf"
Oct 30 22:25:56.450971 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:56.450950    5510 scope.go:122] "RemoveContainer" containerID="da94fd76e781d1aa33e15d725d7e7bef42a183b9ab2696b552621f20ec9e16d5"
Oct 30 22:25:56.451294 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:25:56.451272    5510 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"da94fd76e781d1aa33e15d725d7e7bef42a183b9ab2696b552621f20ec9e16d5\": container with ID starting with da94fd76e781d1aa33e15d725d7e7bef42a183b9ab2696b552621f20ec9e16d5 not found: ID does not exist" containerID="da94fd76e781d1aa33e15d725d7e7bef42a183b9ab2696b552621f20ec9e16d5"
Oct 30 22:25:56.451356 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:56.451306    5510 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"da94fd76e781d1aa33e15d725d7e7bef42a183b9ab2696b552621f20ec9e16d5"} err="failed to get container status \"da94fd76e781d1aa33e15d725d7e7bef42a183b9ab2696b552621f20ec9e16d5\": rpc error: code = NotFound desc = could not find container \"da94fd76e781d1aa33e15d725d7e7bef42a183b9ab2696b552621f20ec9e16d5\": container with ID starting with da94fd76e781d1aa33e15d725d7e7bef42a183b9ab2696b552621f20ec9e16d5 not found: ID does not exist"
Oct 30 22:25:56.498203 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:25:56.498162    5510 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7403-0364cf\" is forbidden: User \"system:node:ip-10-0-136-210.us-west-2.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-136-210.us-west-2.compute.internal' and this object" podUID="0cf97990-91b2-469c-be74-929c28af6ad9" pod="opendatahub/test-curl-7403-0364cf"
Oct 30 22:25:56.630505 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:56.630468    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-9f3ea1-65cfd5854b-lcwjw"]
Oct 30 22:25:56.634385 ip-10-0-136-210 kubenswrapper[5510]: W1030 22:25:56.634356    5510 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod743f9a1e_c695_454d_823d_7939bfbb6aff.slice/crio-46ec1e12dc3486f81d06e592d272c166a7a6b37b4f1c2582a56df73f0d985983 WatchSource:0}: Error finding container 46ec1e12dc3486f81d06e592d272c166a7a6b37b4f1c2582a56df73f0d985983: Status 404 returned error can't find the container with id 46ec1e12dc3486f81d06e592d272c166a7a6b37b4f1c2582a56df73f0d985983
Oct 30 22:25:56.635249 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:56.635230    5510 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 22:25:57.449480 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:57.449441    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-9f3ea1-65cfd5854b-lcwjw" event={"ID":"743f9a1e-c695-454d-823d-7939bfbb6aff","Type":"ContainerStarted","Data":"46ec1e12dc3486f81d06e592d272c166a7a6b37b4f1c2582a56df73f0d985983"}
Oct 30 22:25:57.556147 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:57.556120    5510 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0cf97990-91b2-469c-be74-929c28af6ad9" path="/var/lib/kubelet/pods/0cf97990-91b2-469c-be74-929c28af6ad9/volumes"
Oct 30 22:25:58.459257 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:58.459224    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-9f3ea1-65cfd5854b-lcwjw" event={"ID":"743f9a1e-c695-454d-823d-7939bfbb6aff","Type":"ContainerStarted","Data":"9d9ffef4ecd0bda9ca49728eb104e67aef8a77a96c6c2a27a7b7a50ced01ea84"}
Oct 30 22:25:58.459621 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:58.459314    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-9f3ea1-65cfd5854b-lcwjw"
Oct 30 22:25:58.573931 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:25:58.573870    5510 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-9f3ea1-65cfd5854b-lcwjw" podStartSLOduration=2.674032973 podStartE2EDuration="3.573850226s" totalImagesPullingTime="899.817253ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:25:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:25:58.572146062 +0000 UTC m=+2139.132956067" watchObservedRunningTime="2026-10-30 22:25:58.573850226 +0000 UTC m=+2139.134660232"
Oct 30 22:26:00.019250 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:26:00.019207    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:26:00.274411 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:26:00.274306    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:26:01.109682 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:26:01.106198    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:26:03.219057 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:26:03.219020    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:26:03.425980 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:26:03.425062    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:26:03.650850 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:26:03.650810    5510 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-595fd7d7-5d9b87ff4c-x2njg"]
Oct 30 22:26:03.656380 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:26:03.656354    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-595fd7d7-5d9b87ff4c-x2njg"
Oct 30 22:26:03.714367 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:26:03.714329    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-595fd7d7-5d9b87ff4c-x2njg"]
Oct 30 22:26:03.756270 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:26:03.756070    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-595fd7d7-dockercfg-qmxxm\""
Oct 30 22:26:03.756270 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:26:03.756055    5510 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-595fd7d7\""
Oct 30 22:26:03.786266 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:26:03.786133    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b112302e-a8df-4573-a151-696453d1e7ee-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-595fd7d7-5d9b87ff4c-x2njg\" (UID: \"b112302e-a8df-4573-a151-696453d1e7ee\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-595fd7d7-5d9b87ff4c-x2njg"
Oct 30 22:26:03.786266 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:26:03.786240    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g85st\" (UniqueName: \"kubernetes.io/projected/b112302e-a8df-4573-a151-696453d1e7ee-kube-api-access-g85st\") pod \"payload-pre-processing-e2e-mt-595fd7d7-5d9b87ff4c-x2njg\" (UID: \"b112302e-a8df-4573-a151-696453d1e7ee\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-595fd7d7-5d9b87ff4c-x2njg"
Oct 30 22:26:03.915797 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:26:03.914168    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b112302e-a8df-4573-a151-696453d1e7ee-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-595fd7d7-5d9b87ff4c-x2njg\" (UID: \"b112302e-a8df-4573-a151-696453d1e7ee\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-595fd7d7-5d9b87ff4c-x2njg"
Oct 30 22:26:03.915797 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:26:03.914270    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g85st\" (UniqueName: \"kubernetes.io/projected/b112302e-a8df-4573-a151-696453d1e7ee-kube-api-access-g85st\") pod \"payload-pre-processing-e2e-mt-595fd7d7-5d9b87ff4c-x2njg\" (UID: \"b112302e-a8df-4573-a151-696453d1e7ee\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-595fd7d7-5d9b87ff4c-x2njg"
Oct 30 22:26:03.915797 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:26:03.915570    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b112302e-a8df-4573-a151-696453d1e7ee-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-595fd7d7-5d9b87ff4c-x2njg\" (UID: \"b112302e-a8df-4573-a151-696453d1e7ee\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-595fd7d7-5d9b87ff4c-x2njg"
Oct 30 22:26:03.973821 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:26:03.973786    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g85st\" (UniqueName: \"kubernetes.io/projected/b112302e-a8df-4573-a151-696453d1e7ee-kube-api-access-g85st\") pod \"payload-pre-processing-e2e-mt-595fd7d7-5d9b87ff4c-x2njg\" (UID: \"b112302e-a8df-4573-a151-696453d1e7ee\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-595fd7d7-5d9b87ff4c-x2njg"
Oct 30 22:26:04.272079 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:26:04.272051    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-595fd7d7-5d9b87ff4c-x2njg"
Oct 30 22:26:04.473510 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:26:04.473481    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-9f3ea1-65cfd5854b-lcwjw"
Oct 30 22:26:04.653789 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:26:04.653742    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-595fd7d7-5d9b87ff4c-x2njg"]
Oct 30 22:26:04.654488 ip-10-0-136-210 kubenswrapper[5510]: W1030 22:26:04.654446    5510 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb112302e_a8df_4573_a151_696453d1e7ee.slice/crio-21de033cee9c5396d8c8c35ead3ba9b67b9660f3807a3f8483adcdc2e00fe47d WatchSource:0}: Error finding container 21de033cee9c5396d8c8c35ead3ba9b67b9660f3807a3f8483adcdc2e00fe47d: Status 404 returned error can't find the container with id 21de033cee9c5396d8c8c35ead3ba9b67b9660f3807a3f8483adcdc2e00fe47d
Oct 30 22:26:05.599290 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:26:05.599249    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-595fd7d7-5d9b87ff4c-x2njg" event={"ID":"b112302e-a8df-4573-a151-696453d1e7ee","Type":"ContainerStarted","Data":"937ca6f5a81d90209fc1dac15333e6b99b32e184cca16a13ddbb4c989e17306d"}
Oct 30 22:26:05.600970 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:26:05.599318    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-595fd7d7-5d9b87ff4c-x2njg" event={"ID":"b112302e-a8df-4573-a151-696453d1e7ee","Type":"ContainerStarted","Data":"21de033cee9c5396d8c8c35ead3ba9b67b9660f3807a3f8483adcdc2e00fe47d"}
Oct 30 22:26:05.600970 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:26:05.599585    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-595fd7d7-5d9b87ff4c-x2njg"
Oct 30 22:26:05.714823 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:26:05.714767    5510 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-595fd7d7-5d9b87ff4c-x2njg" podStartSLOduration=2.714752011 podStartE2EDuration="2.714752011s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:26:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:26:05.710516632 +0000 UTC m=+2146.271326640" watchObservedRunningTime="2026-10-30 22:26:05.714752011 +0000 UTC m=+2146.275562084"
Oct 30 22:26:06.347637 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:26:06.347601    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:26:08.421457 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:26:08.421415    5510 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6gjbf" containerName="extractor"
Oct 30 22:26:10.861686 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:26:10.859820    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:26:15.367683 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:26:15.360499    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:26:16.610622 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:26:16.610591    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-595fd7d7-5d9b87ff4c-x2njg"
Oct 30 22:26:19.974684 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:26:19.973779    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:26:20.173534 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:26:20.173494    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:26:20.376393 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:26:20.376356    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:26:22.823002 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:26:22.822972    5510 scope.go:122] "RemoveContainer" containerID="282e85b9efc1a787703d885bb0cfb4a5260dff41039625bdf4d03942d02f3597"
Oct 30 22:26:24.848680 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:26:24.843376    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:26:29.034902 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:26:29.034864    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:26:29.261031 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:26:29.260999    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:26:29.604409 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:26:29.604368    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:26:30.870565 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:26:30.870531    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:26:34.432862 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:26:34.431164    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:26:38.524364 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:26:38.524328    5510 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6gjbf" containerName="extractor"
Oct 30 22:26:39.036869 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:26:39.036836    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:26:39.301351 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:26:39.301263    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:26:39.520283 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:26:39.520245    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:26:43.805551 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:26:43.805512    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:26:48.337685 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:26:48.332427    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:26:53.045611 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:26:53.045571    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:26:57.621877 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:26:57.621771    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:27:02.044760 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:27:02.044721    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:27:02.255359 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:27:02.255308    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:27:02.449592 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:27:02.449504    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:27:06.655651 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:27:06.655323    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:27:06.875621 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:27:06.875581    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:27:08.421828 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:27:08.421791    5510 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6gjbf" containerName="extractor"
Oct 30 22:27:11.337699 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:27:11.337051    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:27:15.914922 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:27:15.914865    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:27:20.564683 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:27:20.560694    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:27:25.663682 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:27:25.650550    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:27:28.368949 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:27:28.368911    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-5637a7-6db64b9ddc-lqdjr"]
Oct 30 22:27:28.468902 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:27:28.468864    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-5637a7-6dfc655fbc-ms5sb"]
Oct 30 22:27:28.765728 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:27:28.765635    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:27:29.081098 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:27:29.081051    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:27:29.831366 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:27:29.831331    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:27:31.089966 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:27:31.089933    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:27:31.495235 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:27:31.494346    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:27:32.577984 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:27:32.577950    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-cf1826-5f6ddbf4c7-7mrgs"]
Oct 30 22:27:32.680525 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:27:32.680490    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-cf1826-556566884c-krgkh"]
Oct 30 22:27:33.025060 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:27:33.024966    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:27:33.226987 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:27:33.226946    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:27:33.369547 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:27:33.369427    5510 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-a-5637a7-6db64b9ddc-lqdjr" podUID="2c98b4d1-9cc0-4482-af1e-1038eaeb1e61" containerName="payload-processing" containerID="cri-o://82f0cc65e4a9cfe93229b93978d1daad54feaea2e2ff58a2e7e97a25977284bc" gracePeriod=25
Oct 30 22:27:33.420170 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:27:33.420119    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:27:33.469282 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:27:33.469242    5510 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-5637a7-6dfc655fbc-ms5sb" podUID="4009806a-5d15-4224-9b63-e3bc115f35ad" containerName="payload-pre-processing" containerID="cri-o://7df82eb9f5f8346aaf90acc553463487049f56e37957a84fdf69d2a6fc69ba9d" gracePeriod=25
Oct 30 22:27:33.534733 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:27:33.534693    5510 generic.go:356] "Generic (PLEG): container finished" podID="2c98b4d1-9cc0-4482-af1e-1038eaeb1e61" containerID="82f0cc65e4a9cfe93229b93978d1daad54feaea2e2ff58a2e7e97a25977284bc" exitCode=0
Oct 30 22:27:33.534904 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:27:33.534772    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-5637a7-6db64b9ddc-lqdjr" event={"ID":"2c98b4d1-9cc0-4482-af1e-1038eaeb1e61","Type":"ContainerDied","Data":"82f0cc65e4a9cfe93229b93978d1daad54feaea2e2ff58a2e7e97a25977284bc"}
Oct 30 22:27:33.665485 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:27:33.665450    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-5637a7-6db64b9ddc-lqdjr"
Oct 30 22:27:33.767132 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:27:33.767103    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2c98b4d1-9cc0-4482-af1e-1038eaeb1e61-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2c98b4d1-9cc0-4482-af1e-1038eaeb1e61-plugins-config-volume\") pod \"2c98b4d1-9cc0-4482-af1e-1038eaeb1e61\" (UID: \"2c98b4d1-9cc0-4482-af1e-1038eaeb1e61\") "
Oct 30 22:27:33.767304 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:27:33.767271    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2c98b4d1-9cc0-4482-af1e-1038eaeb1e61-kube-api-access-cngq7\" (UniqueName: \"kubernetes.io/projected/2c98b4d1-9cc0-4482-af1e-1038eaeb1e61-kube-api-access-cngq7\") pod \"2c98b4d1-9cc0-4482-af1e-1038eaeb1e61\" (UID: \"2c98b4d1-9cc0-4482-af1e-1038eaeb1e61\") "
Oct 30 22:27:33.767513 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:27:33.767490    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2c98b4d1-9cc0-4482-af1e-1038eaeb1e61-plugins-config-volume" pod "2c98b4d1-9cc0-4482-af1e-1038eaeb1e61" (UID: "2c98b4d1-9cc0-4482-af1e-1038eaeb1e61"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:27:33.767919 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:27:33.767903    5510 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2c98b4d1-9cc0-4482-af1e-1038eaeb1e61-plugins-config-volume\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:27:33.769323 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:27:33.769301    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2c98b4d1-9cc0-4482-af1e-1038eaeb1e61-kube-api-access-cngq7" pod "2c98b4d1-9cc0-4482-af1e-1038eaeb1e61" (UID: "2c98b4d1-9cc0-4482-af1e-1038eaeb1e61"). InnerVolumeSpecName "kube-api-access-cngq7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:27:33.777879 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:27:33.777858    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-5637a7-6dfc655fbc-ms5sb"
Oct 30 22:27:33.869735 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:27:33.869699    5510 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cngq7\" (UniqueName: \"kubernetes.io/projected/2c98b4d1-9cc0-4482-af1e-1038eaeb1e61-kube-api-access-cngq7\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:27:34.004220 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:27:34.004108    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4009806a-5d15-4224-9b63-e3bc115f35ad-kube-api-access-6mdh4\" (UniqueName: \"kubernetes.io/projected/4009806a-5d15-4224-9b63-e3bc115f35ad-kube-api-access-6mdh4\") pod \"4009806a-5d15-4224-9b63-e3bc115f35ad\" (UID: \"4009806a-5d15-4224-9b63-e3bc115f35ad\") "
Oct 30 22:27:34.004413 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:27:34.004299    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4009806a-5d15-4224-9b63-e3bc115f35ad-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4009806a-5d15-4224-9b63-e3bc115f35ad-plugins-config-volume\") pod \"4009806a-5d15-4224-9b63-e3bc115f35ad\" (UID: \"4009806a-5d15-4224-9b63-e3bc115f35ad\") "
Oct 30 22:27:34.005551 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:27:34.005511    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4009806a-5d15-4224-9b63-e3bc115f35ad-plugins-config-volume" pod "4009806a-5d15-4224-9b63-e3bc115f35ad" (UID: "4009806a-5d15-4224-9b63-e3bc115f35ad"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:27:34.025881 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:27:34.025844    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4009806a-5d15-4224-9b63-e3bc115f35ad-kube-api-access-6mdh4" pod "4009806a-5d15-4224-9b63-e3bc115f35ad" (UID: "4009806a-5d15-4224-9b63-e3bc115f35ad"). InnerVolumeSpecName "kube-api-access-6mdh4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:27:34.099681 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:27:34.099171    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:27:34.125679 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:27:34.113224    5510 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4009806a-5d15-4224-9b63-e3bc115f35ad-plugins-config-volume\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:27:34.125679 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:27:34.113261    5510 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6mdh4\" (UniqueName: \"kubernetes.io/projected/4009806a-5d15-4224-9b63-e3bc115f35ad-kube-api-access-6mdh4\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:27:34.543791 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:27:34.543749    5510 generic.go:356] "Generic (PLEG): container finished" podID="4009806a-5d15-4224-9b63-e3bc115f35ad" containerID="7df82eb9f5f8346aaf90acc553463487049f56e37957a84fdf69d2a6fc69ba9d" exitCode=0
Oct 30 22:27:34.543988 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:27:34.543805    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-5637a7-6dfc655fbc-ms5sb"
Oct 30 22:27:34.543988 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:27:34.543837    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-5637a7-6dfc655fbc-ms5sb" event={"ID":"4009806a-5d15-4224-9b63-e3bc115f35ad","Type":"ContainerDied","Data":"7df82eb9f5f8346aaf90acc553463487049f56e37957a84fdf69d2a6fc69ba9d"}
Oct 30 22:27:34.543988 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:27:34.543890    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-5637a7-6dfc655fbc-ms5sb" event={"ID":"4009806a-5d15-4224-9b63-e3bc115f35ad","Type":"ContainerDied","Data":"5fe8665ad3c776e037a95fba04166e8175c2a339369438ad18433cf63074424e"}
Oct 30 22:27:34.543988 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:27:34.543918    5510 scope.go:122] "RemoveContainer" containerID="7df82eb9f5f8346aaf90acc553463487049f56e37957a84fdf69d2a6fc69ba9d"
Oct 30 22:27:34.545547 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:27:34.545518    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-5637a7-6db64b9ddc-lqdjr" event={"ID":"2c98b4d1-9cc0-4482-af1e-1038eaeb1e61","Type":"ContainerDied","Data":"bd976e5f80650973cd3f87e38aca07798de3d8a5c2abf6198dd3a4731effac03"}
Oct 30 22:27:34.545713 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:27:34.545581    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-5637a7-6db64b9ddc-lqdjr"
Oct 30 22:27:34.555488 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:27:34.555471    5510 scope.go:122] "RemoveContainer" containerID="7df82eb9f5f8346aaf90acc553463487049f56e37957a84fdf69d2a6fc69ba9d"
Oct 30 22:27:34.555794 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:27:34.555771    5510 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7df82eb9f5f8346aaf90acc553463487049f56e37957a84fdf69d2a6fc69ba9d\": container with ID starting with 7df82eb9f5f8346aaf90acc553463487049f56e37957a84fdf69d2a6fc69ba9d not found: ID does not exist" containerID="7df82eb9f5f8346aaf90acc553463487049f56e37957a84fdf69d2a6fc69ba9d"
Oct 30 22:27:34.555903 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:27:34.555799    5510 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7df82eb9f5f8346aaf90acc553463487049f56e37957a84fdf69d2a6fc69ba9d"} err="failed to get container status \"7df82eb9f5f8346aaf90acc553463487049f56e37957a84fdf69d2a6fc69ba9d\": rpc error: code = NotFound desc = could not find container \"7df82eb9f5f8346aaf90acc553463487049f56e37957a84fdf69d2a6fc69ba9d\": container with ID starting with 7df82eb9f5f8346aaf90acc553463487049f56e37957a84fdf69d2a6fc69ba9d not found: ID does not exist"
Oct 30 22:27:34.555903 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:27:34.555818    5510 scope.go:122] "RemoveContainer" containerID="82f0cc65e4a9cfe93229b93978d1daad54feaea2e2ff58a2e7e97a25977284bc"
Oct 30 22:27:34.702066 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:27:34.701982    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-5637a7-6db64b9ddc-lqdjr"]
Oct 30 22:27:34.708931 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:27:34.708905    5510 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-5637a7-6db64b9ddc-lqdjr"]
Oct 30 22:27:34.812694 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:27:34.812646    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-5637a7-6dfc655fbc-ms5sb"]
Oct 30 22:27:34.817504 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:27:34.817474    5510 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-5637a7-6dfc655fbc-ms5sb"]
Oct 30 22:27:35.555767 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:27:35.555732    5510 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2c98b4d1-9cc0-4482-af1e-1038eaeb1e61" path="/var/lib/kubelet/pods/2c98b4d1-9cc0-4482-af1e-1038eaeb1e61/volumes"
Oct 30 22:27:35.556231 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:27:35.556216    5510 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4009806a-5d15-4224-9b63-e3bc115f35ad" path="/var/lib/kubelet/pods/4009806a-5d15-4224-9b63-e3bc115f35ad/volumes"
Oct 30 22:27:36.318864 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:27:36.317406    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:27:37.579159 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:27:37.579114    5510 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-cf1826-5f6ddbf4c7-7mrgs" podUID="7bdcd87e-ab76-4af6-ae7b-37e1384b6406" containerName="payload-processing" containerID="cri-o://b77c2289be901656adec84752fdad0880eb9e5002f510d139d1a0367c9a5ee85" gracePeriod=25
Oct 30 22:27:37.681769 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:27:37.681710    5510 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-cf1826-556566884c-krgkh" podUID="4d203fdf-00d0-4d25-8a63-c2f355d8d03f" containerName="payload-pre-processing" containerID="cri-o://5976d932463b35a75223277b5967fb906832deb69137e65ef0ad08e49ae6c317" gracePeriod=25
Oct 30 22:27:37.876641 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:27:37.876520    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-cf1826-5f6ddbf4c7-7mrgs"
Oct 30 22:27:37.973611 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:27:37.973581    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-cf1826-556566884c-krgkh"
Oct 30 22:27:37.973878 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:27:37.973858    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7bdcd87e-ab76-4af6-ae7b-37e1384b6406-kube-api-access-wvwqx\" (UniqueName: \"kubernetes.io/projected/7bdcd87e-ab76-4af6-ae7b-37e1384b6406-kube-api-access-wvwqx\") pod \"7bdcd87e-ab76-4af6-ae7b-37e1384b6406\" (UID: \"7bdcd87e-ab76-4af6-ae7b-37e1384b6406\") "
Oct 30 22:27:37.973998 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:27:37.973985    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7bdcd87e-ab76-4af6-ae7b-37e1384b6406-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7bdcd87e-ab76-4af6-ae7b-37e1384b6406-plugins-config-volume\") pod \"7bdcd87e-ab76-4af6-ae7b-37e1384b6406\" (UID: \"7bdcd87e-ab76-4af6-ae7b-37e1384b6406\") "
Oct 30 22:27:37.974366 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:27:37.974338    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7bdcd87e-ab76-4af6-ae7b-37e1384b6406-plugins-config-volume" pod "7bdcd87e-ab76-4af6-ae7b-37e1384b6406" (UID: "7bdcd87e-ab76-4af6-ae7b-37e1384b6406"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:27:37.974731 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:27:37.974697    5510 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7bdcd87e-ab76-4af6-ae7b-37e1384b6406-plugins-config-volume\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:27:37.976098 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:27:37.976077    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7bdcd87e-ab76-4af6-ae7b-37e1384b6406-kube-api-access-wvwqx" pod "7bdcd87e-ab76-4af6-ae7b-37e1384b6406" (UID: "7bdcd87e-ab76-4af6-ae7b-37e1384b6406"). InnerVolumeSpecName "kube-api-access-wvwqx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:27:38.075607 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:27:38.075569    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4d203fdf-00d0-4d25-8a63-c2f355d8d03f-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4d203fdf-00d0-4d25-8a63-c2f355d8d03f-plugins-config-volume\") pod \"4d203fdf-00d0-4d25-8a63-c2f355d8d03f\" (UID: \"4d203fdf-00d0-4d25-8a63-c2f355d8d03f\") "
Oct 30 22:27:38.075832 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:27:38.075736    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4d203fdf-00d0-4d25-8a63-c2f355d8d03f-kube-api-access-gkjlr\" (UniqueName: \"kubernetes.io/projected/4d203fdf-00d0-4d25-8a63-c2f355d8d03f-kube-api-access-gkjlr\") pod \"4d203fdf-00d0-4d25-8a63-c2f355d8d03f\" (UID: \"4d203fdf-00d0-4d25-8a63-c2f355d8d03f\") "
Oct 30 22:27:38.075941 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:27:38.075918    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4d203fdf-00d0-4d25-8a63-c2f355d8d03f-plugins-config-volume" pod "4d203fdf-00d0-4d25-8a63-c2f355d8d03f" (UID: "4d203fdf-00d0-4d25-8a63-c2f355d8d03f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:27:38.076314 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:27:38.076294    5510 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wvwqx\" (UniqueName: \"kubernetes.io/projected/7bdcd87e-ab76-4af6-ae7b-37e1384b6406-kube-api-access-wvwqx\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:27:38.076391 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:27:38.076318    5510 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4d203fdf-00d0-4d25-8a63-c2f355d8d03f-plugins-config-volume\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:27:38.077735 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:27:38.077715    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4d203fdf-00d0-4d25-8a63-c2f355d8d03f-kube-api-access-gkjlr" pod "4d203fdf-00d0-4d25-8a63-c2f355d8d03f" (UID: "4d203fdf-00d0-4d25-8a63-c2f355d8d03f"). InnerVolumeSpecName "kube-api-access-gkjlr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:27:38.178232 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:27:38.178154    5510 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gkjlr\" (UniqueName: \"kubernetes.io/projected/4d203fdf-00d0-4d25-8a63-c2f355d8d03f-kube-api-access-gkjlr\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:27:38.421757 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:27:38.421713    5510 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6gjbf" containerName="extractor"
Oct 30 22:27:38.577324 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:27:38.577289    5510 generic.go:356] "Generic (PLEG): container finished" podID="4d203fdf-00d0-4d25-8a63-c2f355d8d03f" containerID="5976d932463b35a75223277b5967fb906832deb69137e65ef0ad08e49ae6c317" exitCode=0
Oct 30 22:27:38.577523 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:27:38.577337    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-cf1826-556566884c-krgkh"
Oct 30 22:27:38.577523 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:27:38.577375    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-cf1826-556566884c-krgkh" event={"ID":"4d203fdf-00d0-4d25-8a63-c2f355d8d03f","Type":"ContainerDied","Data":"5976d932463b35a75223277b5967fb906832deb69137e65ef0ad08e49ae6c317"}
Oct 30 22:27:38.577523 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:27:38.577431    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-cf1826-556566884c-krgkh" event={"ID":"4d203fdf-00d0-4d25-8a63-c2f355d8d03f","Type":"ContainerDied","Data":"e23c244757a3c640dde799f9527847e41fb477ed690e51693e4d4e02e50fa080"}
Oct 30 22:27:38.577523 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:27:38.577455    5510 scope.go:122] "RemoveContainer" containerID="5976d932463b35a75223277b5967fb906832deb69137e65ef0ad08e49ae6c317"
Oct 30 22:27:38.578946 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:27:38.578898    5510 generic.go:356] "Generic (PLEG): container finished" podID="7bdcd87e-ab76-4af6-ae7b-37e1384b6406" containerID="b77c2289be901656adec84752fdad0880eb9e5002f510d139d1a0367c9a5ee85" exitCode=0
Oct 30 22:27:38.578946 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:27:38.578930    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-cf1826-5f6ddbf4c7-7mrgs" event={"ID":"7bdcd87e-ab76-4af6-ae7b-37e1384b6406","Type":"ContainerDied","Data":"b77c2289be901656adec84752fdad0880eb9e5002f510d139d1a0367c9a5ee85"}
Oct 30 22:27:38.579137 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:27:38.578952    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-cf1826-5f6ddbf4c7-7mrgs"
Oct 30 22:27:38.579137 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:27:38.578978    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-cf1826-5f6ddbf4c7-7mrgs" event={"ID":"7bdcd87e-ab76-4af6-ae7b-37e1384b6406","Type":"ContainerDied","Data":"f48094b7cb3a8f2570286e4d21c4511aa6a48730edc65610adc4898095e97cc5"}
Oct 30 22:27:38.588948 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:27:38.588828    5510 scope.go:122] "RemoveContainer" containerID="5976d932463b35a75223277b5967fb906832deb69137e65ef0ad08e49ae6c317"
Oct 30 22:27:38.589113 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:27:38.589091    5510 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5976d932463b35a75223277b5967fb906832deb69137e65ef0ad08e49ae6c317\": container with ID starting with 5976d932463b35a75223277b5967fb906832deb69137e65ef0ad08e49ae6c317 not found: ID does not exist" containerID="5976d932463b35a75223277b5967fb906832deb69137e65ef0ad08e49ae6c317"
Oct 30 22:27:38.589182 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:27:38.589123    5510 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5976d932463b35a75223277b5967fb906832deb69137e65ef0ad08e49ae6c317"} err="failed to get container status \"5976d932463b35a75223277b5967fb906832deb69137e65ef0ad08e49ae6c317\": rpc error: code = NotFound desc = could not find container \"5976d932463b35a75223277b5967fb906832deb69137e65ef0ad08e49ae6c317\": container with ID starting with 5976d932463b35a75223277b5967fb906832deb69137e65ef0ad08e49ae6c317 not found: ID does not exist"
Oct 30 22:27:38.589182 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:27:38.589146    5510 scope.go:122] "RemoveContainer" containerID="b77c2289be901656adec84752fdad0880eb9e5002f510d139d1a0367c9a5ee85"
Oct 30 22:27:38.607525 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:27:38.607500    5510 scope.go:122] "RemoveContainer" containerID="b77c2289be901656adec84752fdad0880eb9e5002f510d139d1a0367c9a5ee85"
Oct 30 22:27:38.607893 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:27:38.607870    5510 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b77c2289be901656adec84752fdad0880eb9e5002f510d139d1a0367c9a5ee85\": container with ID starting with b77c2289be901656adec84752fdad0880eb9e5002f510d139d1a0367c9a5ee85 not found: ID does not exist" containerID="b77c2289be901656adec84752fdad0880eb9e5002f510d139d1a0367c9a5ee85"
Oct 30 22:27:38.607989 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:27:38.607900    5510 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b77c2289be901656adec84752fdad0880eb9e5002f510d139d1a0367c9a5ee85"} err="failed to get container status \"b77c2289be901656adec84752fdad0880eb9e5002f510d139d1a0367c9a5ee85\": rpc error: code = NotFound desc = could not find container \"b77c2289be901656adec84752fdad0880eb9e5002f510d139d1a0367c9a5ee85\": container with ID starting with b77c2289be901656adec84752fdad0880eb9e5002f510d139d1a0367c9a5ee85 not found: ID does not exist"
Oct 30 22:27:38.740696 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:27:38.740638    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-cf1826-5f6ddbf4c7-7mrgs"]
Oct 30 22:27:38.744185 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:27:38.744160    5510 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-cf1826-5f6ddbf4c7-7mrgs"]
Oct 30 22:27:38.850886 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:27:38.850774    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-cf1826-556566884c-krgkh"]
Oct 30 22:27:38.854514 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:27:38.854488    5510 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-cf1826-556566884c-krgkh"]
Oct 30 22:27:39.555669 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:27:39.555625    5510 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4d203fdf-00d0-4d25-8a63-c2f355d8d03f" path="/var/lib/kubelet/pods/4d203fdf-00d0-4d25-8a63-c2f355d8d03f/volumes"
Oct 30 22:27:39.556127 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:27:39.556113    5510 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7bdcd87e-ab76-4af6-ae7b-37e1384b6406" path="/var/lib/kubelet/pods/7bdcd87e-ab76-4af6-ae7b-37e1384b6406/volumes"
Oct 30 22:27:40.425451 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:27:40.425410    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:27:40.704194 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:27:40.704114    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:27:41.153333 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:27:41.153106    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:27:41.363319 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:27:41.362934    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:27:42.773726 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:27:42.773691    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:27:45.610923 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:27:45.610878    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:27:45.869302 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:27:45.869210    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:27:46.537815 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:27:46.537766    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:27:47.694967 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:27:47.694935    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-7bb1c2/distinct-2-w3-7bb1c2-kserve-c549df844-zt595"]
Oct 30 22:27:47.839426 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:27:47.839388    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:27:51.492857 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:27:51.492805    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:27:54.016681 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:27:54.010827    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:27:54.214518 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:27:54.214454    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:27:55.928280 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:27:55.928237    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-7f76c4-56f886bdb9-w57ph"]
Oct 30 22:27:56.031774 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:27:56.031740    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-7f76c4-6c578b69ff-hgkjj"]
Oct 30 22:27:56.105912 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:27:56.105880    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:27:56.357314 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:27:56.357273    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:27:56.544747 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:27:56.544712    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:28:00.714694 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:28:00.714550    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:28:00.928935 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:28:00.928890    5510 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-7f76c4-56f886bdb9-w57ph" podUID="4e4794d9-9186-4127-b515-3e89b9882981" containerName="payload-processing" containerID="cri-o://c8393a23596fe9ffe4530603d9bceb1fbecb8ca5be75f66d2ce0d59ae5cd0d92" gracePeriod=25
Oct 30 22:28:01.032224 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:28:01.032188    5510 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-7f76c4-6c578b69ff-hgkjj" podUID="446aae1e-35a8-49da-9780-db1c369d7efd" containerName="payload-pre-processing" containerID="cri-o://72a9726d0d1876846b3b24cf4236b315afcfee647cff03f2c45c6cd75632efed" gracePeriod=25
Oct 30 22:28:01.216467 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:28:01.216434    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-7f76c4-56f886bdb9-w57ph"
Oct 30 22:28:01.302749 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:28:01.302721    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4e4794d9-9186-4127-b515-3e89b9882981-kube-api-access-7ffv2\" (UniqueName: \"kubernetes.io/projected/4e4794d9-9186-4127-b515-3e89b9882981-kube-api-access-7ffv2\") pod \"4e4794d9-9186-4127-b515-3e89b9882981\" (UID: \"4e4794d9-9186-4127-b515-3e89b9882981\") "
Oct 30 22:28:01.303006 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:28:01.302988    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4e4794d9-9186-4127-b515-3e89b9882981-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4e4794d9-9186-4127-b515-3e89b9882981-plugins-config-volume\") pod \"4e4794d9-9186-4127-b515-3e89b9882981\" (UID: \"4e4794d9-9186-4127-b515-3e89b9882981\") "
Oct 30 22:28:01.303364 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:28:01.303341    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4e4794d9-9186-4127-b515-3e89b9882981-plugins-config-volume" pod "4e4794d9-9186-4127-b515-3e89b9882981" (UID: "4e4794d9-9186-4127-b515-3e89b9882981"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:28:01.303595 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:28:01.303579    5510 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4e4794d9-9186-4127-b515-3e89b9882981-plugins-config-volume\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:28:01.304895 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:28:01.304871    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4e4794d9-9186-4127-b515-3e89b9882981-kube-api-access-7ffv2" pod "4e4794d9-9186-4127-b515-3e89b9882981" (UID: "4e4794d9-9186-4127-b515-3e89b9882981"). InnerVolumeSpecName "kube-api-access-7ffv2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:28:01.326235 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:28:01.326211    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-7f76c4-6c578b69ff-hgkjj"
Oct 30 22:28:01.404976 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:28:01.404945    5510 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7ffv2\" (UniqueName: \"kubernetes.io/projected/4e4794d9-9186-4127-b515-3e89b9882981-kube-api-access-7ffv2\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:28:01.505756 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:28:01.505716    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/446aae1e-35a8-49da-9780-db1c369d7efd-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/446aae1e-35a8-49da-9780-db1c369d7efd-plugins-config-volume\") pod \"446aae1e-35a8-49da-9780-db1c369d7efd\" (UID: \"446aae1e-35a8-49da-9780-db1c369d7efd\") "
Oct 30 22:28:01.505950 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:28:01.505817    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/446aae1e-35a8-49da-9780-db1c369d7efd-kube-api-access-2bbmj\" (UniqueName: \"kubernetes.io/projected/446aae1e-35a8-49da-9780-db1c369d7efd-kube-api-access-2bbmj\") pod \"446aae1e-35a8-49da-9780-db1c369d7efd\" (UID: \"446aae1e-35a8-49da-9780-db1c369d7efd\") "
Oct 30 22:28:01.506108 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:28:01.506084    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/446aae1e-35a8-49da-9780-db1c369d7efd-plugins-config-volume" pod "446aae1e-35a8-49da-9780-db1c369d7efd" (UID: "446aae1e-35a8-49da-9780-db1c369d7efd"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:28:01.506398 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:28:01.506383    5510 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/446aae1e-35a8-49da-9780-db1c369d7efd-plugins-config-volume\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:28:01.507957 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:28:01.507932    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/446aae1e-35a8-49da-9780-db1c369d7efd-kube-api-access-2bbmj" pod "446aae1e-35a8-49da-9780-db1c369d7efd" (UID: "446aae1e-35a8-49da-9780-db1c369d7efd"). InnerVolumeSpecName "kube-api-access-2bbmj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:28:01.607755 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:28:01.607728    5510 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2bbmj\" (UniqueName: \"kubernetes.io/projected/446aae1e-35a8-49da-9780-db1c369d7efd-kube-api-access-2bbmj\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:28:01.793140 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:28:01.793095    5510 generic.go:356] "Generic (PLEG): container finished" podID="446aae1e-35a8-49da-9780-db1c369d7efd" containerID="72a9726d0d1876846b3b24cf4236b315afcfee647cff03f2c45c6cd75632efed" exitCode=0
Oct 30 22:28:01.793622 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:28:01.793146    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-7f76c4-6c578b69ff-hgkjj"
Oct 30 22:28:01.793622 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:28:01.793179    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-7f76c4-6c578b69ff-hgkjj" event={"ID":"446aae1e-35a8-49da-9780-db1c369d7efd","Type":"ContainerDied","Data":"72a9726d0d1876846b3b24cf4236b315afcfee647cff03f2c45c6cd75632efed"}
Oct 30 22:28:01.793622 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:28:01.793232    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-7f76c4-6c578b69ff-hgkjj" event={"ID":"446aae1e-35a8-49da-9780-db1c369d7efd","Type":"ContainerDied","Data":"83cd2953c39b177970955f9f6099d9269b744fb9f7235830db10832808db40d5"}
Oct 30 22:28:01.793622 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:28:01.793260    5510 scope.go:122] "RemoveContainer" containerID="72a9726d0d1876846b3b24cf4236b315afcfee647cff03f2c45c6cd75632efed"
Oct 30 22:28:01.794921 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:28:01.794892    5510 generic.go:356] "Generic (PLEG): container finished" podID="4e4794d9-9186-4127-b515-3e89b9882981" containerID="c8393a23596fe9ffe4530603d9bceb1fbecb8ca5be75f66d2ce0d59ae5cd0d92" exitCode=0
Oct 30 22:28:01.795042 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:28:01.794939    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-7f76c4-56f886bdb9-w57ph"
Oct 30 22:28:01.795042 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:28:01.794979    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-7f76c4-56f886bdb9-w57ph" event={"ID":"4e4794d9-9186-4127-b515-3e89b9882981","Type":"ContainerDied","Data":"c8393a23596fe9ffe4530603d9bceb1fbecb8ca5be75f66d2ce0d59ae5cd0d92"}
Oct 30 22:28:01.795042 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:28:01.795031    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-7f76c4-56f886bdb9-w57ph" event={"ID":"4e4794d9-9186-4127-b515-3e89b9882981","Type":"ContainerDied","Data":"597b383926b6e10e0c22fca960984699268dcfefe6df38454718b82d18d1f1c6"}
Oct 30 22:28:01.808580 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:28:01.808561    5510 scope.go:122] "RemoveContainer" containerID="72a9726d0d1876846b3b24cf4236b315afcfee647cff03f2c45c6cd75632efed"
Oct 30 22:28:01.808988 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:28:01.808963    5510 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"72a9726d0d1876846b3b24cf4236b315afcfee647cff03f2c45c6cd75632efed\": container with ID starting with 72a9726d0d1876846b3b24cf4236b315afcfee647cff03f2c45c6cd75632efed not found: ID does not exist" containerID="72a9726d0d1876846b3b24cf4236b315afcfee647cff03f2c45c6cd75632efed"
Oct 30 22:28:01.809038 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:28:01.809010    5510 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"72a9726d0d1876846b3b24cf4236b315afcfee647cff03f2c45c6cd75632efed"} err="failed to get container status \"72a9726d0d1876846b3b24cf4236b315afcfee647cff03f2c45c6cd75632efed\": rpc error: code = NotFound desc = could not find container \"72a9726d0d1876846b3b24cf4236b315afcfee647cff03f2c45c6cd75632efed\": container with ID starting with 72a9726d0d1876846b3b24cf4236b315afcfee647cff03f2c45c6cd75632efed not found: ID does not exist"
Oct 30 22:28:01.809038 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:28:01.809034    5510 scope.go:122] "RemoveContainer" containerID="c8393a23596fe9ffe4530603d9bceb1fbecb8ca5be75f66d2ce0d59ae5cd0d92"
Oct 30 22:28:01.818944 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:28:01.818888    5510 scope.go:122] "RemoveContainer" containerID="c8393a23596fe9ffe4530603d9bceb1fbecb8ca5be75f66d2ce0d59ae5cd0d92"
Oct 30 22:28:01.819161 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:28:01.819144    5510 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c8393a23596fe9ffe4530603d9bceb1fbecb8ca5be75f66d2ce0d59ae5cd0d92\": container with ID starting with c8393a23596fe9ffe4530603d9bceb1fbecb8ca5be75f66d2ce0d59ae5cd0d92 not found: ID does not exist" containerID="c8393a23596fe9ffe4530603d9bceb1fbecb8ca5be75f66d2ce0d59ae5cd0d92"
Oct 30 22:28:01.819204 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:28:01.819170    5510 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c8393a23596fe9ffe4530603d9bceb1fbecb8ca5be75f66d2ce0d59ae5cd0d92"} err="failed to get container status \"c8393a23596fe9ffe4530603d9bceb1fbecb8ca5be75f66d2ce0d59ae5cd0d92\": rpc error: code = NotFound desc = could not find container \"c8393a23596fe9ffe4530603d9bceb1fbecb8ca5be75f66d2ce0d59ae5cd0d92\": container with ID starting with c8393a23596fe9ffe4530603d9bceb1fbecb8ca5be75f66d2ce0d59ae5cd0d92 not found: ID does not exist"
Oct 30 22:28:01.950627 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:28:01.950587    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-7f76c4-56f886bdb9-w57ph"]
Oct 30 22:28:01.997938 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:28:01.997903    5510 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-7f76c4-56f886bdb9-w57ph"]
Oct 30 22:28:02.059977 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:28:02.059945    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-7f76c4-6c578b69ff-hgkjj"]
Oct 30 22:28:02.063811 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:28:02.063789    5510 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-7f76c4-6c578b69ff-hgkjj"]
Oct 30 22:28:02.716892 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:28:02.716842    5510 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w3-7bb1c2/distinct-2-w3-7bb1c2-kserve-c549df844-zt595" podUID="1916e87c-a1a4-4967-8640-8bbcd9d12903" containerName="main" containerID="cri-o://f5eb22bd766f1da34b151a1610d37933dbce4a9bd1b928491eac465a30021bea" gracePeriod=45
Oct 30 22:28:02.986980 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:28:02.986916    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-7bb1c2/distinct-2-w3-7bb1c2-kserve-c549df844-zt595"
Oct 30 22:28:03.128556 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:28:03.128520    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1916e87c-a1a4-4967-8640-8bbcd9d12903-home\" (UniqueName: \"kubernetes.io/empty-dir/1916e87c-a1a4-4967-8640-8bbcd9d12903-home\") pod \"1916e87c-a1a4-4967-8640-8bbcd9d12903\" (UID: \"1916e87c-a1a4-4967-8640-8bbcd9d12903\") "
Oct 30 22:28:03.128556 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:28:03.128571    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1916e87c-a1a4-4967-8640-8bbcd9d12903-tls-certs\" (UniqueName: \"kubernetes.io/secret/1916e87c-a1a4-4967-8640-8bbcd9d12903-tls-certs\") pod \"1916e87c-a1a4-4967-8640-8bbcd9d12903\" (UID: \"1916e87c-a1a4-4967-8640-8bbcd9d12903\") "
Oct 30 22:28:03.128880 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:28:03.128799    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1916e87c-a1a4-4967-8640-8bbcd9d12903-dshm\" (UniqueName: \"kubernetes.io/empty-dir/1916e87c-a1a4-4967-8640-8bbcd9d12903-dshm\") pod \"1916e87c-a1a4-4967-8640-8bbcd9d12903\" (UID: \"1916e87c-a1a4-4967-8640-8bbcd9d12903\") "
Oct 30 22:28:03.128880 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:28:03.128822    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1916e87c-a1a4-4967-8640-8bbcd9d12903-kube-api-access-6k57f\" (UniqueName: \"kubernetes.io/projected/1916e87c-a1a4-4967-8640-8bbcd9d12903-kube-api-access-6k57f\") pod \"1916e87c-a1a4-4967-8640-8bbcd9d12903\" (UID: \"1916e87c-a1a4-4967-8640-8bbcd9d12903\") "
Oct 30 22:28:03.128880 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:28:03.128834    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1916e87c-a1a4-4967-8640-8bbcd9d12903-home" pod "1916e87c-a1a4-4967-8640-8bbcd9d12903" (UID: "1916e87c-a1a4-4967-8640-8bbcd9d12903"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:28:03.128880 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:28:03.128855    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1916e87c-a1a4-4967-8640-8bbcd9d12903-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1916e87c-a1a4-4967-8640-8bbcd9d12903-tmp-dir\") pod \"1916e87c-a1a4-4967-8640-8bbcd9d12903\" (UID: \"1916e87c-a1a4-4967-8640-8bbcd9d12903\") "
Oct 30 22:28:03.128880 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:28:03.128873    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1916e87c-a1a4-4967-8640-8bbcd9d12903-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1916e87c-a1a4-4967-8640-8bbcd9d12903-model-cache\") pod \"1916e87c-a1a4-4967-8640-8bbcd9d12903\" (UID: \"1916e87c-a1a4-4967-8640-8bbcd9d12903\") "
Oct 30 22:28:03.129516 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:28:03.129488    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1916e87c-a1a4-4967-8640-8bbcd9d12903-tmp-dir" pod "1916e87c-a1a4-4967-8640-8bbcd9d12903" (UID: "1916e87c-a1a4-4967-8640-8bbcd9d12903"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:28:03.129700 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:28:03.129642    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1916e87c-a1a4-4967-8640-8bbcd9d12903-model-cache" pod "1916e87c-a1a4-4967-8640-8bbcd9d12903" (UID: "1916e87c-a1a4-4967-8640-8bbcd9d12903"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:28:03.129700 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:28:03.129689    5510 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1916e87c-a1a4-4967-8640-8bbcd9d12903-tmp-dir\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:28:03.129804 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:28:03.129712    5510 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1916e87c-a1a4-4967-8640-8bbcd9d12903-home\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:28:03.131750 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:28:03.131726    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1916e87c-a1a4-4967-8640-8bbcd9d12903-dshm" pod "1916e87c-a1a4-4967-8640-8bbcd9d12903" (UID: "1916e87c-a1a4-4967-8640-8bbcd9d12903"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:28:03.131885 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:28:03.131865    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1916e87c-a1a4-4967-8640-8bbcd9d12903-tls-certs" pod "1916e87c-a1a4-4967-8640-8bbcd9d12903" (UID: "1916e87c-a1a4-4967-8640-8bbcd9d12903"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:28:03.131967 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:28:03.131937    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1916e87c-a1a4-4967-8640-8bbcd9d12903-kube-api-access-6k57f" pod "1916e87c-a1a4-4967-8640-8bbcd9d12903" (UID: "1916e87c-a1a4-4967-8640-8bbcd9d12903"). InnerVolumeSpecName "kube-api-access-6k57f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:28:03.231445 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:28:03.231405    5510 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1916e87c-a1a4-4967-8640-8bbcd9d12903-dshm\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:28:03.231445 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:28:03.231447    5510 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6k57f\" (UniqueName: \"kubernetes.io/projected/1916e87c-a1a4-4967-8640-8bbcd9d12903-kube-api-access-6k57f\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:28:03.231770 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:28:03.231462    5510 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1916e87c-a1a4-4967-8640-8bbcd9d12903-model-cache\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:28:03.231770 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:28:03.231476    5510 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1916e87c-a1a4-4967-8640-8bbcd9d12903-tls-certs\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:28:03.564148 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:28:03.564110    5510 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="446aae1e-35a8-49da-9780-db1c369d7efd" path="/var/lib/kubelet/pods/446aae1e-35a8-49da-9780-db1c369d7efd/volumes"
Oct 30 22:28:03.564738 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:28:03.564715    5510 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4e4794d9-9186-4127-b515-3e89b9882981" path="/var/lib/kubelet/pods/4e4794d9-9186-4127-b515-3e89b9882981/volumes"
Oct 30 22:28:03.815487 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:28:03.815397    5510 generic.go:356] "Generic (PLEG): container finished" podID="1916e87c-a1a4-4967-8640-8bbcd9d12903" containerID="f5eb22bd766f1da34b151a1610d37933dbce4a9bd1b928491eac465a30021bea" exitCode=0
Oct 30 22:28:03.815487 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:28:03.815446    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-7bb1c2/distinct-2-w3-7bb1c2-kserve-c549df844-zt595"
Oct 30 22:28:03.815717 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:28:03.815478    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-7bb1c2/distinct-2-w3-7bb1c2-kserve-c549df844-zt595" event={"ID":"1916e87c-a1a4-4967-8640-8bbcd9d12903","Type":"ContainerDied","Data":"f5eb22bd766f1da34b151a1610d37933dbce4a9bd1b928491eac465a30021bea"}
Oct 30 22:28:03.815717 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:28:03.815528    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-7bb1c2/distinct-2-w3-7bb1c2-kserve-c549df844-zt595" event={"ID":"1916e87c-a1a4-4967-8640-8bbcd9d12903","Type":"ContainerDied","Data":"4a3643dba5c1c2fbbf23c448892fe922ba8f446d0bc057c19c31b7f2842769f8"}
Oct 30 22:28:03.815717 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:28:03.815549    5510 scope.go:122] "RemoveContainer" containerID="f5eb22bd766f1da34b151a1610d37933dbce4a9bd1b928491eac465a30021bea"
Oct 30 22:28:03.825925 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:28:03.825909    5510 scope.go:122] "RemoveContainer" containerID="f5eb22bd766f1da34b151a1610d37933dbce4a9bd1b928491eac465a30021bea"
Oct 30 22:28:03.826167 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:28:03.826145    5510 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f5eb22bd766f1da34b151a1610d37933dbce4a9bd1b928491eac465a30021bea\": container with ID starting with f5eb22bd766f1da34b151a1610d37933dbce4a9bd1b928491eac465a30021bea not found: ID does not exist" containerID="f5eb22bd766f1da34b151a1610d37933dbce4a9bd1b928491eac465a30021bea"
Oct 30 22:28:03.826233 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:28:03.826173    5510 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f5eb22bd766f1da34b151a1610d37933dbce4a9bd1b928491eac465a30021bea"} err="failed to get container status \"f5eb22bd766f1da34b151a1610d37933dbce4a9bd1b928491eac465a30021bea\": rpc error: code = NotFound desc = could not find container \"f5eb22bd766f1da34b151a1610d37933dbce4a9bd1b928491eac465a30021bea\": container with ID starting with f5eb22bd766f1da34b151a1610d37933dbce4a9bd1b928491eac465a30021bea not found: ID does not exist"
Oct 30 22:28:03.972876 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:28:03.972839    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-7bb1c2/distinct-2-w3-7bb1c2-kserve-c549df844-zt595"]
Oct 30 22:28:04.019236 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:28:04.019204    5510 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w3-7bb1c2/distinct-2-w3-7bb1c2-kserve-c549df844-zt595"]
Oct 30 22:28:05.309317 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:28:05.309278    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:28:05.555415 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:28:05.555378    5510 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1916e87c-a1a4-4967-8640-8bbcd9d12903" path="/var/lib/kubelet/pods/1916e87c-a1a4-4967-8640-8bbcd9d12903/volumes"
Oct 30 22:28:08.420686 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:28:08.420635    5510 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6gjbf" containerName="extractor"
Oct 30 22:28:10.019682 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:28:10.019397    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:28:14.790075 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:28:14.790034    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:28:15.043311 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:28:15.043223    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:28:15.236857 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:28:15.236819    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:28:19.079258 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:28:19.079221    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:28:20.578819 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:28:20.578773    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:28:20.959016 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:28:20.958922    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:28:21.180063 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:28:21.180022    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:28:21.565032 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:28:21.564992    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:28:25.229713 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:28:25.229671    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:28:25.492961 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:28:25.492868    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:28:25.989710 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:28:25.989668    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:28:27.244316 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:28:27.244278    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:28:29.128086 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:28:29.128049    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:28:29.388281 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:28:29.388195    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:28:29.591923 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:28:29.591887    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:28:29.797971 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:28:29.797937    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:28:31.344348 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:28:31.344300    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:28:31.541520 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:28:31.541486    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:28:33.643905 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:28:33.642956    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:28:33.903223 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:28:33.903128    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:28:34.314887 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:28:34.314849    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:28:34.698192 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:28:34.698105    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:28:36.387992 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:28:36.387956    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:28:36.614198 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:28:36.614159    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:28:37.441162 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:28:37.441118    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-0b6a3f-f4bfcfb9f-fvtq8"]
Oct 30 22:28:37.771350 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:28:37.771234    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:28:37.960093 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:28:37.960052    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:28:38.422281 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:28:38.422245    5510 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6gjbf" containerName="extractor"
Oct 30 22:28:40.704084 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:28:40.704044    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:28:41.964361 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:28:41.964324    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:28:42.349050 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:28:42.349014    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:28:42.443794 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:28:42.443744    5510 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-0b6a3f-f4bfcfb9f-fvtq8" podUID="8023fe04-37eb-4458-bfee-ef8170d34ab6" containerName="payload-pre-processing" containerID="cri-o://df4370ed198e7b1f06806d9c2917ae30be569e48db0e52c0cc8dc673d96a67b7" gracePeriod=25
Oct 30 22:28:42.668217 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:28:42.666732    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:28:42.792874 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:28:42.792842    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-0b6a3f-f4bfcfb9f-fvtq8"
Oct 30 22:28:42.803240 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:28:42.803207    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8023fe04-37eb-4458-bfee-ef8170d34ab6-kube-api-access-xgv5f\" (UniqueName: \"kubernetes.io/projected/8023fe04-37eb-4458-bfee-ef8170d34ab6-kube-api-access-xgv5f\") pod \"8023fe04-37eb-4458-bfee-ef8170d34ab6\" (UID: \"8023fe04-37eb-4458-bfee-ef8170d34ab6\") "
Oct 30 22:28:42.803399 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:28:42.803268    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8023fe04-37eb-4458-bfee-ef8170d34ab6-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8023fe04-37eb-4458-bfee-ef8170d34ab6-plugins-config-volume\") pod \"8023fe04-37eb-4458-bfee-ef8170d34ab6\" (UID: \"8023fe04-37eb-4458-bfee-ef8170d34ab6\") "
Oct 30 22:28:42.803728 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:28:42.803695    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8023fe04-37eb-4458-bfee-ef8170d34ab6-plugins-config-volume" pod "8023fe04-37eb-4458-bfee-ef8170d34ab6" (UID: "8023fe04-37eb-4458-bfee-ef8170d34ab6"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:28:42.803903 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:28:42.803881    5510 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8023fe04-37eb-4458-bfee-ef8170d34ab6-plugins-config-volume\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:28:42.805702 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:28:42.805668    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8023fe04-37eb-4458-bfee-ef8170d34ab6-kube-api-access-xgv5f" pod "8023fe04-37eb-4458-bfee-ef8170d34ab6" (UID: "8023fe04-37eb-4458-bfee-ef8170d34ab6"). InnerVolumeSpecName "kube-api-access-xgv5f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:28:42.905229 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:28:42.905142    5510 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xgv5f\" (UniqueName: \"kubernetes.io/projected/8023fe04-37eb-4458-bfee-ef8170d34ab6-kube-api-access-xgv5f\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:28:43.217968 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:28:43.217881    5510 generic.go:356] "Generic (PLEG): container finished" podID="8023fe04-37eb-4458-bfee-ef8170d34ab6" containerID="df4370ed198e7b1f06806d9c2917ae30be569e48db0e52c0cc8dc673d96a67b7" exitCode=0
Oct 30 22:28:43.217968 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:28:43.217932    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-0b6a3f-f4bfcfb9f-fvtq8"
Oct 30 22:28:43.218496 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:28:43.217966    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-0b6a3f-f4bfcfb9f-fvtq8" event={"ID":"8023fe04-37eb-4458-bfee-ef8170d34ab6","Type":"ContainerDied","Data":"df4370ed198e7b1f06806d9c2917ae30be569e48db0e52c0cc8dc673d96a67b7"}
Oct 30 22:28:43.218496 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:28:43.218022    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-0b6a3f-f4bfcfb9f-fvtq8" event={"ID":"8023fe04-37eb-4458-bfee-ef8170d34ab6","Type":"ContainerDied","Data":"2c8a8fefe43b82d55b9ae8fd69eb330b43c6305ffdad82f480d7cef4cebbbf59"}
Oct 30 22:28:43.218496 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:28:43.218050    5510 scope.go:122] "RemoveContainer" containerID="df4370ed198e7b1f06806d9c2917ae30be569e48db0e52c0cc8dc673d96a67b7"
Oct 30 22:28:43.229312 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:28:43.229289    5510 scope.go:122] "RemoveContainer" containerID="df4370ed198e7b1f06806d9c2917ae30be569e48db0e52c0cc8dc673d96a67b7"
Oct 30 22:28:43.229590 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:28:43.229567    5510 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"df4370ed198e7b1f06806d9c2917ae30be569e48db0e52c0cc8dc673d96a67b7\": container with ID starting with df4370ed198e7b1f06806d9c2917ae30be569e48db0e52c0cc8dc673d96a67b7 not found: ID does not exist" containerID="df4370ed198e7b1f06806d9c2917ae30be569e48db0e52c0cc8dc673d96a67b7"
Oct 30 22:28:43.229681 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:28:43.229595    5510 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"df4370ed198e7b1f06806d9c2917ae30be569e48db0e52c0cc8dc673d96a67b7"} err="failed to get container status \"df4370ed198e7b1f06806d9c2917ae30be569e48db0e52c0cc8dc673d96a67b7\": rpc error: code = NotFound desc = could not find container \"df4370ed198e7b1f06806d9c2917ae30be569e48db0e52c0cc8dc673d96a67b7\": container with ID starting with df4370ed198e7b1f06806d9c2917ae30be569e48db0e52c0cc8dc673d96a67b7 not found: ID does not exist"
Oct 30 22:28:43.280029 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:28:43.279993    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:28:43.375587 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:28:43.375549    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-0b6a3f-f4bfcfb9f-fvtq8"]
Oct 30 22:28:43.421316 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:28:43.421287    5510 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-0b6a3f-f4bfcfb9f-fvtq8"]
Oct 30 22:28:43.555179 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:28:43.555146    5510 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8023fe04-37eb-4458-bfee-ef8170d34ab6" path="/var/lib/kubelet/pods/8023fe04-37eb-4458-bfee-ef8170d34ab6/volumes"
Oct 30 22:28:43.708408 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:28:43.708367    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:28:43.896212 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:28:43.896122    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:28:44.092534 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:28:44.092490    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:28:44.529678 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:28:44.525765    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:28:44.739216 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:28:44.739182    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:28:49.237677 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:28:49.235812    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:28:49.808534 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:28:49.808491    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:28:50.017868 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:28:50.017827    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:28:50.247632 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:28:50.247511    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:28:51.802114 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:28:51.802073    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:28:52.046602 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:28:52.046561    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:28:52.288707 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:28:52.284556    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:28:53.632157 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:28:53.632118    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-cf1826-54f5f6c8c4-d972s"]
Oct 30 22:28:53.632591 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:28:53.632369    5510 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-cf1826-54f5f6c8c4-d972s" podUID="58572a25-fdb3-417c-9ade-f20e2d714b78" containerName="maas-api" containerID="cri-o://0deb080acbc693aa01ed07d33c34b48459d268dcc7c5fa87f46fb0e3b1333450" gracePeriod=30
Oct 30 22:28:53.766931 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:28:53.766887    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:28:53.982735 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:28:53.982696    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-cf1826-54f5f6c8c4-d972s"
Oct 30 22:28:54.076190 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:28:54.076152    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/58572a25-fdb3-417c-9ade-f20e2d714b78-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/58572a25-fdb3-417c-9ade-f20e2d714b78-maas-api-tls\") pod \"58572a25-fdb3-417c-9ade-f20e2d714b78\" (UID: \"58572a25-fdb3-417c-9ade-f20e2d714b78\") "
Oct 30 22:28:54.076956 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:28:54.076935    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/58572a25-fdb3-417c-9ade-f20e2d714b78-kube-api-access-drgk9\" (UniqueName: \"kubernetes.io/projected/58572a25-fdb3-417c-9ade-f20e2d714b78-kube-api-access-drgk9\") pod \"58572a25-fdb3-417c-9ade-f20e2d714b78\" (UID: \"58572a25-fdb3-417c-9ade-f20e2d714b78\") "
Oct 30 22:28:54.077761 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:28:54.077740    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/58572a25-fdb3-417c-9ade-f20e2d714b78-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/58572a25-fdb3-417c-9ade-f20e2d714b78-openshift-service-ca\") pod \"58572a25-fdb3-417c-9ade-f20e2d714b78\" (UID: \"58572a25-fdb3-417c-9ade-f20e2d714b78\") "
Oct 30 22:28:54.081909 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:28:54.079301    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/58572a25-fdb3-417c-9ade-f20e2d714b78-metrics-certs\" (UniqueName: \"kubernetes.io/secret/58572a25-fdb3-417c-9ade-f20e2d714b78-metrics-certs\") pod \"58572a25-fdb3-417c-9ade-f20e2d714b78\" (UID: \"58572a25-fdb3-417c-9ade-f20e2d714b78\") "
Oct 30 22:28:54.081909 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:28:54.079005    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/58572a25-fdb3-417c-9ade-f20e2d714b78-openshift-service-ca" pod "58572a25-fdb3-417c-9ade-f20e2d714b78" (UID: "58572a25-fdb3-417c-9ade-f20e2d714b78"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:28:54.091714 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:28:54.087109    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/58572a25-fdb3-417c-9ade-f20e2d714b78-metrics-certs" pod "58572a25-fdb3-417c-9ade-f20e2d714b78" (UID: "58572a25-fdb3-417c-9ade-f20e2d714b78"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:28:54.091714 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:28:54.087478    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/58572a25-fdb3-417c-9ade-f20e2d714b78-kube-api-access-drgk9" pod "58572a25-fdb3-417c-9ade-f20e2d714b78" (UID: "58572a25-fdb3-417c-9ade-f20e2d714b78"). InnerVolumeSpecName "kube-api-access-drgk9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:28:54.092022 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:28:54.091993    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/58572a25-fdb3-417c-9ade-f20e2d714b78-maas-api-tls" pod "58572a25-fdb3-417c-9ade-f20e2d714b78" (UID: "58572a25-fdb3-417c-9ade-f20e2d714b78"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:28:54.181810 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:28:54.181774    5510 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/58572a25-fdb3-417c-9ade-f20e2d714b78-openshift-service-ca\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:28:54.181810 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:28:54.181807    5510 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/58572a25-fdb3-417c-9ade-f20e2d714b78-metrics-certs\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:28:54.182053 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:28:54.181816    5510 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/58572a25-fdb3-417c-9ade-f20e2d714b78-maas-api-tls\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:28:54.182053 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:28:54.181834    5510 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-drgk9\" (UniqueName: \"kubernetes.io/projected/58572a25-fdb3-417c-9ade-f20e2d714b78-kube-api-access-drgk9\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:28:54.270066 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:28:54.268397    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:28:54.386571 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:28:54.386537    5510 generic.go:356] "Generic (PLEG): container finished" podID="58572a25-fdb3-417c-9ade-f20e2d714b78" containerID="0deb080acbc693aa01ed07d33c34b48459d268dcc7c5fa87f46fb0e3b1333450" exitCode=0
Oct 30 22:28:54.386789 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:28:54.386595    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-cf1826-54f5f6c8c4-d972s"
Oct 30 22:28:54.386789 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:28:54.386621    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-cf1826-54f5f6c8c4-d972s" event={"ID":"58572a25-fdb3-417c-9ade-f20e2d714b78","Type":"ContainerDied","Data":"0deb080acbc693aa01ed07d33c34b48459d268dcc7c5fa87f46fb0e3b1333450"}
Oct 30 22:28:54.386789 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:28:54.386691    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-cf1826-54f5f6c8c4-d972s" event={"ID":"58572a25-fdb3-417c-9ade-f20e2d714b78","Type":"ContainerDied","Data":"847c81f4764479569b7b66d71c7986c27dad1734a26cf19e7d30fa81a510f9aa"}
Oct 30 22:28:54.386789 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:28:54.386721    5510 scope.go:122] "RemoveContainer" containerID="0deb080acbc693aa01ed07d33c34b48459d268dcc7c5fa87f46fb0e3b1333450"
Oct 30 22:28:54.398915 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:28:54.398898    5510 scope.go:122] "RemoveContainer" containerID="0deb080acbc693aa01ed07d33c34b48459d268dcc7c5fa87f46fb0e3b1333450"
Oct 30 22:28:54.399213 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:28:54.399195    5510 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0deb080acbc693aa01ed07d33c34b48459d268dcc7c5fa87f46fb0e3b1333450\": container with ID starting with 0deb080acbc693aa01ed07d33c34b48459d268dcc7c5fa87f46fb0e3b1333450 not found: ID does not exist" containerID="0deb080acbc693aa01ed07d33c34b48459d268dcc7c5fa87f46fb0e3b1333450"
Oct 30 22:28:54.399282 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:28:54.399219    5510 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0deb080acbc693aa01ed07d33c34b48459d268dcc7c5fa87f46fb0e3b1333450"} err="failed to get container status \"0deb080acbc693aa01ed07d33c34b48459d268dcc7c5fa87f46fb0e3b1333450\": rpc error: code = NotFound desc = could not find container \"0deb080acbc693aa01ed07d33c34b48459d268dcc7c5fa87f46fb0e3b1333450\": container with ID starting with 0deb080acbc693aa01ed07d33c34b48459d268dcc7c5fa87f46fb0e3b1333450 not found: ID does not exist"
Oct 30 22:28:54.546242 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:28:54.546163    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-cf1826-54f5f6c8c4-d972s"]
Oct 30 22:28:54.550393 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:28:54.550363    5510 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-cf1826-54f5f6c8c4-d972s"]
Oct 30 22:28:55.254684 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:28:55.254541    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:28:55.485936 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:28:55.485892    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:28:55.555959 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:28:55.555926    5510 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="58572a25-fdb3-417c-9ade-f20e2d714b78" path="/var/lib/kubelet/pods/58572a25-fdb3-417c-9ade-f20e2d714b78/volumes"
Oct 30 22:28:55.685880 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:28:55.685846    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:28:55.911477 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:28:55.911379    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:28:56.500457 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:28:56.500416    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:28:59.711851 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:28:59.711807    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:29:00.097269 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:00.097226    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:29:00.778862 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:00.778828    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:29:02.912748 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:02.912696    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:29:03.187513 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:03.187429    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:29:03.935492 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:03.931907    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:29:03.988005 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:03.987970    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w5-3cc1b3-f5bd94dd5-pkc6c"]
Oct 30 22:29:04.096263 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:04.096229    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w5-3cc1b3-58b54f64ff-q8k4w"]
Oct 30 22:29:04.133352 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:04.133317    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:29:04.195378 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:04.195293    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-cf1826-openshift-default-cf9479bdc-jgkhc"]
Oct 30 22:29:04.195960 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:04.195924    5510 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w2-cf1826-openshift-default-cf9479bdc-jgkhc" podUID="13760c22-8ba0-45ed-b2e4-4e23d5273901" containerName="istio-proxy" containerID="cri-o://70eff1dd984d70819eb2acafb12e2d5504714cd7fb22aff55584bd04524f915b" gracePeriod=30
Oct 30 22:29:04.406681 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:04.406577    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:29:04.616823 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:04.616597    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:29:06.281896 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:06.281819    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:29:06.850599 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:06.850561    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:29:07.043166 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:07.043117    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:29:08.420712 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:29:08.420639    5510 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6gjbf" containerName="extractor"
Oct 30 22:29:08.989210 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:08.989141    5510 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w5-3cc1b3-f5bd94dd5-pkc6c" podUID="7f89f8ac-ad11-4aac-9f27-95d05190fd97" containerName="payload-processing" containerID="cri-o://3298ca3870d7ec5968d8437f46b95557e3053a6c4643e5cb8c41735dd0e2abeb" gracePeriod=25
Oct 30 22:29:09.096564 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:09.096524    5510 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w5-3cc1b3-58b54f64ff-q8k4w" podUID="d5825ebb-76c4-4a3a-aa17-2bbedab22772" containerName="payload-pre-processing" containerID="cri-o://72897713dc02410a1922c9888fd4a294b0f318d98846c5030ebb92eb08c0f5ea" gracePeriod=25
Oct 30 22:29:09.381522 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:09.381494    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w5-3cc1b3-f5bd94dd5-pkc6c"
Oct 30 22:29:09.400266 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:09.400233    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w5-3cc1b3-58b54f64ff-q8k4w"
Oct 30 22:29:09.441698 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:09.440408    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7f89f8ac-ad11-4aac-9f27-95d05190fd97-kube-api-access-t7wqz\" (UniqueName: \"kubernetes.io/projected/7f89f8ac-ad11-4aac-9f27-95d05190fd97-kube-api-access-t7wqz\") pod \"7f89f8ac-ad11-4aac-9f27-95d05190fd97\" (UID: \"7f89f8ac-ad11-4aac-9f27-95d05190fd97\") "
Oct 30 22:29:09.441698 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:09.440574    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7f89f8ac-ad11-4aac-9f27-95d05190fd97-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7f89f8ac-ad11-4aac-9f27-95d05190fd97-plugins-config-volume\") pod \"7f89f8ac-ad11-4aac-9f27-95d05190fd97\" (UID: \"7f89f8ac-ad11-4aac-9f27-95d05190fd97\") "
Oct 30 22:29:09.441698 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:09.441221    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7f89f8ac-ad11-4aac-9f27-95d05190fd97-plugins-config-volume" pod "7f89f8ac-ad11-4aac-9f27-95d05190fd97" (UID: "7f89f8ac-ad11-4aac-9f27-95d05190fd97"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:29:09.441698 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:09.441389    5510 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7f89f8ac-ad11-4aac-9f27-95d05190fd97-plugins-config-volume\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:29:09.442906 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:09.442882    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7f89f8ac-ad11-4aac-9f27-95d05190fd97-kube-api-access-t7wqz" pod "7f89f8ac-ad11-4aac-9f27-95d05190fd97" (UID: "7f89f8ac-ad11-4aac-9f27-95d05190fd97"). InnerVolumeSpecName "kube-api-access-t7wqz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:29:09.510479 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:09.510454    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-cf1826-openshift-default-cf9479bdc-jgkhc"
Oct 30 22:29:09.541987 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:09.541946    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d5825ebb-76c4-4a3a-aa17-2bbedab22772-kube-api-access-rsvdp\" (UniqueName: \"kubernetes.io/projected/d5825ebb-76c4-4a3a-aa17-2bbedab22772-kube-api-access-rsvdp\") pod \"d5825ebb-76c4-4a3a-aa17-2bbedab22772\" (UID: \"d5825ebb-76c4-4a3a-aa17-2bbedab22772\") "
Oct 30 22:29:09.542229 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:09.542212    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d5825ebb-76c4-4a3a-aa17-2bbedab22772-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d5825ebb-76c4-4a3a-aa17-2bbedab22772-plugins-config-volume\") pod \"d5825ebb-76c4-4a3a-aa17-2bbedab22772\" (UID: \"d5825ebb-76c4-4a3a-aa17-2bbedab22772\") "
Oct 30 22:29:09.542704 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:09.542651    5510 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t7wqz\" (UniqueName: \"kubernetes.io/projected/7f89f8ac-ad11-4aac-9f27-95d05190fd97-kube-api-access-t7wqz\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:29:09.542704 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:09.542682    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d5825ebb-76c4-4a3a-aa17-2bbedab22772-plugins-config-volume" pod "d5825ebb-76c4-4a3a-aa17-2bbedab22772" (UID: "d5825ebb-76c4-4a3a-aa17-2bbedab22772"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:29:09.545011 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:09.544987    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d5825ebb-76c4-4a3a-aa17-2bbedab22772-kube-api-access-rsvdp" pod "d5825ebb-76c4-4a3a-aa17-2bbedab22772" (UID: "d5825ebb-76c4-4a3a-aa17-2bbedab22772"). InnerVolumeSpecName "kube-api-access-rsvdp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:29:09.628190 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:09.628130    5510 generic.go:356] "Generic (PLEG): container finished" podID="d5825ebb-76c4-4a3a-aa17-2bbedab22772" containerID="72897713dc02410a1922c9888fd4a294b0f318d98846c5030ebb92eb08c0f5ea" exitCode=0
Oct 30 22:29:09.628403 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:09.628281    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w5-3cc1b3-58b54f64ff-q8k4w" event={"ID":"d5825ebb-76c4-4a3a-aa17-2bbedab22772","Type":"ContainerDied","Data":"72897713dc02410a1922c9888fd4a294b0f318d98846c5030ebb92eb08c0f5ea"}
Oct 30 22:29:09.628403 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:09.628332    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w5-3cc1b3-58b54f64ff-q8k4w" event={"ID":"d5825ebb-76c4-4a3a-aa17-2bbedab22772","Type":"ContainerDied","Data":"8a3e44f4b1de967cdf3173434fb40111e7d8b0ddc4c5aa2f64c4645c4be7eccf"}
Oct 30 22:29:09.628403 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:09.628362    5510 scope.go:122] "RemoveContainer" containerID="72897713dc02410a1922c9888fd4a294b0f318d98846c5030ebb92eb08c0f5ea"
Oct 30 22:29:09.628710 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:09.628507    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w5-3cc1b3-58b54f64ff-q8k4w"
Oct 30 22:29:09.633131 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:09.633109    5510 generic.go:356] "Generic (PLEG): container finished" podID="7f89f8ac-ad11-4aac-9f27-95d05190fd97" containerID="3298ca3870d7ec5968d8437f46b95557e3053a6c4643e5cb8c41735dd0e2abeb" exitCode=0
Oct 30 22:29:09.633341 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:09.633304    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w5-3cc1b3-f5bd94dd5-pkc6c"
Oct 30 22:29:09.633443 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:09.633141    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w5-3cc1b3-f5bd94dd5-pkc6c" event={"ID":"7f89f8ac-ad11-4aac-9f27-95d05190fd97","Type":"ContainerDied","Data":"3298ca3870d7ec5968d8437f46b95557e3053a6c4643e5cb8c41735dd0e2abeb"}
Oct 30 22:29:09.633443 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:09.633422    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w5-3cc1b3-f5bd94dd5-pkc6c" event={"ID":"7f89f8ac-ad11-4aac-9f27-95d05190fd97","Type":"ContainerDied","Data":"c39236d9de7e4efe987a9876c15f31dd9347fdb7844b20e06e7cedcfa2edaf01"}
Oct 30 22:29:09.636604 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:09.636578    5510 generic.go:356] "Generic (PLEG): container finished" podID="13760c22-8ba0-45ed-b2e4-4e23d5273901" containerID="70eff1dd984d70819eb2acafb12e2d5504714cd7fb22aff55584bd04524f915b" exitCode=0
Oct 30 22:29:09.636760 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:09.636630    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-cf1826-openshift-default-cf9479bdc-jgkhc" event={"ID":"13760c22-8ba0-45ed-b2e4-4e23d5273901","Type":"ContainerDied","Data":"70eff1dd984d70819eb2acafb12e2d5504714cd7fb22aff55584bd04524f915b"}
Oct 30 22:29:09.636760 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:09.636694    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-cf1826-openshift-default-cf9479bdc-jgkhc" event={"ID":"13760c22-8ba0-45ed-b2e4-4e23d5273901","Type":"ContainerDied","Data":"660a35f724c314692b44f27205866eb8ce26685930826dd76ce54bc20019cdcc"}
Oct 30 22:29:09.636902 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:09.636770    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-cf1826-openshift-default-cf9479bdc-jgkhc"
Oct 30 22:29:09.643666 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:09.643624    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/13760c22-8ba0-45ed-b2e4-4e23d5273901-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/13760c22-8ba0-45ed-b2e4-4e23d5273901-istio-data\") pod \"13760c22-8ba0-45ed-b2e4-4e23d5273901\" (UID: \"13760c22-8ba0-45ed-b2e4-4e23d5273901\") "
Oct 30 22:29:09.643872 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:09.643798    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/13760c22-8ba0-45ed-b2e4-4e23d5273901-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/13760c22-8ba0-45ed-b2e4-4e23d5273901-workload-certs\") pod \"13760c22-8ba0-45ed-b2e4-4e23d5273901\" (UID: \"13760c22-8ba0-45ed-b2e4-4e23d5273901\") "
Oct 30 22:29:09.643983 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:09.643894    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/13760c22-8ba0-45ed-b2e4-4e23d5273901-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/13760c22-8ba0-45ed-b2e4-4e23d5273901-workload-socket\") pod \"13760c22-8ba0-45ed-b2e4-4e23d5273901\" (UID: \"13760c22-8ba0-45ed-b2e4-4e23d5273901\") "
Oct 30 22:29:09.643983 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:09.643977    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/13760c22-8ba0-45ed-b2e4-4e23d5273901-kube-api-access-f6tfk\" (UniqueName: \"kubernetes.io/projected/13760c22-8ba0-45ed-b2e4-4e23d5273901-kube-api-access-f6tfk\") pod \"13760c22-8ba0-45ed-b2e4-4e23d5273901\" (UID: \"13760c22-8ba0-45ed-b2e4-4e23d5273901\") "
Oct 30 22:29:09.644142 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:09.644012    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/13760c22-8ba0-45ed-b2e4-4e23d5273901-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/13760c22-8ba0-45ed-b2e4-4e23d5273901-istio-envoy\") pod \"13760c22-8ba0-45ed-b2e4-4e23d5273901\" (UID: \"13760c22-8ba0-45ed-b2e4-4e23d5273901\") "
Oct 30 22:29:09.644142 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:09.644080    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/13760c22-8ba0-45ed-b2e4-4e23d5273901-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/13760c22-8ba0-45ed-b2e4-4e23d5273901-istiod-ca-cert\") pod \"13760c22-8ba0-45ed-b2e4-4e23d5273901\" (UID: \"13760c22-8ba0-45ed-b2e4-4e23d5273901\") "
Oct 30 22:29:09.644142 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:09.644127    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/13760c22-8ba0-45ed-b2e4-4e23d5273901-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/13760c22-8ba0-45ed-b2e4-4e23d5273901-credential-socket\") pod \"13760c22-8ba0-45ed-b2e4-4e23d5273901\" (UID: \"13760c22-8ba0-45ed-b2e4-4e23d5273901\") "
Oct 30 22:29:09.644343 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:09.644207    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/13760c22-8ba0-45ed-b2e4-4e23d5273901-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/13760c22-8ba0-45ed-b2e4-4e23d5273901-istio-podinfo\") pod \"13760c22-8ba0-45ed-b2e4-4e23d5273901\" (UID: \"13760c22-8ba0-45ed-b2e4-4e23d5273901\") "
Oct 30 22:29:09.644343 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:09.644293    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/13760c22-8ba0-45ed-b2e4-4e23d5273901-istio-token\" (UniqueName: \"kubernetes.io/projected/13760c22-8ba0-45ed-b2e4-4e23d5273901-istio-token\") pod \"13760c22-8ba0-45ed-b2e4-4e23d5273901\" (UID: \"13760c22-8ba0-45ed-b2e4-4e23d5273901\") "
Oct 30 22:29:09.644593 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:09.644483    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/13760c22-8ba0-45ed-b2e4-4e23d5273901-istio-data" pod "13760c22-8ba0-45ed-b2e4-4e23d5273901" (UID: "13760c22-8ba0-45ed-b2e4-4e23d5273901"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:29:09.645376 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:09.644887    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/13760c22-8ba0-45ed-b2e4-4e23d5273901-istiod-ca-cert" pod "13760c22-8ba0-45ed-b2e4-4e23d5273901" (UID: "13760c22-8ba0-45ed-b2e4-4e23d5273901"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:29:09.645376 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:09.644968    5510 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/13760c22-8ba0-45ed-b2e4-4e23d5273901-istio-data\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:29:09.645376 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:09.644990    5510 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d5825ebb-76c4-4a3a-aa17-2bbedab22772-plugins-config-volume\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:29:09.645376 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:09.645006    5510 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rsvdp\" (UniqueName: \"kubernetes.io/projected/d5825ebb-76c4-4a3a-aa17-2bbedab22772-kube-api-access-rsvdp\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:29:09.645376 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:09.645161    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/13760c22-8ba0-45ed-b2e4-4e23d5273901-workload-certs" pod "13760c22-8ba0-45ed-b2e4-4e23d5273901" (UID: "13760c22-8ba0-45ed-b2e4-4e23d5273901"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:29:09.645376 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:09.645217    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/13760c22-8ba0-45ed-b2e4-4e23d5273901-credential-socket" pod "13760c22-8ba0-45ed-b2e4-4e23d5273901" (UID: "13760c22-8ba0-45ed-b2e4-4e23d5273901"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:29:09.645376 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:09.645272    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/13760c22-8ba0-45ed-b2e4-4e23d5273901-workload-socket" pod "13760c22-8ba0-45ed-b2e4-4e23d5273901" (UID: "13760c22-8ba0-45ed-b2e4-4e23d5273901"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:29:09.645852 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:09.645674    5510 scope.go:122] "RemoveContainer" containerID="72897713dc02410a1922c9888fd4a294b0f318d98846c5030ebb92eb08c0f5ea"
Oct 30 22:29:09.646098 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:29:09.646034    5510 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"72897713dc02410a1922c9888fd4a294b0f318d98846c5030ebb92eb08c0f5ea\": container with ID starting with 72897713dc02410a1922c9888fd4a294b0f318d98846c5030ebb92eb08c0f5ea not found: ID does not exist" containerID="72897713dc02410a1922c9888fd4a294b0f318d98846c5030ebb92eb08c0f5ea"
Oct 30 22:29:09.646174 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:09.646109    5510 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"72897713dc02410a1922c9888fd4a294b0f318d98846c5030ebb92eb08c0f5ea"} err="failed to get container status \"72897713dc02410a1922c9888fd4a294b0f318d98846c5030ebb92eb08c0f5ea\": rpc error: code = NotFound desc = could not find container \"72897713dc02410a1922c9888fd4a294b0f318d98846c5030ebb92eb08c0f5ea\": container with ID starting with 72897713dc02410a1922c9888fd4a294b0f318d98846c5030ebb92eb08c0f5ea not found: ID does not exist"
Oct 30 22:29:09.646174 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:09.646133    5510 scope.go:122] "RemoveContainer" containerID="3298ca3870d7ec5968d8437f46b95557e3053a6c4643e5cb8c41735dd0e2abeb"
Oct 30 22:29:09.647392 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:09.647267    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/13760c22-8ba0-45ed-b2e4-4e23d5273901-istio-token" pod "13760c22-8ba0-45ed-b2e4-4e23d5273901" (UID: "13760c22-8ba0-45ed-b2e4-4e23d5273901"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:29:09.647728 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:09.647703    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/13760c22-8ba0-45ed-b2e4-4e23d5273901-kube-api-access-f6tfk" pod "13760c22-8ba0-45ed-b2e4-4e23d5273901" (UID: "13760c22-8ba0-45ed-b2e4-4e23d5273901"). InnerVolumeSpecName "kube-api-access-f6tfk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:29:09.647839 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:09.647750    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/13760c22-8ba0-45ed-b2e4-4e23d5273901-istio-podinfo" pod "13760c22-8ba0-45ed-b2e4-4e23d5273901" (UID: "13760c22-8ba0-45ed-b2e4-4e23d5273901"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:29:09.648410 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:09.648385    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/13760c22-8ba0-45ed-b2e4-4e23d5273901-istio-envoy" pod "13760c22-8ba0-45ed-b2e4-4e23d5273901" (UID: "13760c22-8ba0-45ed-b2e4-4e23d5273901"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:29:09.658275 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:09.658250    5510 scope.go:122] "RemoveContainer" containerID="3298ca3870d7ec5968d8437f46b95557e3053a6c4643e5cb8c41735dd0e2abeb"
Oct 30 22:29:09.658546 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:29:09.658527    5510 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3298ca3870d7ec5968d8437f46b95557e3053a6c4643e5cb8c41735dd0e2abeb\": container with ID starting with 3298ca3870d7ec5968d8437f46b95557e3053a6c4643e5cb8c41735dd0e2abeb not found: ID does not exist" containerID="3298ca3870d7ec5968d8437f46b95557e3053a6c4643e5cb8c41735dd0e2abeb"
Oct 30 22:29:09.658603 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:09.658555    5510 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3298ca3870d7ec5968d8437f46b95557e3053a6c4643e5cb8c41735dd0e2abeb"} err="failed to get container status \"3298ca3870d7ec5968d8437f46b95557e3053a6c4643e5cb8c41735dd0e2abeb\": rpc error: code = NotFound desc = could not find container \"3298ca3870d7ec5968d8437f46b95557e3053a6c4643e5cb8c41735dd0e2abeb\": container with ID starting with 3298ca3870d7ec5968d8437f46b95557e3053a6c4643e5cb8c41735dd0e2abeb not found: ID does not exist"
Oct 30 22:29:09.658603 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:09.658577    5510 scope.go:122] "RemoveContainer" containerID="70eff1dd984d70819eb2acafb12e2d5504714cd7fb22aff55584bd04524f915b"
Oct 30 22:29:09.668322 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:09.668307    5510 scope.go:122] "RemoveContainer" containerID="f572ac7fae6b82a9bf28a0c16eee7c0515f93747676b59f2afe6e114fda3a496"
Oct 30 22:29:09.677594 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:09.677578    5510 scope.go:122] "RemoveContainer" containerID="70eff1dd984d70819eb2acafb12e2d5504714cd7fb22aff55584bd04524f915b"
Oct 30 22:29:09.677954 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:29:09.677935    5510 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"70eff1dd984d70819eb2acafb12e2d5504714cd7fb22aff55584bd04524f915b\": container with ID starting with 70eff1dd984d70819eb2acafb12e2d5504714cd7fb22aff55584bd04524f915b not found: ID does not exist" containerID="70eff1dd984d70819eb2acafb12e2d5504714cd7fb22aff55584bd04524f915b"
Oct 30 22:29:09.678024 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:09.677962    5510 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"70eff1dd984d70819eb2acafb12e2d5504714cd7fb22aff55584bd04524f915b"} err="failed to get container status \"70eff1dd984d70819eb2acafb12e2d5504714cd7fb22aff55584bd04524f915b\": rpc error: code = NotFound desc = could not find container \"70eff1dd984d70819eb2acafb12e2d5504714cd7fb22aff55584bd04524f915b\": container with ID starting with 70eff1dd984d70819eb2acafb12e2d5504714cd7fb22aff55584bd04524f915b not found: ID does not exist"
Oct 30 22:29:09.678024 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:09.677977    5510 scope.go:122] "RemoveContainer" containerID="f572ac7fae6b82a9bf28a0c16eee7c0515f93747676b59f2afe6e114fda3a496"
Oct 30 22:29:09.678171 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:29:09.678153    5510 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f572ac7fae6b82a9bf28a0c16eee7c0515f93747676b59f2afe6e114fda3a496\": container with ID starting with f572ac7fae6b82a9bf28a0c16eee7c0515f93747676b59f2afe6e114fda3a496 not found: ID does not exist" containerID="f572ac7fae6b82a9bf28a0c16eee7c0515f93747676b59f2afe6e114fda3a496"
Oct 30 22:29:09.678210 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:09.678175    5510 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f572ac7fae6b82a9bf28a0c16eee7c0515f93747676b59f2afe6e114fda3a496"} err="failed to get container status \"f572ac7fae6b82a9bf28a0c16eee7c0515f93747676b59f2afe6e114fda3a496\": rpc error: code = NotFound desc = could not find container \"f572ac7fae6b82a9bf28a0c16eee7c0515f93747676b59f2afe6e114fda3a496\": container with ID starting with f572ac7fae6b82a9bf28a0c16eee7c0515f93747676b59f2afe6e114fda3a496 not found: ID does not exist"
Oct 30 22:29:09.746368 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:09.746323    5510 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/13760c22-8ba0-45ed-b2e4-4e23d5273901-workload-socket\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:29:09.746368 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:09.746365    5510 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f6tfk\" (UniqueName: \"kubernetes.io/projected/13760c22-8ba0-45ed-b2e4-4e23d5273901-kube-api-access-f6tfk\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:29:09.746581 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:09.746383    5510 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/13760c22-8ba0-45ed-b2e4-4e23d5273901-istio-envoy\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:29:09.746581 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:09.746398    5510 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/13760c22-8ba0-45ed-b2e4-4e23d5273901-istiod-ca-cert\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:29:09.746581 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:09.746412    5510 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/13760c22-8ba0-45ed-b2e4-4e23d5273901-credential-socket\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:29:09.746581 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:09.746425    5510 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/13760c22-8ba0-45ed-b2e4-4e23d5273901-istio-podinfo\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:29:09.746581 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:09.746438    5510 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/13760c22-8ba0-45ed-b2e4-4e23d5273901-istio-token\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:29:09.746581 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:09.746453    5510 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/13760c22-8ba0-45ed-b2e4-4e23d5273901-workload-certs\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:29:09.788489 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:09.788453    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-595fd7d7-5d9b87ff4c-x2njg"]
Oct 30 22:29:09.835591 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:09.835557    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w5-3cc1b3-f5bd94dd5-pkc6c"]
Oct 30 22:29:09.842476 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:09.842438    5510 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w5-3cc1b3-f5bd94dd5-pkc6c"]
Oct 30 22:29:09.998450 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:09.998411    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w5-3cc1b3-58b54f64ff-q8k4w"]
Oct 30 22:29:10.003982 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:10.003946    5510 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w5-3cc1b3-58b54f64ff-q8k4w"]
Oct 30 22:29:10.120017 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:10.119343    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:29:10.265953 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:10.264280    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-cf1826-openshift-default-cf9479bdc-jgkhc"]
Oct 30 22:29:10.277027 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:10.276971    5510 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-cf1826-openshift-default-cf9479bdc-jgkhc"]
Oct 30 22:29:10.376690 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:10.376570    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:29:10.626682 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:10.625943    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:29:11.554871 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:11.554832    5510 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="13760c22-8ba0-45ed-b2e4-4e23d5273901" path="/var/lib/kubelet/pods/13760c22-8ba0-45ed-b2e4-4e23d5273901/volumes"
Oct 30 22:29:11.555447 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:11.555424    5510 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7f89f8ac-ad11-4aac-9f27-95d05190fd97" path="/var/lib/kubelet/pods/7f89f8ac-ad11-4aac-9f27-95d05190fd97/volumes"
Oct 30 22:29:11.555942 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:11.555922    5510 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d5825ebb-76c4-4a3a-aa17-2bbedab22772" path="/var/lib/kubelet/pods/d5825ebb-76c4-4a3a-aa17-2bbedab22772/volumes"
Oct 30 22:29:12.354881 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:12.354688    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:29:13.220839 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:13.220804    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:29:13.551808 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:13.551768    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:29:14.142490 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:14.142457    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:29:14.430766 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:14.429815    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:29:14.788994 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:14.788942    5510 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-595fd7d7-5d9b87ff4c-x2njg" podUID="b112302e-a8df-4573-a151-696453d1e7ee" containerName="payload-pre-processing" containerID="cri-o://937ca6f5a81d90209fc1dac15333e6b99b32e184cca16a13ddbb4c989e17306d" gracePeriod=25
Oct 30 22:29:15.101725 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:15.076855    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:29:15.158034 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:15.158003    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-595fd7d7-5d9b87ff4c-x2njg"
Oct 30 22:29:15.162334 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:15.162030    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b112302e-a8df-4573-a151-696453d1e7ee-kube-api-access-g85st\" (UniqueName: \"kubernetes.io/projected/b112302e-a8df-4573-a151-696453d1e7ee-kube-api-access-g85st\") pod \"b112302e-a8df-4573-a151-696453d1e7ee\" (UID: \"b112302e-a8df-4573-a151-696453d1e7ee\") "
Oct 30 22:29:15.162334 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:15.162226    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b112302e-a8df-4573-a151-696453d1e7ee-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b112302e-a8df-4573-a151-696453d1e7ee-plugins-config-volume\") pod \"b112302e-a8df-4573-a151-696453d1e7ee\" (UID: \"b112302e-a8df-4573-a151-696453d1e7ee\") "
Oct 30 22:29:15.162621 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:15.162596    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b112302e-a8df-4573-a151-696453d1e7ee-plugins-config-volume" pod "b112302e-a8df-4573-a151-696453d1e7ee" (UID: "b112302e-a8df-4573-a151-696453d1e7ee"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:29:15.162847 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:15.162786    5510 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b112302e-a8df-4573-a151-696453d1e7ee-plugins-config-volume\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:29:15.164925 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:15.164895    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b112302e-a8df-4573-a151-696453d1e7ee-kube-api-access-g85st" pod "b112302e-a8df-4573-a151-696453d1e7ee" (UID: "b112302e-a8df-4573-a151-696453d1e7ee"). InnerVolumeSpecName "kube-api-access-g85st". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:29:15.263644 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:15.263606    5510 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g85st\" (UniqueName: \"kubernetes.io/projected/b112302e-a8df-4573-a151-696453d1e7ee-kube-api-access-g85st\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:29:15.409646 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:15.409564    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-9f3ea1-67cc67f7bf-sw4km"]
Oct 30 22:29:15.438268 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:15.438231    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:29:15.511138 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:15.511097    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-9f3ea1-6cf88f97f5-24dql"]
Oct 30 22:29:15.721707 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:15.721588    5510 generic.go:356] "Generic (PLEG): container finished" podID="b112302e-a8df-4573-a151-696453d1e7ee" containerID="937ca6f5a81d90209fc1dac15333e6b99b32e184cca16a13ddbb4c989e17306d" exitCode=0
Oct 30 22:29:15.721892 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:15.721765    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-595fd7d7-5d9b87ff4c-x2njg" event={"ID":"b112302e-a8df-4573-a151-696453d1e7ee","Type":"ContainerDied","Data":"937ca6f5a81d90209fc1dac15333e6b99b32e184cca16a13ddbb4c989e17306d"}
Oct 30 22:29:15.721892 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:15.721819    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-595fd7d7-5d9b87ff4c-x2njg" event={"ID":"b112302e-a8df-4573-a151-696453d1e7ee","Type":"ContainerDied","Data":"21de033cee9c5396d8c8c35ead3ba9b67b9660f3807a3f8483adcdc2e00fe47d"}
Oct 30 22:29:15.721892 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:15.721850    5510 scope.go:122] "RemoveContainer" containerID="937ca6f5a81d90209fc1dac15333e6b99b32e184cca16a13ddbb4c989e17306d"
Oct 30 22:29:15.722062 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:15.722004    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-595fd7d7-5d9b87ff4c-x2njg"
Oct 30 22:29:15.737152 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:15.737129    5510 scope.go:122] "RemoveContainer" containerID="937ca6f5a81d90209fc1dac15333e6b99b32e184cca16a13ddbb4c989e17306d"
Oct 30 22:29:15.737470 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:29:15.737449    5510 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"937ca6f5a81d90209fc1dac15333e6b99b32e184cca16a13ddbb4c989e17306d\": container with ID starting with 937ca6f5a81d90209fc1dac15333e6b99b32e184cca16a13ddbb4c989e17306d not found: ID does not exist" containerID="937ca6f5a81d90209fc1dac15333e6b99b32e184cca16a13ddbb4c989e17306d"
Oct 30 22:29:15.737518 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:15.737483    5510 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"937ca6f5a81d90209fc1dac15333e6b99b32e184cca16a13ddbb4c989e17306d"} err="failed to get container status \"937ca6f5a81d90209fc1dac15333e6b99b32e184cca16a13ddbb4c989e17306d\": rpc error: code = NotFound desc = could not find container \"937ca6f5a81d90209fc1dac15333e6b99b32e184cca16a13ddbb4c989e17306d\": container with ID starting with 937ca6f5a81d90209fc1dac15333e6b99b32e184cca16a13ddbb4c989e17306d not found: ID does not exist"
Oct 30 22:29:15.832185 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:15.832151    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:29:15.879671 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:15.879617    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-595fd7d7-5d9b87ff4c-x2njg"]
Oct 30 22:29:15.885429 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:15.885403    5510 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-595fd7d7-5d9b87ff4c-x2njg"]
Oct 30 22:29:16.032325 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:16.032284    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:29:17.517721 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:17.514550    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:29:17.562466 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:17.562430    5510 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b112302e-a8df-4573-a151-696453d1e7ee" path="/var/lib/kubelet/pods/b112302e-a8df-4573-a151-696453d1e7ee/volumes"
Oct 30 22:29:20.410631 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:20.410588    5510 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-9f3ea1-67cc67f7bf-sw4km" podUID="ecfca1f1-d4b8-421b-ac73-3b726a8afdf4" containerName="payload-processing" containerID="cri-o://248a30a993fbee3b65eb019dcb18e4f644a9dac99ea09596d223a69f700d415d" gracePeriod=25
Oct 30 22:29:20.511684 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:20.511631    5510 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-9f3ea1-6cf88f97f5-24dql" podUID="0796eecd-5044-4c1b-981b-14e8a16c4a9c" containerName="payload-pre-processing" containerID="cri-o://2ec718b390c6bdc61ef3cb7ff35497233b820dc4fd94553ce7a84f1524c5a5d6" gracePeriod=25
Oct 30 22:29:20.703562 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:20.703533    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-9f3ea1-67cc67f7bf-sw4km"
Oct 30 22:29:20.742143 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:20.741140    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ecfca1f1-d4b8-421b-ac73-3b726a8afdf4-kube-api-access-mb42j\" (UniqueName: \"kubernetes.io/projected/ecfca1f1-d4b8-421b-ac73-3b726a8afdf4-kube-api-access-mb42j\") pod \"ecfca1f1-d4b8-421b-ac73-3b726a8afdf4\" (UID: \"ecfca1f1-d4b8-421b-ac73-3b726a8afdf4\") "
Oct 30 22:29:20.742143 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:20.741366    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ecfca1f1-d4b8-421b-ac73-3b726a8afdf4-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ecfca1f1-d4b8-421b-ac73-3b726a8afdf4-plugins-config-volume\") pod \"ecfca1f1-d4b8-421b-ac73-3b726a8afdf4\" (UID: \"ecfca1f1-d4b8-421b-ac73-3b726a8afdf4\") "
Oct 30 22:29:20.742512 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:20.742470    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ecfca1f1-d4b8-421b-ac73-3b726a8afdf4-plugins-config-volume" pod "ecfca1f1-d4b8-421b-ac73-3b726a8afdf4" (UID: "ecfca1f1-d4b8-421b-ac73-3b726a8afdf4"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:29:20.744861 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:20.744828    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ecfca1f1-d4b8-421b-ac73-3b726a8afdf4-kube-api-access-mb42j" pod "ecfca1f1-d4b8-421b-ac73-3b726a8afdf4" (UID: "ecfca1f1-d4b8-421b-ac73-3b726a8afdf4"). InnerVolumeSpecName "kube-api-access-mb42j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:29:20.767849 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:20.767816    5510 generic.go:356] "Generic (PLEG): container finished" podID="ecfca1f1-d4b8-421b-ac73-3b726a8afdf4" containerID="248a30a993fbee3b65eb019dcb18e4f644a9dac99ea09596d223a69f700d415d" exitCode=0
Oct 30 22:29:20.767986 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:20.767876    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-9f3ea1-67cc67f7bf-sw4km"
Oct 30 22:29:20.768191 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:20.768170    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-9f3ea1-67cc67f7bf-sw4km" event={"ID":"ecfca1f1-d4b8-421b-ac73-3b726a8afdf4","Type":"ContainerDied","Data":"248a30a993fbee3b65eb019dcb18e4f644a9dac99ea09596d223a69f700d415d"}
Oct 30 22:29:20.768280 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:20.768224    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-9f3ea1-67cc67f7bf-sw4km" event={"ID":"ecfca1f1-d4b8-421b-ac73-3b726a8afdf4","Type":"ContainerDied","Data":"b997fe48c474ab3b126bdb8895958ada492565cdf93170c75e6a57b51c91795e"}
Oct 30 22:29:20.768280 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:20.768255    5510 scope.go:122] "RemoveContainer" containerID="248a30a993fbee3b65eb019dcb18e4f644a9dac99ea09596d223a69f700d415d"
Oct 30 22:29:20.770972 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:20.770860    5510 generic.go:356] "Generic (PLEG): container finished" podID="0796eecd-5044-4c1b-981b-14e8a16c4a9c" containerID="2ec718b390c6bdc61ef3cb7ff35497233b820dc4fd94553ce7a84f1524c5a5d6" exitCode=0
Oct 30 22:29:20.770972 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:20.770910    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-9f3ea1-6cf88f97f5-24dql" event={"ID":"0796eecd-5044-4c1b-981b-14e8a16c4a9c","Type":"ContainerDied","Data":"2ec718b390c6bdc61ef3cb7ff35497233b820dc4fd94553ce7a84f1524c5a5d6"}
Oct 30 22:29:20.782558 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:20.782535    5510 scope.go:122] "RemoveContainer" containerID="248a30a993fbee3b65eb019dcb18e4f644a9dac99ea09596d223a69f700d415d"
Oct 30 22:29:20.783226 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:29:20.783201    5510 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"248a30a993fbee3b65eb019dcb18e4f644a9dac99ea09596d223a69f700d415d\": container with ID starting with 248a30a993fbee3b65eb019dcb18e4f644a9dac99ea09596d223a69f700d415d not found: ID does not exist" containerID="248a30a993fbee3b65eb019dcb18e4f644a9dac99ea09596d223a69f700d415d"
Oct 30 22:29:20.783314 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:20.783238    5510 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"248a30a993fbee3b65eb019dcb18e4f644a9dac99ea09596d223a69f700d415d"} err="failed to get container status \"248a30a993fbee3b65eb019dcb18e4f644a9dac99ea09596d223a69f700d415d\": rpc error: code = NotFound desc = could not find container \"248a30a993fbee3b65eb019dcb18e4f644a9dac99ea09596d223a69f700d415d\": container with ID starting with 248a30a993fbee3b65eb019dcb18e4f644a9dac99ea09596d223a69f700d415d not found: ID does not exist"
Oct 30 22:29:20.799697 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:20.799672    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-9f3ea1-6cf88f97f5-24dql"
Oct 30 22:29:20.843000 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:20.842953    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0796eecd-5044-4c1b-981b-14e8a16c4a9c-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0796eecd-5044-4c1b-981b-14e8a16c4a9c-plugins-config-volume\") pod \"0796eecd-5044-4c1b-981b-14e8a16c4a9c\" (UID: \"0796eecd-5044-4c1b-981b-14e8a16c4a9c\") "
Oct 30 22:29:20.843271 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:20.843249    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0796eecd-5044-4c1b-981b-14e8a16c4a9c-kube-api-access-k26cf\" (UniqueName: \"kubernetes.io/projected/0796eecd-5044-4c1b-981b-14e8a16c4a9c-kube-api-access-k26cf\") pod \"0796eecd-5044-4c1b-981b-14e8a16c4a9c\" (UID: \"0796eecd-5044-4c1b-981b-14e8a16c4a9c\") "
Oct 30 22:29:20.843361 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:20.843303    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0796eecd-5044-4c1b-981b-14e8a16c4a9c-plugins-config-volume" pod "0796eecd-5044-4c1b-981b-14e8a16c4a9c" (UID: "0796eecd-5044-4c1b-981b-14e8a16c4a9c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:29:20.843945 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:20.843918    5510 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ecfca1f1-d4b8-421b-ac73-3b726a8afdf4-plugins-config-volume\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:29:20.844045 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:20.843948    5510 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0796eecd-5044-4c1b-981b-14e8a16c4a9c-plugins-config-volume\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:29:20.844045 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:20.843963    5510 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mb42j\" (UniqueName: \"kubernetes.io/projected/ecfca1f1-d4b8-421b-ac73-3b726a8afdf4-kube-api-access-mb42j\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:29:20.845500 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:20.845476    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0796eecd-5044-4c1b-981b-14e8a16c4a9c-kube-api-access-k26cf" pod "0796eecd-5044-4c1b-981b-14e8a16c4a9c" (UID: "0796eecd-5044-4c1b-981b-14e8a16c4a9c"). InnerVolumeSpecName "kube-api-access-k26cf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:29:20.946038 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:20.945943    5510 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k26cf\" (UniqueName: \"kubernetes.io/projected/0796eecd-5044-4c1b-981b-14e8a16c4a9c-kube-api-access-k26cf\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:29:21.036266 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:21.036235    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-9f3ea1-67cc67f7bf-sw4km"]
Oct 30 22:29:21.040242 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:21.040210    5510 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-9f3ea1-67cc67f7bf-sw4km"]
Oct 30 22:29:21.554909 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:21.554875    5510 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ecfca1f1-d4b8-421b-ac73-3b726a8afdf4" path="/var/lib/kubelet/pods/ecfca1f1-d4b8-421b-ac73-3b726a8afdf4/volumes"
Oct 30 22:29:21.779879 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:21.779849    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-9f3ea1-6cf88f97f5-24dql" event={"ID":"0796eecd-5044-4c1b-981b-14e8a16c4a9c","Type":"ContainerDied","Data":"fbd1f491530f559d5b2c0890b1cb0ce9266866b924a73655923b242a8b0956d4"}
Oct 30 22:29:21.780082 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:21.779886    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-9f3ea1-6cf88f97f5-24dql"
Oct 30 22:29:21.780082 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:21.779906    5510 scope.go:122] "RemoveContainer" containerID="2ec718b390c6bdc61ef3cb7ff35497233b820dc4fd94553ce7a84f1524c5a5d6"
Oct 30 22:29:21.787666 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:21.787590    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:29:21.940044 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:21.940014    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-9f3ea1-6cf88f97f5-24dql"]
Oct 30 22:29:21.986378 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:21.986343    5510 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-9f3ea1-6cf88f97f5-24dql"]
Oct 30 22:29:22.014698 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:22.014618    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:29:22.261737 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:22.248967    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:29:22.464902 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:22.464868    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:29:22.785170 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:22.785133    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:29:23.023997 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:23.023965    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:29:23.638368 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:23.625293    5510 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0796eecd-5044-4c1b-981b-14e8a16c4a9c" path="/var/lib/kubelet/pods/0796eecd-5044-4c1b-981b-14e8a16c4a9c/volumes"
Oct 30 22:29:23.696680 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:23.696383    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:29:24.131531 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:24.131497    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:29:27.035681 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:27.028478    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:29:28.011786 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:28.011741    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:29:28.271195 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:28.271163    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:29:28.918835 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:28.918800    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:29:31.166251 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:31.166169    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:29:31.258606 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:31.258570    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-3cc1b3-6b779bbcc6-dmbbd"]
Oct 30 22:29:31.258870 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:31.258840    5510 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-3cc1b3-6b779bbcc6-dmbbd" podUID="f7fdf681-44d0-45d0-b2bd-c8e4df1af3bd" containerName="maas-api" containerID="cri-o://d234967cdbe954b92f8b290d34db24100d622ec488fe62088c69b1796f7ccbf4" gracePeriod=30
Oct 30 22:29:31.535221 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:31.535192    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-3cc1b3-6b779bbcc6-dmbbd"
Oct 30 22:29:31.593274 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:31.593239    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f7fdf681-44d0-45d0-b2bd-c8e4df1af3bd-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f7fdf681-44d0-45d0-b2bd-c8e4df1af3bd-maas-api-tls\") pod \"f7fdf681-44d0-45d0-b2bd-c8e4df1af3bd\" (UID: \"f7fdf681-44d0-45d0-b2bd-c8e4df1af3bd\") "
Oct 30 22:29:31.593448 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:31.593332    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f7fdf681-44d0-45d0-b2bd-c8e4df1af3bd-metrics-certs\" (UniqueName: \"kubernetes.io/secret/f7fdf681-44d0-45d0-b2bd-c8e4df1af3bd-metrics-certs\") pod \"f7fdf681-44d0-45d0-b2bd-c8e4df1af3bd\" (UID: \"f7fdf681-44d0-45d0-b2bd-c8e4df1af3bd\") "
Oct 30 22:29:31.593516 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:31.593461    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f7fdf681-44d0-45d0-b2bd-c8e4df1af3bd-kube-api-access-cvngh\" (UniqueName: \"kubernetes.io/projected/f7fdf681-44d0-45d0-b2bd-c8e4df1af3bd-kube-api-access-cvngh\") pod \"f7fdf681-44d0-45d0-b2bd-c8e4df1af3bd\" (UID: \"f7fdf681-44d0-45d0-b2bd-c8e4df1af3bd\") "
Oct 30 22:29:31.593575 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:31.593563    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f7fdf681-44d0-45d0-b2bd-c8e4df1af3bd-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f7fdf681-44d0-45d0-b2bd-c8e4df1af3bd-openshift-service-ca\") pod \"f7fdf681-44d0-45d0-b2bd-c8e4df1af3bd\" (UID: \"f7fdf681-44d0-45d0-b2bd-c8e4df1af3bd\") "
Oct 30 22:29:31.594394 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:31.593993    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f7fdf681-44d0-45d0-b2bd-c8e4df1af3bd-openshift-service-ca" pod "f7fdf681-44d0-45d0-b2bd-c8e4df1af3bd" (UID: "f7fdf681-44d0-45d0-b2bd-c8e4df1af3bd"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:29:31.595519 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:31.595492    5510 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f7fdf681-44d0-45d0-b2bd-c8e4df1af3bd-openshift-service-ca\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:29:31.596062 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:31.596040    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f7fdf681-44d0-45d0-b2bd-c8e4df1af3bd-maas-api-tls" pod "f7fdf681-44d0-45d0-b2bd-c8e4df1af3bd" (UID: "f7fdf681-44d0-45d0-b2bd-c8e4df1af3bd"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:29:31.596386 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:31.596348    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f7fdf681-44d0-45d0-b2bd-c8e4df1af3bd-metrics-certs" pod "f7fdf681-44d0-45d0-b2bd-c8e4df1af3bd" (UID: "f7fdf681-44d0-45d0-b2bd-c8e4df1af3bd"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:29:31.597330 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:31.597307    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f7fdf681-44d0-45d0-b2bd-c8e4df1af3bd-kube-api-access-cvngh" pod "f7fdf681-44d0-45d0-b2bd-c8e4df1af3bd" (UID: "f7fdf681-44d0-45d0-b2bd-c8e4df1af3bd"). InnerVolumeSpecName "kube-api-access-cvngh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:29:31.697355 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:31.697273    5510 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f7fdf681-44d0-45d0-b2bd-c8e4df1af3bd-maas-api-tls\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:29:31.697355 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:31.697305    5510 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f7fdf681-44d0-45d0-b2bd-c8e4df1af3bd-metrics-certs\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:29:31.697355 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:31.697316    5510 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cvngh\" (UniqueName: \"kubernetes.io/projected/f7fdf681-44d0-45d0-b2bd-c8e4df1af3bd-kube-api-access-cvngh\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:29:31.933681 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:31.926692    5510 generic.go:356] "Generic (PLEG): container finished" podID="f7fdf681-44d0-45d0-b2bd-c8e4df1af3bd" containerID="d234967cdbe954b92f8b290d34db24100d622ec488fe62088c69b1796f7ccbf4" exitCode=0
Oct 30 22:29:31.933681 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:31.926846    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-3cc1b3-6b779bbcc6-dmbbd" event={"ID":"f7fdf681-44d0-45d0-b2bd-c8e4df1af3bd","Type":"ContainerDied","Data":"d234967cdbe954b92f8b290d34db24100d622ec488fe62088c69b1796f7ccbf4"}
Oct 30 22:29:31.933681 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:31.926889    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-3cc1b3-6b779bbcc6-dmbbd"
Oct 30 22:29:31.933681 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:31.926905    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-3cc1b3-6b779bbcc6-dmbbd" event={"ID":"f7fdf681-44d0-45d0-b2bd-c8e4df1af3bd","Type":"ContainerDied","Data":"8127b28ec5d42f7492fa5b8c72ed985728837b2ab1e358a2a59ea8c397c230cd"}
Oct 30 22:29:31.933681 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:31.926936    5510 scope.go:122] "RemoveContainer" containerID="d234967cdbe954b92f8b290d34db24100d622ec488fe62088c69b1796f7ccbf4"
Oct 30 22:29:32.026677 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:32.024827    5510 scope.go:122] "RemoveContainer" containerID="d234967cdbe954b92f8b290d34db24100d622ec488fe62088c69b1796f7ccbf4"
Oct 30 22:29:32.026677 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:29:32.025342    5510 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d234967cdbe954b92f8b290d34db24100d622ec488fe62088c69b1796f7ccbf4\": container with ID starting with d234967cdbe954b92f8b290d34db24100d622ec488fe62088c69b1796f7ccbf4 not found: ID does not exist" containerID="d234967cdbe954b92f8b290d34db24100d622ec488fe62088c69b1796f7ccbf4"
Oct 30 22:29:32.026677 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:32.025380    5510 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d234967cdbe954b92f8b290d34db24100d622ec488fe62088c69b1796f7ccbf4"} err="failed to get container status \"d234967cdbe954b92f8b290d34db24100d622ec488fe62088c69b1796f7ccbf4\": rpc error: code = NotFound desc = could not find container \"d234967cdbe954b92f8b290d34db24100d622ec488fe62088c69b1796f7ccbf4\": container with ID starting with d234967cdbe954b92f8b290d34db24100d622ec488fe62088c69b1796f7ccbf4 not found: ID does not exist"
Oct 30 22:29:32.139307 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:32.139270    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-3cc1b3-6b779bbcc6-dmbbd"]
Oct 30 22:29:32.184872 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:32.184836    5510 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-3cc1b3-6b779bbcc6-dmbbd"]
Oct 30 22:29:32.674219 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:32.674191    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:29:32.993167 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:32.993080    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:29:33.297291 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:33.297165    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:29:33.497227 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:33.497192    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:29:33.554478 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:33.554405    5510 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f7fdf681-44d0-45d0-b2bd-c8e4df1af3bd" path="/var/lib/kubelet/pods/f7fdf681-44d0-45d0-b2bd-c8e4df1af3bd/volumes"
Oct 30 22:29:33.931009 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:33.930926    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:29:35.889125 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:35.889059    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:29:36.143086 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:36.143005    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:29:38.296166 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:38.296123    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:29:38.422323 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:29:38.422274    5510 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6gjbf" containerName="extractor"
Oct 30 22:29:40.278024 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:40.277977    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:29:40.474845 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:40.474812    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:29:40.688301 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:40.688210    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:29:43.402678 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:43.386060    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:29:47.533243 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:47.533207    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-7bb1c2-6746d48b7f-8bqqt"]
Oct 30 22:29:47.535946 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:47.533504    5510 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-7bb1c2-6746d48b7f-8bqqt" podUID="3266de62-2593-4496-b3e6-7600ebb710ce" containerName="maas-api" containerID="cri-o://6410f2ac44e9b1c8e76dcf326d66cd9b1d2bcf9b7d51e52dc499695b93a5df4f" gracePeriod=30
Oct 30 22:29:47.656816 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:47.656784    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:29:47.874760 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:47.874736    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-7bb1c2-6746d48b7f-8bqqt"
Oct 30 22:29:47.965681 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:47.965200    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3266de62-2593-4496-b3e6-7600ebb710ce-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3266de62-2593-4496-b3e6-7600ebb710ce-maas-api-tls\") pod \"3266de62-2593-4496-b3e6-7600ebb710ce\" (UID: \"3266de62-2593-4496-b3e6-7600ebb710ce\") "
Oct 30 22:29:47.965681 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:47.965449    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3266de62-2593-4496-b3e6-7600ebb710ce-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3266de62-2593-4496-b3e6-7600ebb710ce-openshift-service-ca\") pod \"3266de62-2593-4496-b3e6-7600ebb710ce\" (UID: \"3266de62-2593-4496-b3e6-7600ebb710ce\") "
Oct 30 22:29:47.965681 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:47.965502    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3266de62-2593-4496-b3e6-7600ebb710ce-kube-api-access-2d9nr\" (UniqueName: \"kubernetes.io/projected/3266de62-2593-4496-b3e6-7600ebb710ce-kube-api-access-2d9nr\") pod \"3266de62-2593-4496-b3e6-7600ebb710ce\" (UID: \"3266de62-2593-4496-b3e6-7600ebb710ce\") "
Oct 30 22:29:47.965681 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:47.965589    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3266de62-2593-4496-b3e6-7600ebb710ce-metrics-certs\" (UniqueName: \"kubernetes.io/secret/3266de62-2593-4496-b3e6-7600ebb710ce-metrics-certs\") pod \"3266de62-2593-4496-b3e6-7600ebb710ce\" (UID: \"3266de62-2593-4496-b3e6-7600ebb710ce\") "
Oct 30 22:29:47.975675 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:47.967060    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3266de62-2593-4496-b3e6-7600ebb710ce-openshift-service-ca" pod "3266de62-2593-4496-b3e6-7600ebb710ce" (UID: "3266de62-2593-4496-b3e6-7600ebb710ce"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:29:47.986325 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:47.986285    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3266de62-2593-4496-b3e6-7600ebb710ce-metrics-certs" pod "3266de62-2593-4496-b3e6-7600ebb710ce" (UID: "3266de62-2593-4496-b3e6-7600ebb710ce"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:29:47.987021 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:47.986962    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3266de62-2593-4496-b3e6-7600ebb710ce-kube-api-access-2d9nr" pod "3266de62-2593-4496-b3e6-7600ebb710ce" (UID: "3266de62-2593-4496-b3e6-7600ebb710ce"). InnerVolumeSpecName "kube-api-access-2d9nr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:29:47.989513 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:47.989487    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3266de62-2593-4496-b3e6-7600ebb710ce-maas-api-tls" pod "3266de62-2593-4496-b3e6-7600ebb710ce" (UID: "3266de62-2593-4496-b3e6-7600ebb710ce"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:29:48.040739 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:48.040708    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-7bb1c2-6746d48b7f-8bqqt"
Oct 30 22:29:48.041212 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:48.040725    5510 generic.go:356] "Generic (PLEG): container finished" podID="3266de62-2593-4496-b3e6-7600ebb710ce" containerID="6410f2ac44e9b1c8e76dcf326d66cd9b1d2bcf9b7d51e52dc499695b93a5df4f" exitCode=0
Oct 30 22:29:48.041394 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:48.040747    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-7bb1c2-6746d48b7f-8bqqt" event={"ID":"3266de62-2593-4496-b3e6-7600ebb710ce","Type":"ContainerDied","Data":"6410f2ac44e9b1c8e76dcf326d66cd9b1d2bcf9b7d51e52dc499695b93a5df4f"}
Oct 30 22:29:48.041500 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:48.041488    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-7bb1c2-6746d48b7f-8bqqt" event={"ID":"3266de62-2593-4496-b3e6-7600ebb710ce","Type":"ContainerDied","Data":"a03783270fec980133b005b05c3f832b2ad717a95a20f4b2419b214b6c5f466e"}
Oct 30 22:29:48.041580 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:48.041571    5510 scope.go:122] "RemoveContainer" containerID="6410f2ac44e9b1c8e76dcf326d66cd9b1d2bcf9b7d51e52dc499695b93a5df4f"
Oct 30 22:29:48.056741 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:48.056713    5510 scope.go:122] "RemoveContainer" containerID="6410f2ac44e9b1c8e76dcf326d66cd9b1d2bcf9b7d51e52dc499695b93a5df4f"
Oct 30 22:29:48.057146 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:29:48.057117    5510 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6410f2ac44e9b1c8e76dcf326d66cd9b1d2bcf9b7d51e52dc499695b93a5df4f\": container with ID starting with 6410f2ac44e9b1c8e76dcf326d66cd9b1d2bcf9b7d51e52dc499695b93a5df4f not found: ID does not exist" containerID="6410f2ac44e9b1c8e76dcf326d66cd9b1d2bcf9b7d51e52dc499695b93a5df4f"
Oct 30 22:29:48.057274 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:48.057146    5510 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6410f2ac44e9b1c8e76dcf326d66cd9b1d2bcf9b7d51e52dc499695b93a5df4f"} err="failed to get container status \"6410f2ac44e9b1c8e76dcf326d66cd9b1d2bcf9b7d51e52dc499695b93a5df4f\": rpc error: code = NotFound desc = could not find container \"6410f2ac44e9b1c8e76dcf326d66cd9b1d2bcf9b7d51e52dc499695b93a5df4f\": container with ID starting with 6410f2ac44e9b1c8e76dcf326d66cd9b1d2bcf9b7d51e52dc499695b93a5df4f not found: ID does not exist"
Oct 30 22:29:48.068588 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:48.068560    5510 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3266de62-2593-4496-b3e6-7600ebb710ce-maas-api-tls\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:29:48.068731 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:48.068596    5510 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3266de62-2593-4496-b3e6-7600ebb710ce-openshift-service-ca\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:29:48.068731 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:48.068615    5510 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2d9nr\" (UniqueName: \"kubernetes.io/projected/3266de62-2593-4496-b3e6-7600ebb710ce-kube-api-access-2d9nr\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:29:48.068731 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:48.068630    5510 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3266de62-2593-4496-b3e6-7600ebb710ce-metrics-certs\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:29:48.198294 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:48.198215    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-7bb1c2-6746d48b7f-8bqqt"]
Oct 30 22:29:48.204908 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:48.204880    5510 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-7bb1c2-6746d48b7f-8bqqt"]
Oct 30 22:29:49.212330 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:49.212297    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:29:49.552708 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:49.552674    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:29:49.554089 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:49.554068    5510 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3266de62-2593-4496-b3e6-7600ebb710ce" path="/var/lib/kubelet/pods/3266de62-2593-4496-b3e6-7600ebb710ce/volumes"
Oct 30 22:29:49.766887 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:49.766846    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:29:50.251624 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:50.251587    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:29:54.027184 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:54.027147    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w5-99304b-6c6775f47f-bd9fp"]
Oct 30 22:29:54.125991 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:54.125953    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w5-99304b-7dc47b455-f2xqz"]
Oct 30 22:29:54.488040 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:54.486633    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:29:54.682102 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:54.682066    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:29:54.916667 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:54.916633    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:29:56.558170 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:56.558134    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-5jqnb"]
Oct 30 22:29:56.558699 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:56.558393    5510 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-5jqnb" podUID="eb812de4-a46c-41c5-adc7-7b325b1de62b" containerName="istio-proxy" containerID="cri-o://db66658c383f8cf461bd43d30d252998ed414cbeeb79e8126c36be3ed4a3ca86" gracePeriod=30
Oct 30 22:29:57.071036 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:57.071001    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:29:57.312008 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:57.311972    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:29:57.517046 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:57.515389    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:29:58.929943 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:58.929901    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:29:59.028397 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:59.028357    5510 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w5-99304b-6c6775f47f-bd9fp" podUID="1ddea0c3-4738-4353-a0df-726c86a635be" containerName="payload-processing" containerID="cri-o://e96ec942921f6989eeaf6b31328c91ddabd21c8aa58b21c623a9c003b7b9be2a" gracePeriod=25
Oct 30 22:29:59.117207 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:59.117173    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:29:59.127022 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:59.126970    5510 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w5-99304b-7dc47b455-f2xqz" podUID="9151b5e0-534d-4f10-8c6e-6a0f0a36a942" containerName="payload-pre-processing" containerID="cri-o://cd9742ea1b0ad2686ab3222f6d7164fa2af51430b113e3d311a92aa8dd3b3703" gracePeriod=25
Oct 30 22:29:59.151298 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:59.151273    5510 generic.go:356] "Generic (PLEG): container finished" podID="1ddea0c3-4738-4353-a0df-726c86a635be" containerID="e96ec942921f6989eeaf6b31328c91ddabd21c8aa58b21c623a9c003b7b9be2a" exitCode=0
Oct 30 22:29:59.151399 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:59.151344    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w5-99304b-6c6775f47f-bd9fp" event={"ID":"1ddea0c3-4738-4353-a0df-726c86a635be","Type":"ContainerDied","Data":"e96ec942921f6989eeaf6b31328c91ddabd21c8aa58b21c623a9c003b7b9be2a"}
Oct 30 22:29:59.317331 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:59.317294    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w5-99304b-6c6775f47f-bd9fp"
Oct 30 22:29:59.414238 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:59.414212    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1ddea0c3-4738-4353-a0df-726c86a635be-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1ddea0c3-4738-4353-a0df-726c86a635be-plugins-config-volume\") pod \"1ddea0c3-4738-4353-a0df-726c86a635be\" (UID: \"1ddea0c3-4738-4353-a0df-726c86a635be\") "
Oct 30 22:29:59.414335 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:59.414294    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1ddea0c3-4738-4353-a0df-726c86a635be-kube-api-access-lzmwn\" (UniqueName: \"kubernetes.io/projected/1ddea0c3-4738-4353-a0df-726c86a635be-kube-api-access-lzmwn\") pod \"1ddea0c3-4738-4353-a0df-726c86a635be\" (UID: \"1ddea0c3-4738-4353-a0df-726c86a635be\") "
Oct 30 22:29:59.414624 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:59.414602    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1ddea0c3-4738-4353-a0df-726c86a635be-plugins-config-volume" pod "1ddea0c3-4738-4353-a0df-726c86a635be" (UID: "1ddea0c3-4738-4353-a0df-726c86a635be"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:29:59.414853 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:59.414836    5510 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1ddea0c3-4738-4353-a0df-726c86a635be-plugins-config-volume\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:29:59.416201 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:59.416179    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1ddea0c3-4738-4353-a0df-726c86a635be-kube-api-access-lzmwn" pod "1ddea0c3-4738-4353-a0df-726c86a635be" (UID: "1ddea0c3-4738-4353-a0df-726c86a635be"). InnerVolumeSpecName "kube-api-access-lzmwn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:29:59.437559 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:59.437528    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w5-99304b-7dc47b455-f2xqz"
Oct 30 22:29:59.515535 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:59.515447    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9151b5e0-534d-4f10-8c6e-6a0f0a36a942-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9151b5e0-534d-4f10-8c6e-6a0f0a36a942-plugins-config-volume\") pod \"9151b5e0-534d-4f10-8c6e-6a0f0a36a942\" (UID: \"9151b5e0-534d-4f10-8c6e-6a0f0a36a942\") "
Oct 30 22:29:59.515778 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:59.515762    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9151b5e0-534d-4f10-8c6e-6a0f0a36a942-kube-api-access-svttq\" (UniqueName: \"kubernetes.io/projected/9151b5e0-534d-4f10-8c6e-6a0f0a36a942-kube-api-access-svttq\") pod \"9151b5e0-534d-4f10-8c6e-6a0f0a36a942\" (UID: \"9151b5e0-534d-4f10-8c6e-6a0f0a36a942\") "
Oct 30 22:29:59.515870 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:59.515846    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9151b5e0-534d-4f10-8c6e-6a0f0a36a942-plugins-config-volume" pod "9151b5e0-534d-4f10-8c6e-6a0f0a36a942" (UID: "9151b5e0-534d-4f10-8c6e-6a0f0a36a942"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:29:59.516123 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:59.516108    5510 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9151b5e0-534d-4f10-8c6e-6a0f0a36a942-plugins-config-volume\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:29:59.516123 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:59.516123    5510 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lzmwn\" (UniqueName: \"kubernetes.io/projected/1ddea0c3-4738-4353-a0df-726c86a635be-kube-api-access-lzmwn\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:29:59.517842 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:59.517821    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9151b5e0-534d-4f10-8c6e-6a0f0a36a942-kube-api-access-svttq" pod "9151b5e0-534d-4f10-8c6e-6a0f0a36a942" (UID: "9151b5e0-534d-4f10-8c6e-6a0f0a36a942"). InnerVolumeSpecName "kube-api-access-svttq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:29:59.618092 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:59.618058    5510 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-svttq\" (UniqueName: \"kubernetes.io/projected/9151b5e0-534d-4f10-8c6e-6a0f0a36a942-kube-api-access-svttq\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:29:59.943687 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:29:59.941422    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:30:00.160532 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:00.160504    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w5-99304b-6c6775f47f-bd9fp"
Oct 30 22:30:00.160743 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:00.160535    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w5-99304b-6c6775f47f-bd9fp" event={"ID":"1ddea0c3-4738-4353-a0df-726c86a635be","Type":"ContainerDied","Data":"72b3a05304c1094348366ce2368de8a149333f30525bf02e50ce7fad035490fb"}
Oct 30 22:30:00.160743 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:00.160595    5510 scope.go:122] "RemoveContainer" containerID="e96ec942921f6989eeaf6b31328c91ddabd21c8aa58b21c623a9c003b7b9be2a"
Oct 30 22:30:00.162222 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:00.162198    5510 generic.go:356] "Generic (PLEG): container finished" podID="9151b5e0-534d-4f10-8c6e-6a0f0a36a942" containerID="cd9742ea1b0ad2686ab3222f6d7164fa2af51430b113e3d311a92aa8dd3b3703" exitCode=0
Oct 30 22:30:00.162336 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:00.162246    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w5-99304b-7dc47b455-f2xqz"
Oct 30 22:30:00.162336 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:00.162314    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w5-99304b-7dc47b455-f2xqz" event={"ID":"9151b5e0-534d-4f10-8c6e-6a0f0a36a942","Type":"ContainerDied","Data":"cd9742ea1b0ad2686ab3222f6d7164fa2af51430b113e3d311a92aa8dd3b3703"}
Oct 30 22:30:00.162446 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:00.162356    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w5-99304b-7dc47b455-f2xqz" event={"ID":"9151b5e0-534d-4f10-8c6e-6a0f0a36a942","Type":"ContainerDied","Data":"a1960d31da11f4285a446406a508f28034361614cd2664950cb9c18df29394e3"}
Oct 30 22:30:00.171980 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:00.171959    5510 scope.go:122] "RemoveContainer" containerID="cd9742ea1b0ad2686ab3222f6d7164fa2af51430b113e3d311a92aa8dd3b3703"
Oct 30 22:30:00.181448 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:00.181427    5510 scope.go:122] "RemoveContainer" containerID="cd9742ea1b0ad2686ab3222f6d7164fa2af51430b113e3d311a92aa8dd3b3703"
Oct 30 22:30:00.181767 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:30:00.181745    5510 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cd9742ea1b0ad2686ab3222f6d7164fa2af51430b113e3d311a92aa8dd3b3703\": container with ID starting with cd9742ea1b0ad2686ab3222f6d7164fa2af51430b113e3d311a92aa8dd3b3703 not found: ID does not exist" containerID="cd9742ea1b0ad2686ab3222f6d7164fa2af51430b113e3d311a92aa8dd3b3703"
Oct 30 22:30:00.181882 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:00.181775    5510 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cd9742ea1b0ad2686ab3222f6d7164fa2af51430b113e3d311a92aa8dd3b3703"} err="failed to get container status \"cd9742ea1b0ad2686ab3222f6d7164fa2af51430b113e3d311a92aa8dd3b3703\": rpc error: code = NotFound desc = could not find container \"cd9742ea1b0ad2686ab3222f6d7164fa2af51430b113e3d311a92aa8dd3b3703\": container with ID starting with cd9742ea1b0ad2686ab3222f6d7164fa2af51430b113e3d311a92aa8dd3b3703 not found: ID does not exist"
Oct 30 22:30:00.264409 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:00.264333    5510 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29889990-27cbr"]
Oct 30 22:30:00.265014 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:00.265001    5510 cpu_manager.go:414] "containerMap: removing container" podUID="4d203fdf-00d0-4d25-8a63-c2f355d8d03f" containerName="payload-pre-processing"
Oct 30 22:30:00.265075 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:00.265016    5510 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4d203fdf-00d0-4d25-8a63-c2f355d8d03f" containerName="payload-pre-processing"
Oct 30 22:30:00.265075 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:00.265028    5510 cpu_manager.go:414] "containerMap: removing container" podUID="0796eecd-5044-4c1b-981b-14e8a16c4a9c" containerName="payload-pre-processing"
Oct 30 22:30:00.265075 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:00.265034    5510 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0796eecd-5044-4c1b-981b-14e8a16c4a9c" containerName="payload-pre-processing"
Oct 30 22:30:00.265075 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:00.265045    5510 cpu_manager.go:414] "containerMap: removing container" podUID="7f89f8ac-ad11-4aac-9f27-95d05190fd97" containerName="payload-processing"
Oct 30 22:30:00.265075 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:00.265051    5510 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7f89f8ac-ad11-4aac-9f27-95d05190fd97" containerName="payload-processing"
Oct 30 22:30:00.265075 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:00.265058    5510 cpu_manager.go:414] "containerMap: removing container" podUID="4009806a-5d15-4224-9b63-e3bc115f35ad" containerName="payload-pre-processing"
Oct 30 22:30:00.265075 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:00.265064    5510 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4009806a-5d15-4224-9b63-e3bc115f35ad" containerName="payload-pre-processing"
Oct 30 22:30:00.265075 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:00.265071    5510 cpu_manager.go:414] "containerMap: removing container" podUID="13760c22-8ba0-45ed-b2e4-4e23d5273901" containerName="istio-proxy"
Oct 30 22:30:00.265075 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:00.265076    5510 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="13760c22-8ba0-45ed-b2e4-4e23d5273901" containerName="istio-proxy"
Oct 30 22:30:00.265347 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:00.265086    5510 cpu_manager.go:414] "containerMap: removing container" podUID="2c98b4d1-9cc0-4482-af1e-1038eaeb1e61" containerName="payload-processing"
Oct 30 22:30:00.265347 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:00.265090    5510 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2c98b4d1-9cc0-4482-af1e-1038eaeb1e61" containerName="payload-processing"
Oct 30 22:30:00.265347 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:00.265106    5510 cpu_manager.go:414] "containerMap: removing container" podUID="1ddea0c3-4738-4353-a0df-726c86a635be" containerName="payload-processing"
Oct 30 22:30:00.265347 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:00.265110    5510 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1ddea0c3-4738-4353-a0df-726c86a635be" containerName="payload-processing"
Oct 30 22:30:00.265347 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:00.265120    5510 cpu_manager.go:414] "containerMap: removing container" podUID="8023fe04-37eb-4458-bfee-ef8170d34ab6" containerName="payload-pre-processing"
Oct 30 22:30:00.265347 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:00.265124    5510 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8023fe04-37eb-4458-bfee-ef8170d34ab6" containerName="payload-pre-processing"
Oct 30 22:30:00.265347 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:00.265132    5510 cpu_manager.go:414] "containerMap: removing container" podUID="f7fdf681-44d0-45d0-b2bd-c8e4df1af3bd" containerName="maas-api"
Oct 30 22:30:00.265347 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:00.265137    5510 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f7fdf681-44d0-45d0-b2bd-c8e4df1af3bd" containerName="maas-api"
Oct 30 22:30:00.265347 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:00.265145    5510 cpu_manager.go:414] "containerMap: removing container" podUID="b112302e-a8df-4573-a151-696453d1e7ee" containerName="payload-pre-processing"
Oct 30 22:30:00.265347 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:00.265150    5510 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b112302e-a8df-4573-a151-696453d1e7ee" containerName="payload-pre-processing"
Oct 30 22:30:00.265347 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:00.265158    5510 cpu_manager.go:414] "containerMap: removing container" podUID="d5825ebb-76c4-4a3a-aa17-2bbedab22772" containerName="payload-pre-processing"
Oct 30 22:30:00.265347 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:00.265162    5510 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d5825ebb-76c4-4a3a-aa17-2bbedab22772" containerName="payload-pre-processing"
Oct 30 22:30:00.265347 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:00.265176    5510 cpu_manager.go:414] "containerMap: removing container" podUID="3266de62-2593-4496-b3e6-7600ebb710ce" containerName="maas-api"
Oct 30 22:30:00.265347 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:00.265180    5510 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3266de62-2593-4496-b3e6-7600ebb710ce" containerName="maas-api"
Oct 30 22:30:00.265347 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:00.265187    5510 cpu_manager.go:414] "containerMap: removing container" podUID="7bdcd87e-ab76-4af6-ae7b-37e1384b6406" containerName="payload-processing"
Oct 30 22:30:00.265347 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:00.265191    5510 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7bdcd87e-ab76-4af6-ae7b-37e1384b6406" containerName="payload-processing"
Oct 30 22:30:00.265347 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:00.265197    5510 cpu_manager.go:414] "containerMap: removing container" podUID="58572a25-fdb3-417c-9ade-f20e2d714b78" containerName="maas-api"
Oct 30 22:30:00.265347 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:00.265201    5510 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="58572a25-fdb3-417c-9ade-f20e2d714b78" containerName="maas-api"
Oct 30 22:30:00.265347 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:00.265213    5510 cpu_manager.go:414] "containerMap: removing container" podUID="446aae1e-35a8-49da-9780-db1c369d7efd" containerName="payload-pre-processing"
Oct 30 22:30:00.265347 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:00.265217    5510 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="446aae1e-35a8-49da-9780-db1c369d7efd" containerName="payload-pre-processing"
Oct 30 22:30:00.265347 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:00.265223    5510 cpu_manager.go:414] "containerMap: removing container" podUID="9151b5e0-534d-4f10-8c6e-6a0f0a36a942" containerName="payload-pre-processing"
Oct 30 22:30:00.265347 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:00.265228    5510 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9151b5e0-534d-4f10-8c6e-6a0f0a36a942" containerName="payload-pre-processing"
Oct 30 22:30:00.265347 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:00.265234    5510 cpu_manager.go:414] "containerMap: removing container" podUID="ecfca1f1-d4b8-421b-ac73-3b726a8afdf4" containerName="payload-processing"
Oct 30 22:30:00.265347 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:00.265238    5510 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ecfca1f1-d4b8-421b-ac73-3b726a8afdf4" containerName="payload-processing"
Oct 30 22:30:00.265347 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:00.265250    5510 cpu_manager.go:414] "containerMap: removing container" podUID="4e4794d9-9186-4127-b515-3e89b9882981" containerName="payload-processing"
Oct 30 22:30:00.265347 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:00.265255    5510 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4e4794d9-9186-4127-b515-3e89b9882981" containerName="payload-processing"
Oct 30 22:30:00.265347 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:00.265265    5510 cpu_manager.go:414] "containerMap: removing container" podUID="1916e87c-a1a4-4967-8640-8bbcd9d12903" containerName="main"
Oct 30 22:30:00.265347 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:00.265270    5510 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1916e87c-a1a4-4967-8640-8bbcd9d12903" containerName="main"
Oct 30 22:30:00.265347 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:00.265277    5510 cpu_manager.go:414] "containerMap: removing container" podUID="13760c22-8ba0-45ed-b2e4-4e23d5273901" containerName="istio-proxy"
Oct 30 22:30:00.265347 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:00.265281    5510 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="13760c22-8ba0-45ed-b2e4-4e23d5273901" containerName="istio-proxy"
Oct 30 22:30:00.265347 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:00.265359    5510 memory_manager.go:365] "RemoveStaleState removing state" podUID="4009806a-5d15-4224-9b63-e3bc115f35ad" containerName="payload-pre-processing"
Oct 30 22:30:00.266259 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:00.265368    5510 memory_manager.go:365] "RemoveStaleState removing state" podUID="13760c22-8ba0-45ed-b2e4-4e23d5273901" containerName="istio-proxy"
Oct 30 22:30:00.266259 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:00.265377    5510 memory_manager.go:365] "RemoveStaleState removing state" podUID="2c98b4d1-9cc0-4482-af1e-1038eaeb1e61" containerName="payload-processing"
Oct 30 22:30:00.266259 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:00.265385    5510 memory_manager.go:365] "RemoveStaleState removing state" podUID="1ddea0c3-4738-4353-a0df-726c86a635be" containerName="payload-processing"
Oct 30 22:30:00.266259 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:00.265392    5510 memory_manager.go:365] "RemoveStaleState removing state" podUID="8023fe04-37eb-4458-bfee-ef8170d34ab6" containerName="payload-pre-processing"
Oct 30 22:30:00.266259 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:00.265399    5510 memory_manager.go:365] "RemoveStaleState removing state" podUID="f7fdf681-44d0-45d0-b2bd-c8e4df1af3bd" containerName="maas-api"
Oct 30 22:30:00.266259 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:00.265406    5510 memory_manager.go:365] "RemoveStaleState removing state" podUID="b112302e-a8df-4573-a151-696453d1e7ee" containerName="payload-pre-processing"
Oct 30 22:30:00.266259 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:00.265411    5510 memory_manager.go:365] "RemoveStaleState removing state" podUID="d5825ebb-76c4-4a3a-aa17-2bbedab22772" containerName="payload-pre-processing"
Oct 30 22:30:00.266259 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:00.265418    5510 memory_manager.go:365] "RemoveStaleState removing state" podUID="3266de62-2593-4496-b3e6-7600ebb710ce" containerName="maas-api"
Oct 30 22:30:00.266259 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:00.265434    5510 memory_manager.go:365] "RemoveStaleState removing state" podUID="7bdcd87e-ab76-4af6-ae7b-37e1384b6406" containerName="payload-processing"
Oct 30 22:30:00.266259 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:00.265439    5510 memory_manager.go:365] "RemoveStaleState removing state" podUID="58572a25-fdb3-417c-9ade-f20e2d714b78" containerName="maas-api"
Oct 30 22:30:00.266259 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:00.265447    5510 memory_manager.go:365] "RemoveStaleState removing state" podUID="446aae1e-35a8-49da-9780-db1c369d7efd" containerName="payload-pre-processing"
Oct 30 22:30:00.266259 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:00.265452    5510 memory_manager.go:365] "RemoveStaleState removing state" podUID="9151b5e0-534d-4f10-8c6e-6a0f0a36a942" containerName="payload-pre-processing"
Oct 30 22:30:00.266259 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:00.265459    5510 memory_manager.go:365] "RemoveStaleState removing state" podUID="ecfca1f1-d4b8-421b-ac73-3b726a8afdf4" containerName="payload-processing"
Oct 30 22:30:00.266259 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:00.265467    5510 memory_manager.go:365] "RemoveStaleState removing state" podUID="4e4794d9-9186-4127-b515-3e89b9882981" containerName="payload-processing"
Oct 30 22:30:00.266259 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:00.265472    5510 memory_manager.go:365] "RemoveStaleState removing state" podUID="1916e87c-a1a4-4967-8640-8bbcd9d12903" containerName="main"
Oct 30 22:30:00.266259 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:00.265480    5510 memory_manager.go:365] "RemoveStaleState removing state" podUID="4d203fdf-00d0-4d25-8a63-c2f355d8d03f" containerName="payload-pre-processing"
Oct 30 22:30:00.266259 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:00.265484    5510 memory_manager.go:365] "RemoveStaleState removing state" podUID="0796eecd-5044-4c1b-981b-14e8a16c4a9c" containerName="payload-pre-processing"
Oct 30 22:30:00.266259 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:00.265490    5510 memory_manager.go:365] "RemoveStaleState removing state" podUID="7f89f8ac-ad11-4aac-9f27-95d05190fd97" containerName="payload-processing"
Oct 30 22:30:00.270373 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:00.270349    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889990-27cbr"
Oct 30 22:30:00.327006 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:00.326969    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29889990-27cbr"]
Oct 30 22:30:00.332577 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:00.332540    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/abac65c3-1048-47a8-b307-c2e002e6f7d5-maas-api-service-ca\") pod \"maas-api-key-cleanup-29889990-27cbr\" (UID: \"abac65c3-1048-47a8-b307-c2e002e6f7d5\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889990-27cbr"
Oct 30 22:30:00.332810 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:00.332727    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-27tjp\" (UniqueName: \"kubernetes.io/projected/abac65c3-1048-47a8-b307-c2e002e6f7d5-kube-api-access-27tjp\") pod \"maas-api-key-cleanup-29889990-27cbr\" (UID: \"abac65c3-1048-47a8-b307-c2e002e6f7d5\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889990-27cbr"
Oct 30 22:30:00.348266 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:00.348114    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w5-99304b-6c6775f47f-bd9fp"]
Oct 30 22:30:00.355468 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:00.355438    5510 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w5-99304b-6c6775f47f-bd9fp"]
Oct 30 22:30:00.396076 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:00.396046    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5rvcj\""
Oct 30 22:30:00.442610 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:00.442572    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-27tjp\" (UniqueName: \"kubernetes.io/projected/abac65c3-1048-47a8-b307-c2e002e6f7d5-kube-api-access-27tjp\") pod \"maas-api-key-cleanup-29889990-27cbr\" (UID: \"abac65c3-1048-47a8-b307-c2e002e6f7d5\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889990-27cbr"
Oct 30 22:30:00.442855 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:00.442779    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/abac65c3-1048-47a8-b307-c2e002e6f7d5-maas-api-service-ca\") pod \"maas-api-key-cleanup-29889990-27cbr\" (UID: \"abac65c3-1048-47a8-b307-c2e002e6f7d5\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889990-27cbr"
Oct 30 22:30:00.443378 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:00.443357    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/abac65c3-1048-47a8-b307-c2e002e6f7d5-maas-api-service-ca\") pod \"maas-api-key-cleanup-29889990-27cbr\" (UID: \"abac65c3-1048-47a8-b307-c2e002e6f7d5\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889990-27cbr"
Oct 30 22:30:00.458039 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:00.458012    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w5-99304b-7dc47b455-f2xqz"]
Oct 30 22:30:00.465855 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:00.465814    5510 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w5-99304b-7dc47b455-f2xqz"]
Oct 30 22:30:00.501229 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:00.501198    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-27tjp\" (UniqueName: \"kubernetes.io/projected/abac65c3-1048-47a8-b307-c2e002e6f7d5-kube-api-access-27tjp\") pod \"maas-api-key-cleanup-29889990-27cbr\" (UID: \"abac65c3-1048-47a8-b307-c2e002e6f7d5\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889990-27cbr"
Oct 30 22:30:00.594883 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:00.594847    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889990-27cbr"
Oct 30 22:30:00.969370 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:00.969340    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29889990-27cbr"]
Oct 30 22:30:00.970813 ip-10-0-136-210 kubenswrapper[5510]: W1030 22:30:00.970790    5510 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podabac65c3_1048_47a8_b307_c2e002e6f7d5.slice/crio-03b79c4e57b83437667d6c546fb9b103be05be0365856d4c1cc1c7608618ee0a WatchSource:0}: Error finding container 03b79c4e57b83437667d6c546fb9b103be05be0365856d4c1cc1c7608618ee0a: Status 404 returned error can't find the container with id 03b79c4e57b83437667d6c546fb9b103be05be0365856d4c1cc1c7608618ee0a
Oct 30 22:30:01.170094 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:01.170058    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889990-27cbr" event={"ID":"abac65c3-1048-47a8-b307-c2e002e6f7d5","Type":"ContainerStarted","Data":"03b79c4e57b83437667d6c546fb9b103be05be0365856d4c1cc1c7608618ee0a"}
Oct 30 22:30:01.556547 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:01.556508    5510 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1ddea0c3-4738-4353-a0df-726c86a635be" path="/var/lib/kubelet/pods/1ddea0c3-4738-4353-a0df-726c86a635be/volumes"
Oct 30 22:30:01.557024 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:01.557008    5510 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9151b5e0-534d-4f10-8c6e-6a0f0a36a942" path="/var/lib/kubelet/pods/9151b5e0-534d-4f10-8c6e-6a0f0a36a942/volumes"
Oct 30 22:30:01.850383 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:01.850356    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-5jqnb"
Oct 30 22:30:01.962566 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:01.962529    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/eb812de4-a46c-41c5-adc7-7b325b1de62b-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eb812de4-a46c-41c5-adc7-7b325b1de62b-workload-certs\") pod \"eb812de4-a46c-41c5-adc7-7b325b1de62b\" (UID: \"eb812de4-a46c-41c5-adc7-7b325b1de62b\") "
Oct 30 22:30:01.962566 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:01.962565    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/eb812de4-a46c-41c5-adc7-7b325b1de62b-istio-token\" (UniqueName: \"kubernetes.io/projected/eb812de4-a46c-41c5-adc7-7b325b1de62b-istio-token\") pod \"eb812de4-a46c-41c5-adc7-7b325b1de62b\" (UID: \"eb812de4-a46c-41c5-adc7-7b325b1de62b\") "
Oct 30 22:30:01.962831 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:01.962607    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/eb812de4-a46c-41c5-adc7-7b325b1de62b-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eb812de4-a46c-41c5-adc7-7b325b1de62b-istiod-ca-cert\") pod \"eb812de4-a46c-41c5-adc7-7b325b1de62b\" (UID: \"eb812de4-a46c-41c5-adc7-7b325b1de62b\") "
Oct 30 22:30:01.962831 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:01.962675    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/eb812de4-a46c-41c5-adc7-7b325b1de62b-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eb812de4-a46c-41c5-adc7-7b325b1de62b-istio-data\") pod \"eb812de4-a46c-41c5-adc7-7b325b1de62b\" (UID: \"eb812de4-a46c-41c5-adc7-7b325b1de62b\") "
Oct 30 22:30:01.962831 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:01.962728    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/eb812de4-a46c-41c5-adc7-7b325b1de62b-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb812de4-a46c-41c5-adc7-7b325b1de62b-credential-socket\") pod \"eb812de4-a46c-41c5-adc7-7b325b1de62b\" (UID: \"eb812de4-a46c-41c5-adc7-7b325b1de62b\") "
Oct 30 22:30:01.962831 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:01.962800    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/eb812de4-a46c-41c5-adc7-7b325b1de62b-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eb812de4-a46c-41c5-adc7-7b325b1de62b-istio-podinfo\") pod \"eb812de4-a46c-41c5-adc7-7b325b1de62b\" (UID: \"eb812de4-a46c-41c5-adc7-7b325b1de62b\") "
Oct 30 22:30:01.963042 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:01.962832    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/eb812de4-a46c-41c5-adc7-7b325b1de62b-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eb812de4-a46c-41c5-adc7-7b325b1de62b-istio-envoy\") pod \"eb812de4-a46c-41c5-adc7-7b325b1de62b\" (UID: \"eb812de4-a46c-41c5-adc7-7b325b1de62b\") "
Oct 30 22:30:01.963042 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:01.962883    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/eb812de4-a46c-41c5-adc7-7b325b1de62b-kube-api-access-9m88b\" (UniqueName: \"kubernetes.io/projected/eb812de4-a46c-41c5-adc7-7b325b1de62b-kube-api-access-9m88b\") pod \"eb812de4-a46c-41c5-adc7-7b325b1de62b\" (UID: \"eb812de4-a46c-41c5-adc7-7b325b1de62b\") "
Oct 30 22:30:01.963042 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:01.962905    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb812de4-a46c-41c5-adc7-7b325b1de62b-workload-certs" pod "eb812de4-a46c-41c5-adc7-7b325b1de62b" (UID: "eb812de4-a46c-41c5-adc7-7b325b1de62b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:30:01.963042 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:01.963008    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb812de4-a46c-41c5-adc7-7b325b1de62b-istio-data" pod "eb812de4-a46c-41c5-adc7-7b325b1de62b" (UID: "eb812de4-a46c-41c5-adc7-7b325b1de62b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:30:01.963252 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:01.963111    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eb812de4-a46c-41c5-adc7-7b325b1de62b-istiod-ca-cert" pod "eb812de4-a46c-41c5-adc7-7b325b1de62b" (UID: "eb812de4-a46c-41c5-adc7-7b325b1de62b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:30:01.963252 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:01.963115    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/eb812de4-a46c-41c5-adc7-7b325b1de62b-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb812de4-a46c-41c5-adc7-7b325b1de62b-workload-socket\") pod \"eb812de4-a46c-41c5-adc7-7b325b1de62b\" (UID: \"eb812de4-a46c-41c5-adc7-7b325b1de62b\") "
Oct 30 22:30:01.963366 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:01.963308    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb812de4-a46c-41c5-adc7-7b325b1de62b-workload-socket" pod "eb812de4-a46c-41c5-adc7-7b325b1de62b" (UID: "eb812de4-a46c-41c5-adc7-7b325b1de62b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:30:01.963465 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:01.963439    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb812de4-a46c-41c5-adc7-7b325b1de62b-credential-socket" pod "eb812de4-a46c-41c5-adc7-7b325b1de62b" (UID: "eb812de4-a46c-41c5-adc7-7b325b1de62b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:30:01.963864 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:01.963842    5510 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eb812de4-a46c-41c5-adc7-7b325b1de62b-workload-certs\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:30:01.963947 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:01.963866    5510 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eb812de4-a46c-41c5-adc7-7b325b1de62b-istiod-ca-cert\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:30:01.963947 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:01.963879    5510 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eb812de4-a46c-41c5-adc7-7b325b1de62b-istio-data\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:30:01.963947 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:01.963892    5510 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb812de4-a46c-41c5-adc7-7b325b1de62b-credential-socket\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:30:01.963947 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:01.963906    5510 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb812de4-a46c-41c5-adc7-7b325b1de62b-workload-socket\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:30:01.965423 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:01.965401    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb812de4-a46c-41c5-adc7-7b325b1de62b-istio-envoy" pod "eb812de4-a46c-41c5-adc7-7b325b1de62b" (UID: "eb812de4-a46c-41c5-adc7-7b325b1de62b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:30:01.965474 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:01.965434    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eb812de4-a46c-41c5-adc7-7b325b1de62b-istio-token" pod "eb812de4-a46c-41c5-adc7-7b325b1de62b" (UID: "eb812de4-a46c-41c5-adc7-7b325b1de62b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:30:01.965710 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:01.965686    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/eb812de4-a46c-41c5-adc7-7b325b1de62b-istio-podinfo" pod "eb812de4-a46c-41c5-adc7-7b325b1de62b" (UID: "eb812de4-a46c-41c5-adc7-7b325b1de62b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:30:01.965764 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:01.965704    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eb812de4-a46c-41c5-adc7-7b325b1de62b-kube-api-access-9m88b" pod "eb812de4-a46c-41c5-adc7-7b325b1de62b" (UID: "eb812de4-a46c-41c5-adc7-7b325b1de62b"). InnerVolumeSpecName "kube-api-access-9m88b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:30:02.065587 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:02.065550    5510 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eb812de4-a46c-41c5-adc7-7b325b1de62b-istio-token\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:30:02.065587 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:02.065582    5510 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eb812de4-a46c-41c5-adc7-7b325b1de62b-istio-podinfo\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:30:02.066053 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:02.065622    5510 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eb812de4-a46c-41c5-adc7-7b325b1de62b-istio-envoy\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:30:02.066053 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:02.065634    5510 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9m88b\" (UniqueName: \"kubernetes.io/projected/eb812de4-a46c-41c5-adc7-7b325b1de62b-kube-api-access-9m88b\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:30:02.180070 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:02.179983    5510 generic.go:356] "Generic (PLEG): container finished" podID="eb812de4-a46c-41c5-adc7-7b325b1de62b" containerID="db66658c383f8cf461bd43d30d252998ed414cbeeb79e8126c36be3ed4a3ca86" exitCode=0
Oct 30 22:30:02.180070 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:02.180035    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-5jqnb"
Oct 30 22:30:02.180313 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:02.180070    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-5jqnb" event={"ID":"eb812de4-a46c-41c5-adc7-7b325b1de62b","Type":"ContainerDied","Data":"db66658c383f8cf461bd43d30d252998ed414cbeeb79e8126c36be3ed4a3ca86"}
Oct 30 22:30:02.180313 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:02.180136    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-5jqnb" event={"ID":"eb812de4-a46c-41c5-adc7-7b325b1de62b","Type":"ContainerDied","Data":"f8259c87f9dc4c6fb17d7b4a38fd1787ffb5236baacde2ece9442d4e0882b0f5"}
Oct 30 22:30:02.180313 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:02.180166    5510 scope.go:122] "RemoveContainer" containerID="db66658c383f8cf461bd43d30d252998ed414cbeeb79e8126c36be3ed4a3ca86"
Oct 30 22:30:02.191293 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:02.191273    5510 scope.go:122] "RemoveContainer" containerID="db66658c383f8cf461bd43d30d252998ed414cbeeb79e8126c36be3ed4a3ca86"
Oct 30 22:30:02.191620 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:30:02.191600    5510 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"db66658c383f8cf461bd43d30d252998ed414cbeeb79e8126c36be3ed4a3ca86\": container with ID starting with db66658c383f8cf461bd43d30d252998ed414cbeeb79e8126c36be3ed4a3ca86 not found: ID does not exist" containerID="db66658c383f8cf461bd43d30d252998ed414cbeeb79e8126c36be3ed4a3ca86"
Oct 30 22:30:02.191700 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:02.191631    5510 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"db66658c383f8cf461bd43d30d252998ed414cbeeb79e8126c36be3ed4a3ca86"} err="failed to get container status \"db66658c383f8cf461bd43d30d252998ed414cbeeb79e8126c36be3ed4a3ca86\": rpc error: code = NotFound desc = could not find container \"db66658c383f8cf461bd43d30d252998ed414cbeeb79e8126c36be3ed4a3ca86\": container with ID starting with db66658c383f8cf461bd43d30d252998ed414cbeeb79e8126c36be3ed4a3ca86 not found: ID does not exist"
Oct 30 22:30:02.338183 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:02.338140    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-5jqnb"]
Oct 30 22:30:02.343765 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:02.343733    5510 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-7bb1c2-openshift-default-5dffb59bcc-5jqnb"]
Oct 30 22:30:03.555844 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:03.555799    5510 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="eb812de4-a46c-41c5-adc7-7b325b1de62b" path="/var/lib/kubelet/pods/eb812de4-a46c-41c5-adc7-7b325b1de62b/volumes"
Oct 30 22:30:03.878811 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:03.878728    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:30:04.524317 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:04.521969    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:30:06.213297 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:06.213201    5510 generic.go:356] "Generic (PLEG): container finished" podID="abac65c3-1048-47a8-b307-c2e002e6f7d5" containerID="5559aec20dacd05eefb507887fa84571ee1792000b1d39bc990d4ab03aaead7e" exitCode=60
Oct 30 22:30:06.213297 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:06.213282    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889990-27cbr" event={"ID":"abac65c3-1048-47a8-b307-c2e002e6f7d5","Type":"ContainerDied","Data":"5559aec20dacd05eefb507887fa84571ee1792000b1d39bc990d4ab03aaead7e"}
Oct 30 22:30:06.213818 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:06.213621    5510 scope.go:122] "RemoveContainer" containerID="5559aec20dacd05eefb507887fa84571ee1792000b1d39bc990d4ab03aaead7e"
Oct 30 22:30:07.222739 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:07.222684    5510 generic.go:356] "Generic (PLEG): container finished" podID="abac65c3-1048-47a8-b307-c2e002e6f7d5" containerID="d4eddbf72059bcb5da1bccb5e0524779180897ab9ec4d1b0d6ecb06521b60860" exitCode=60
Oct 30 22:30:07.223211 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:07.222757    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889990-27cbr" event={"ID":"abac65c3-1048-47a8-b307-c2e002e6f7d5","Type":"ContainerDied","Data":"d4eddbf72059bcb5da1bccb5e0524779180897ab9ec4d1b0d6ecb06521b60860"}
Oct 30 22:30:07.223211 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:07.222820    5510 scope.go:122] "RemoveContainer" containerID="5559aec20dacd05eefb507887fa84571ee1792000b1d39bc990d4ab03aaead7e"
Oct 30 22:30:07.223211 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:07.223199    5510 scope.go:122] "RemoveContainer" containerID="d4eddbf72059bcb5da1bccb5e0524779180897ab9ec4d1b0d6ecb06521b60860"
Oct 30 22:30:07.223570 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:30:07.223552    5510 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-29889990-27cbr_odh-ai-gateway-infra(abac65c3-1048-47a8-b307-c2e002e6f7d5)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889990-27cbr" podUID="abac65c3-1048-47a8-b307-c2e002e6f7d5"
Oct 30 22:30:08.230992 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:08.230963    5510 scope.go:122] "RemoveContainer" containerID="d4eddbf72059bcb5da1bccb5e0524779180897ab9ec4d1b0d6ecb06521b60860"
Oct 30 22:30:08.231386 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:30:08.231302    5510 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-29889990-27cbr_odh-ai-gateway-infra(abac65c3-1048-47a8-b307-c2e002e6f7d5)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889990-27cbr" podUID="abac65c3-1048-47a8-b307-c2e002e6f7d5"
Oct 30 22:30:08.420693 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:30:08.420631    5510 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6gjbf" containerName="extractor"
Oct 30 22:30:09.018682 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:09.009474    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:30:09.204810 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:09.204778    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:30:13.821851 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:13.821809    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:30:17.861687 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:17.861636    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-9f3ea1-65cfd5854b-lcwjw"]
Oct 30 22:30:17.862170 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:17.861947    5510 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-9f3ea1-65cfd5854b-lcwjw" podUID="743f9a1e-c695-454d-823d-7939bfbb6aff" containerName="maas-api" containerID="cri-o://9d9ffef4ecd0bda9ca49728eb104e67aef8a77a96c6c2a27a7b7a50ced01ea84" gracePeriod=30
Oct 30 22:30:18.010494 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:18.010425    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:30:18.205297 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:18.205266    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-9f3ea1-65cfd5854b-lcwjw"
Oct 30 22:30:18.229463 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:18.229432    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/743f9a1e-c695-454d-823d-7939bfbb6aff-metrics-certs\" (UniqueName: \"kubernetes.io/secret/743f9a1e-c695-454d-823d-7939bfbb6aff-metrics-certs\") pod \"743f9a1e-c695-454d-823d-7939bfbb6aff\" (UID: \"743f9a1e-c695-454d-823d-7939bfbb6aff\") "
Oct 30 22:30:18.229612 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:18.229470    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/743f9a1e-c695-454d-823d-7939bfbb6aff-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/743f9a1e-c695-454d-823d-7939bfbb6aff-maas-api-tls\") pod \"743f9a1e-c695-454d-823d-7939bfbb6aff\" (UID: \"743f9a1e-c695-454d-823d-7939bfbb6aff\") "
Oct 30 22:30:18.229612 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:18.229519    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/743f9a1e-c695-454d-823d-7939bfbb6aff-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/743f9a1e-c695-454d-823d-7939bfbb6aff-openshift-service-ca\") pod \"743f9a1e-c695-454d-823d-7939bfbb6aff\" (UID: \"743f9a1e-c695-454d-823d-7939bfbb6aff\") "
Oct 30 22:30:18.229714 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:18.229697    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/743f9a1e-c695-454d-823d-7939bfbb6aff-kube-api-access-v2gzl\" (UniqueName: \"kubernetes.io/projected/743f9a1e-c695-454d-823d-7939bfbb6aff-kube-api-access-v2gzl\") pod \"743f9a1e-c695-454d-823d-7939bfbb6aff\" (UID: \"743f9a1e-c695-454d-823d-7939bfbb6aff\") "
Oct 30 22:30:18.230002 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:18.229971    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/743f9a1e-c695-454d-823d-7939bfbb6aff-openshift-service-ca" pod "743f9a1e-c695-454d-823d-7939bfbb6aff" (UID: "743f9a1e-c695-454d-823d-7939bfbb6aff"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:30:18.230255 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:18.230232    5510 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/743f9a1e-c695-454d-823d-7939bfbb6aff-openshift-service-ca\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:30:18.231616 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:18.231596    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/743f9a1e-c695-454d-823d-7939bfbb6aff-maas-api-tls" pod "743f9a1e-c695-454d-823d-7939bfbb6aff" (UID: "743f9a1e-c695-454d-823d-7939bfbb6aff"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:30:18.231750 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:18.231614    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/743f9a1e-c695-454d-823d-7939bfbb6aff-metrics-certs" pod "743f9a1e-c695-454d-823d-7939bfbb6aff" (UID: "743f9a1e-c695-454d-823d-7939bfbb6aff"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:30:18.231812 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:18.231795    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/743f9a1e-c695-454d-823d-7939bfbb6aff-kube-api-access-v2gzl" pod "743f9a1e-c695-454d-823d-7939bfbb6aff" (UID: "743f9a1e-c695-454d-823d-7939bfbb6aff"). InnerVolumeSpecName "kube-api-access-v2gzl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:30:18.338750 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:18.331797    5510 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v2gzl\" (UniqueName: \"kubernetes.io/projected/743f9a1e-c695-454d-823d-7939bfbb6aff-kube-api-access-v2gzl\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:30:18.338750 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:18.331835    5510 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/743f9a1e-c695-454d-823d-7939bfbb6aff-metrics-certs\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:30:18.338750 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:18.331851    5510 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/743f9a1e-c695-454d-823d-7939bfbb6aff-maas-api-tls\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:30:18.338750 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:18.334472    5510 generic.go:356] "Generic (PLEG): container finished" podID="743f9a1e-c695-454d-823d-7939bfbb6aff" containerID="9d9ffef4ecd0bda9ca49728eb104e67aef8a77a96c6c2a27a7b7a50ced01ea84" exitCode=0
Oct 30 22:30:18.338750 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:18.334546    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-9f3ea1-65cfd5854b-lcwjw" event={"ID":"743f9a1e-c695-454d-823d-7939bfbb6aff","Type":"ContainerDied","Data":"9d9ffef4ecd0bda9ca49728eb104e67aef8a77a96c6c2a27a7b7a50ced01ea84"}
Oct 30 22:30:18.338750 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:18.334613    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-9f3ea1-65cfd5854b-lcwjw" event={"ID":"743f9a1e-c695-454d-823d-7939bfbb6aff","Type":"ContainerDied","Data":"46ec1e12dc3486f81d06e592d272c166a7a6b37b4f1c2582a56df73f0d985983"}
Oct 30 22:30:18.338750 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:18.334644    5510 scope.go:122] "RemoveContainer" containerID="9d9ffef4ecd0bda9ca49728eb104e67aef8a77a96c6c2a27a7b7a50ced01ea84"
Oct 30 22:30:18.338750 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:18.335054    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-9f3ea1-65cfd5854b-lcwjw"
Oct 30 22:30:18.370179 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:18.370158    5510 scope.go:122] "RemoveContainer" containerID="9d9ffef4ecd0bda9ca49728eb104e67aef8a77a96c6c2a27a7b7a50ced01ea84"
Oct 30 22:30:18.370578 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:30:18.370554    5510 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9d9ffef4ecd0bda9ca49728eb104e67aef8a77a96c6c2a27a7b7a50ced01ea84\": container with ID starting with 9d9ffef4ecd0bda9ca49728eb104e67aef8a77a96c6c2a27a7b7a50ced01ea84 not found: ID does not exist" containerID="9d9ffef4ecd0bda9ca49728eb104e67aef8a77a96c6c2a27a7b7a50ced01ea84"
Oct 30 22:30:18.370707 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:18.370590    5510 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9d9ffef4ecd0bda9ca49728eb104e67aef8a77a96c6c2a27a7b7a50ced01ea84"} err="failed to get container status \"9d9ffef4ecd0bda9ca49728eb104e67aef8a77a96c6c2a27a7b7a50ced01ea84\": rpc error: code = NotFound desc = could not find container \"9d9ffef4ecd0bda9ca49728eb104e67aef8a77a96c6c2a27a7b7a50ced01ea84\": container with ID starting with 9d9ffef4ecd0bda9ca49728eb104e67aef8a77a96c6c2a27a7b7a50ced01ea84 not found: ID does not exist"
Oct 30 22:30:18.493411 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:18.493318    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-9f3ea1-65cfd5854b-lcwjw"]
Oct 30 22:30:18.500934 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:18.500900    5510 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-9f3ea1-65cfd5854b-lcwjw"]
Oct 30 22:30:19.520857 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:19.520824    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:30:19.568468 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:19.568429    5510 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="743f9a1e-c695-454d-823d-7939bfbb6aff" path="/var/lib/kubelet/pods/743f9a1e-c695-454d-823d-7939bfbb6aff/volumes"
Oct 30 22:30:19.810213 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:19.810119    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:30:20.015870 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:20.015835    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:30:20.485355 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:20.485297    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:30:21.547482 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:21.547452    5510 scope.go:122] "RemoveContainer" containerID="d4eddbf72059bcb5da1bccb5e0524779180897ab9ec4d1b0d6ecb06521b60860"
Oct 30 22:30:22.365255 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:22.365223    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889990-27cbr" event={"ID":"abac65c3-1048-47a8-b307-c2e002e6f7d5","Type":"ContainerStarted","Data":"a56f602d434454f06b2949ea3f6f50a0111b5a1cd053aa3ea2b0f0a84a2cfcd1"}
Oct 30 22:30:22.365711 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:22.365666    5510 scope.go:122] "RemoveContainer" containerID="a56f602d434454f06b2949ea3f6f50a0111b5a1cd053aa3ea2b0f0a84a2cfcd1"
Oct 30 22:30:22.366027 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:30:22.366008    5510 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-29889990-27cbr_odh-ai-gateway-infra(abac65c3-1048-47a8-b307-c2e002e6f7d5)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889990-27cbr" podUID="abac65c3-1048-47a8-b307-c2e002e6f7d5"
Oct 30 22:30:23.453679 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:23.450001    5510 generic.go:356] "Generic (PLEG): container finished" podID="abac65c3-1048-47a8-b307-c2e002e6f7d5" containerID="a56f602d434454f06b2949ea3f6f50a0111b5a1cd053aa3ea2b0f0a84a2cfcd1" exitCode=60
Oct 30 22:30:23.453679 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:23.450189    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889990-27cbr" event={"ID":"abac65c3-1048-47a8-b307-c2e002e6f7d5","Type":"ContainerDied","Data":"a56f602d434454f06b2949ea3f6f50a0111b5a1cd053aa3ea2b0f0a84a2cfcd1"}
Oct 30 22:30:23.453679 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:23.450275    5510 scope.go:122] "RemoveContainer" containerID="d4eddbf72059bcb5da1bccb5e0524779180897ab9ec4d1b0d6ecb06521b60860"
Oct 30 22:30:23.453679 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:23.450830    5510 scope.go:122] "RemoveContainer" containerID="a56f602d434454f06b2949ea3f6f50a0111b5a1cd053aa3ea2b0f0a84a2cfcd1"
Oct 30 22:30:23.453679 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:30:23.451219    5510 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-29889990-27cbr_odh-ai-gateway-infra(abac65c3-1048-47a8-b307-c2e002e6f7d5)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889990-27cbr" podUID="abac65c3-1048-47a8-b307-c2e002e6f7d5"
Oct 30 22:30:23.494763 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:23.494717    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29889990-27cbr"]
Oct 30 22:30:24.241362 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:24.241324    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:30:24.625875 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:24.625844    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889990-27cbr"
Oct 30 22:30:24.796302 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:24.796204    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/abac65c3-1048-47a8-b307-c2e002e6f7d5-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/abac65c3-1048-47a8-b307-c2e002e6f7d5-maas-api-service-ca\") pod \"abac65c3-1048-47a8-b307-c2e002e6f7d5\" (UID: \"abac65c3-1048-47a8-b307-c2e002e6f7d5\") "
Oct 30 22:30:24.796491 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:24.796410    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/abac65c3-1048-47a8-b307-c2e002e6f7d5-kube-api-access-27tjp\" (UniqueName: \"kubernetes.io/projected/abac65c3-1048-47a8-b307-c2e002e6f7d5-kube-api-access-27tjp\") pod \"abac65c3-1048-47a8-b307-c2e002e6f7d5\" (UID: \"abac65c3-1048-47a8-b307-c2e002e6f7d5\") "
Oct 30 22:30:24.796706 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:24.796643    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/abac65c3-1048-47a8-b307-c2e002e6f7d5-maas-api-service-ca" pod "abac65c3-1048-47a8-b307-c2e002e6f7d5" (UID: "abac65c3-1048-47a8-b307-c2e002e6f7d5"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:30:24.797093 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:24.797069    5510 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/abac65c3-1048-47a8-b307-c2e002e6f7d5-maas-api-service-ca\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:30:24.802505 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:24.802474    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/abac65c3-1048-47a8-b307-c2e002e6f7d5-kube-api-access-27tjp" pod "abac65c3-1048-47a8-b307-c2e002e6f7d5" (UID: "abac65c3-1048-47a8-b307-c2e002e6f7d5"). InnerVolumeSpecName "kube-api-access-27tjp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:30:24.897964 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:24.897927    5510 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-27tjp\" (UniqueName: \"kubernetes.io/projected/abac65c3-1048-47a8-b307-c2e002e6f7d5-kube-api-access-27tjp\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:30:25.343481 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:25.343444    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:30:25.466698 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:25.466639    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889990-27cbr"
Oct 30 22:30:25.466914 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:25.466640    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889990-27cbr" event={"ID":"abac65c3-1048-47a8-b307-c2e002e6f7d5","Type":"ContainerDied","Data":"03b79c4e57b83437667d6c546fb9b103be05be0365856d4c1cc1c7608618ee0a"}
Oct 30 22:30:25.466914 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:25.466819    5510 scope.go:122] "RemoveContainer" containerID="a56f602d434454f06b2949ea3f6f50a0111b5a1cd053aa3ea2b0f0a84a2cfcd1"
Oct 30 22:30:25.623208 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:25.623108    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29889990-27cbr"]
Oct 30 22:30:25.637393 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:25.637360    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:30:25.669146 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:25.669112    5510 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29889990-27cbr"]
Oct 30 22:30:25.895355 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:25.895264    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:30:26.167036 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:26.166940    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:30:26.396283 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:26.396243    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:30:26.890146 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:26.890110    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:30:27.553807 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:27.553772    5510 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="abac65c3-1048-47a8-b307-c2e002e6f7d5" path="/var/lib/kubelet/pods/abac65c3-1048-47a8-b307-c2e002e6f7d5/volumes"
Oct 30 22:30:28.107928 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:28.107897    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:30:28.606260 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:28.606225    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:30:28.845106 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:28.845068    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:30:29.612795 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:29.612755    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:30:29.836213 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:29.836179    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:30:31.348410 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:31.348368    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:30:32.143732 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:32.143688    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:30:33.173567 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:33.173419    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:30:33.383156 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:33.383119    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:30:33.487189 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:33.487109    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-7f76c4-openshift-default-5969946659-5dwws"]
Oct 30 22:30:33.487381 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:33.487344    5510 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-7f76c4-openshift-default-5969946659-5dwws" podUID="11bc04c5-986f-47f9-93d1-0f803b775eb5" containerName="istio-proxy" containerID="cri-o://0f5feb6ac3de99eb05c29e507b3479c1cbb6551a68e5b1b7b816e2ca5ff6a31c" gracePeriod=30
Oct 30 22:30:33.668676 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:33.668302    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:30:36.204624 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:36.199297    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:30:36.405892 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:36.405855    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:30:36.562921 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:36.562871    5510 patch_prober.go:29] interesting pod/e2e-api-a-7f76c4-openshift-default-5969946659-5dwws container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 22:30:36.563112 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:36.562951    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-7f76c4-openshift-default-5969946659-5dwws" podUID="11bc04c5-986f-47f9-93d1-0f803b775eb5" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 22:30:36.623015 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:36.622981    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:30:38.420946 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:30:38.420900    5510 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6gjbf" containerName="extractor"
Oct 30 22:30:38.761455 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:38.761426    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-7f76c4-openshift-default-5969946659-5dwws"
Oct 30 22:30:38.836583 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:38.836547    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/11bc04c5-986f-47f9-93d1-0f803b775eb5-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/11bc04c5-986f-47f9-93d1-0f803b775eb5-istio-podinfo\") pod \"11bc04c5-986f-47f9-93d1-0f803b775eb5\" (UID: \"11bc04c5-986f-47f9-93d1-0f803b775eb5\") "
Oct 30 22:30:38.836800 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:38.836626    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/11bc04c5-986f-47f9-93d1-0f803b775eb5-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/11bc04c5-986f-47f9-93d1-0f803b775eb5-istio-envoy\") pod \"11bc04c5-986f-47f9-93d1-0f803b775eb5\" (UID: \"11bc04c5-986f-47f9-93d1-0f803b775eb5\") "
Oct 30 22:30:38.836800 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:38.836689    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/11bc04c5-986f-47f9-93d1-0f803b775eb5-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/11bc04c5-986f-47f9-93d1-0f803b775eb5-workload-socket\") pod \"11bc04c5-986f-47f9-93d1-0f803b775eb5\" (UID: \"11bc04c5-986f-47f9-93d1-0f803b775eb5\") "
Oct 30 22:30:38.836800 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:38.836740    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/11bc04c5-986f-47f9-93d1-0f803b775eb5-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/11bc04c5-986f-47f9-93d1-0f803b775eb5-istiod-ca-cert\") pod \"11bc04c5-986f-47f9-93d1-0f803b775eb5\" (UID: \"11bc04c5-986f-47f9-93d1-0f803b775eb5\") "
Oct 30 22:30:38.836800 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:38.836768    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/11bc04c5-986f-47f9-93d1-0f803b775eb5-kube-api-access-rdx2c\" (UniqueName: \"kubernetes.io/projected/11bc04c5-986f-47f9-93d1-0f803b775eb5-kube-api-access-rdx2c\") pod \"11bc04c5-986f-47f9-93d1-0f803b775eb5\" (UID: \"11bc04c5-986f-47f9-93d1-0f803b775eb5\") "
Oct 30 22:30:38.837094 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:38.836902    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/11bc04c5-986f-47f9-93d1-0f803b775eb5-istio-token\" (UniqueName: \"kubernetes.io/projected/11bc04c5-986f-47f9-93d1-0f803b775eb5-istio-token\") pod \"11bc04c5-986f-47f9-93d1-0f803b775eb5\" (UID: \"11bc04c5-986f-47f9-93d1-0f803b775eb5\") "
Oct 30 22:30:38.837094 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:38.837067    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/11bc04c5-986f-47f9-93d1-0f803b775eb5-workload-socket" pod "11bc04c5-986f-47f9-93d1-0f803b775eb5" (UID: "11bc04c5-986f-47f9-93d1-0f803b775eb5"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:30:38.837214 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:38.837089    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/11bc04c5-986f-47f9-93d1-0f803b775eb5-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/11bc04c5-986f-47f9-93d1-0f803b775eb5-istio-data\") pod \"11bc04c5-986f-47f9-93d1-0f803b775eb5\" (UID: \"11bc04c5-986f-47f9-93d1-0f803b775eb5\") "
Oct 30 22:30:38.837214 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:38.837185    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/11bc04c5-986f-47f9-93d1-0f803b775eb5-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/11bc04c5-986f-47f9-93d1-0f803b775eb5-workload-certs\") pod \"11bc04c5-986f-47f9-93d1-0f803b775eb5\" (UID: \"11bc04c5-986f-47f9-93d1-0f803b775eb5\") "
Oct 30 22:30:38.837329 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:38.837212    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/11bc04c5-986f-47f9-93d1-0f803b775eb5-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/11bc04c5-986f-47f9-93d1-0f803b775eb5-credential-socket\") pod \"11bc04c5-986f-47f9-93d1-0f803b775eb5\" (UID: \"11bc04c5-986f-47f9-93d1-0f803b775eb5\") "
Oct 30 22:30:38.837329 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:38.837180    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/11bc04c5-986f-47f9-93d1-0f803b775eb5-istiod-ca-cert" pod "11bc04c5-986f-47f9-93d1-0f803b775eb5" (UID: "11bc04c5-986f-47f9-93d1-0f803b775eb5"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:30:38.837464 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:38.837372    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/11bc04c5-986f-47f9-93d1-0f803b775eb5-istio-data" pod "11bc04c5-986f-47f9-93d1-0f803b775eb5" (UID: "11bc04c5-986f-47f9-93d1-0f803b775eb5"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:30:38.837464 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:38.837434    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/11bc04c5-986f-47f9-93d1-0f803b775eb5-workload-certs" pod "11bc04c5-986f-47f9-93d1-0f803b775eb5" (UID: "11bc04c5-986f-47f9-93d1-0f803b775eb5"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:30:38.837577 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:38.837531    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/11bc04c5-986f-47f9-93d1-0f803b775eb5-credential-socket" pod "11bc04c5-986f-47f9-93d1-0f803b775eb5" (UID: "11bc04c5-986f-47f9-93d1-0f803b775eb5"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:30:38.838024 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:38.837991    5510 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/11bc04c5-986f-47f9-93d1-0f803b775eb5-istio-data\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:30:38.838024 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:38.838020    5510 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/11bc04c5-986f-47f9-93d1-0f803b775eb5-workload-certs\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:30:38.838202 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:38.838038    5510 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/11bc04c5-986f-47f9-93d1-0f803b775eb5-credential-socket\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:30:38.838202 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:38.838052    5510 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/11bc04c5-986f-47f9-93d1-0f803b775eb5-workload-socket\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:30:38.838202 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:38.838067    5510 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/11bc04c5-986f-47f9-93d1-0f803b775eb5-istiod-ca-cert\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:30:38.839213 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:38.839184    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/11bc04c5-986f-47f9-93d1-0f803b775eb5-istio-envoy" pod "11bc04c5-986f-47f9-93d1-0f803b775eb5" (UID: "11bc04c5-986f-47f9-93d1-0f803b775eb5"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:30:38.839415 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:38.839386    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/11bc04c5-986f-47f9-93d1-0f803b775eb5-kube-api-access-rdx2c" pod "11bc04c5-986f-47f9-93d1-0f803b775eb5" (UID: "11bc04c5-986f-47f9-93d1-0f803b775eb5"). InnerVolumeSpecName "kube-api-access-rdx2c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:30:38.839506 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:38.839439    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/11bc04c5-986f-47f9-93d1-0f803b775eb5-istio-podinfo" pod "11bc04c5-986f-47f9-93d1-0f803b775eb5" (UID: "11bc04c5-986f-47f9-93d1-0f803b775eb5"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:30:38.839612 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:38.839596    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/11bc04c5-986f-47f9-93d1-0f803b775eb5-istio-token" pod "11bc04c5-986f-47f9-93d1-0f803b775eb5" (UID: "11bc04c5-986f-47f9-93d1-0f803b775eb5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:30:38.945724 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:38.942823    5510 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/11bc04c5-986f-47f9-93d1-0f803b775eb5-istio-podinfo\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:30:38.945724 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:38.942862    5510 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/11bc04c5-986f-47f9-93d1-0f803b775eb5-istio-envoy\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:30:38.945724 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:38.942877    5510 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rdx2c\" (UniqueName: \"kubernetes.io/projected/11bc04c5-986f-47f9-93d1-0f803b775eb5-kube-api-access-rdx2c\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:30:38.945724 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:38.942892    5510 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/11bc04c5-986f-47f9-93d1-0f803b775eb5-istio-token\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:30:39.557621 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:39.557587    5510 generic.go:356] "Generic (PLEG): container finished" podID="11bc04c5-986f-47f9-93d1-0f803b775eb5" containerID="0f5feb6ac3de99eb05c29e507b3479c1cbb6551a68e5b1b7b816e2ca5ff6a31c" exitCode=0
Oct 30 22:30:39.558149 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:39.557630    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-7f76c4-openshift-default-5969946659-5dwws"
Oct 30 22:30:39.558149 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:39.557675    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-7f76c4-openshift-default-5969946659-5dwws" event={"ID":"11bc04c5-986f-47f9-93d1-0f803b775eb5","Type":"ContainerDied","Data":"0f5feb6ac3de99eb05c29e507b3479c1cbb6551a68e5b1b7b816e2ca5ff6a31c"}
Oct 30 22:30:39.558149 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:39.557740    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-7f76c4-openshift-default-5969946659-5dwws" event={"ID":"11bc04c5-986f-47f9-93d1-0f803b775eb5","Type":"ContainerDied","Data":"f54a19eb0a6b8a12aaf386c5b58af2a50880c997015b29a2dae1d949fc66fac8"}
Oct 30 22:30:39.558149 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:39.557772    5510 scope.go:122] "RemoveContainer" containerID="0f5feb6ac3de99eb05c29e507b3479c1cbb6551a68e5b1b7b816e2ca5ff6a31c"
Oct 30 22:30:39.568813 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:39.568797    5510 scope.go:122] "RemoveContainer" containerID="128a7a0a8c823b9a180f97a1b06a35c9fe3cc8c144345631ac24025deac1a841"
Oct 30 22:30:39.578861 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:39.578842    5510 scope.go:122] "RemoveContainer" containerID="0f5feb6ac3de99eb05c29e507b3479c1cbb6551a68e5b1b7b816e2ca5ff6a31c"
Oct 30 22:30:39.579180 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:30:39.579161    5510 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0f5feb6ac3de99eb05c29e507b3479c1cbb6551a68e5b1b7b816e2ca5ff6a31c\": container with ID starting with 0f5feb6ac3de99eb05c29e507b3479c1cbb6551a68e5b1b7b816e2ca5ff6a31c not found: ID does not exist" containerID="0f5feb6ac3de99eb05c29e507b3479c1cbb6551a68e5b1b7b816e2ca5ff6a31c"
Oct 30 22:30:39.579243 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:39.579188    5510 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0f5feb6ac3de99eb05c29e507b3479c1cbb6551a68e5b1b7b816e2ca5ff6a31c"} err="failed to get container status \"0f5feb6ac3de99eb05c29e507b3479c1cbb6551a68e5b1b7b816e2ca5ff6a31c\": rpc error: code = NotFound desc = could not find container \"0f5feb6ac3de99eb05c29e507b3479c1cbb6551a68e5b1b7b816e2ca5ff6a31c\": container with ID starting with 0f5feb6ac3de99eb05c29e507b3479c1cbb6551a68e5b1b7b816e2ca5ff6a31c not found: ID does not exist"
Oct 30 22:30:39.579243 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:39.579206    5510 scope.go:122] "RemoveContainer" containerID="128a7a0a8c823b9a180f97a1b06a35c9fe3cc8c144345631ac24025deac1a841"
Oct 30 22:30:39.579517 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:30:39.579497    5510 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"128a7a0a8c823b9a180f97a1b06a35c9fe3cc8c144345631ac24025deac1a841\": container with ID starting with 128a7a0a8c823b9a180f97a1b06a35c9fe3cc8c144345631ac24025deac1a841 not found: ID does not exist" containerID="128a7a0a8c823b9a180f97a1b06a35c9fe3cc8c144345631ac24025deac1a841"
Oct 30 22:30:39.579570 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:39.579524    5510 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"128a7a0a8c823b9a180f97a1b06a35c9fe3cc8c144345631ac24025deac1a841"} err="failed to get container status \"128a7a0a8c823b9a180f97a1b06a35c9fe3cc8c144345631ac24025deac1a841\": rpc error: code = NotFound desc = could not find container \"128a7a0a8c823b9a180f97a1b06a35c9fe3cc8c144345631ac24025deac1a841\": container with ID starting with 128a7a0a8c823b9a180f97a1b06a35c9fe3cc8c144345631ac24025deac1a841 not found: ID does not exist"
Oct 30 22:30:39.716725 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:39.716684    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-7f76c4-openshift-default-5969946659-5dwws"]
Oct 30 22:30:39.768303 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:39.768267    5510 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-7f76c4-openshift-default-5969946659-5dwws"]
Oct 30 22:30:40.573599 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:40.573569    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:30:40.867758 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:40.867644    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:30:41.402315 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:41.402276    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:30:41.554922 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:41.554890    5510 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="11bc04c5-986f-47f9-93d1-0f803b775eb5" path="/var/lib/kubelet/pods/11bc04c5-986f-47f9-93d1-0f803b775eb5/volumes"
Oct 30 22:30:44.785245 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:44.785206    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-5686cb-7b8659769-g92j9"]
Oct 30 22:30:44.884516 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:44.884480    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-5686cb-6c9b4b9b7f-pt4gf"]
Oct 30 22:30:45.193566 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:45.193474    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:30:45.407450 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:45.407417    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:30:46.251971 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:46.232247    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:30:49.786604 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:49.786558    5510 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-b-5686cb-7b8659769-g92j9" podUID="bc1e8851-bb44-4e8d-a9da-387642b1c23c" containerName="payload-processing" containerID="cri-o://d06c3b6bf9e2e69f4be3ef8b14048417b6e082f6158eaeeafb57c72f29a354f3" gracePeriod=25
Oct 30 22:30:49.885512 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:49.885467    5510 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-b-5686cb-6c9b4b9b7f-pt4gf" podUID="32e3490e-853c-4d7b-8c67-4423a2b89be3" containerName="payload-pre-processing" containerID="cri-o://6272f039d4715f19aba5aaf7a13df82dcdc8f52aa8681ded3e7aed98801ba026" gracePeriod=25
Oct 30 22:30:50.069192 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:50.069165    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-5686cb-7b8659769-g92j9"
Oct 30 22:30:50.153423 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:50.153392    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-5686cb-6c9b4b9b7f-pt4gf"
Oct 30 22:30:50.222141 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:50.222111    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/bc1e8851-bb44-4e8d-a9da-387642b1c23c-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bc1e8851-bb44-4e8d-a9da-387642b1c23c-plugins-config-volume\") pod \"bc1e8851-bb44-4e8d-a9da-387642b1c23c\" (UID: \"bc1e8851-bb44-4e8d-a9da-387642b1c23c\") "
Oct 30 22:30:50.222341 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:50.222269    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bc1e8851-bb44-4e8d-a9da-387642b1c23c-kube-api-access-8zpzv\" (UniqueName: \"kubernetes.io/projected/bc1e8851-bb44-4e8d-a9da-387642b1c23c-kube-api-access-8zpzv\") pod \"bc1e8851-bb44-4e8d-a9da-387642b1c23c\" (UID: \"bc1e8851-bb44-4e8d-a9da-387642b1c23c\") "
Oct 30 22:30:50.222483 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:50.222460    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bc1e8851-bb44-4e8d-a9da-387642b1c23c-plugins-config-volume" pod "bc1e8851-bb44-4e8d-a9da-387642b1c23c" (UID: "bc1e8851-bb44-4e8d-a9da-387642b1c23c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:30:50.222819 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:50.222800    5510 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bc1e8851-bb44-4e8d-a9da-387642b1c23c-plugins-config-volume\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:30:50.224488 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:50.224459    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bc1e8851-bb44-4e8d-a9da-387642b1c23c-kube-api-access-8zpzv" pod "bc1e8851-bb44-4e8d-a9da-387642b1c23c" (UID: "bc1e8851-bb44-4e8d-a9da-387642b1c23c"). InnerVolumeSpecName "kube-api-access-8zpzv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:30:50.323935 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:50.323841    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/32e3490e-853c-4d7b-8c67-4423a2b89be3-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/32e3490e-853c-4d7b-8c67-4423a2b89be3-plugins-config-volume\") pod \"32e3490e-853c-4d7b-8c67-4423a2b89be3\" (UID: \"32e3490e-853c-4d7b-8c67-4423a2b89be3\") "
Oct 30 22:30:50.324122 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:50.324105    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/32e3490e-853c-4d7b-8c67-4423a2b89be3-kube-api-access-7j49g\" (UniqueName: \"kubernetes.io/projected/32e3490e-853c-4d7b-8c67-4423a2b89be3-kube-api-access-7j49g\") pod \"32e3490e-853c-4d7b-8c67-4423a2b89be3\" (UID: \"32e3490e-853c-4d7b-8c67-4423a2b89be3\") "
Oct 30 22:30:50.324289 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:50.324265    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/32e3490e-853c-4d7b-8c67-4423a2b89be3-plugins-config-volume" pod "32e3490e-853c-4d7b-8c67-4423a2b89be3" (UID: "32e3490e-853c-4d7b-8c67-4423a2b89be3"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:30:50.324613 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:50.324593    5510 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8zpzv\" (UniqueName: \"kubernetes.io/projected/bc1e8851-bb44-4e8d-a9da-387642b1c23c-kube-api-access-8zpzv\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:30:50.324718 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:50.324619    5510 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/32e3490e-853c-4d7b-8c67-4423a2b89be3-plugins-config-volume\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:30:50.326334 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:50.326301    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/32e3490e-853c-4d7b-8c67-4423a2b89be3-kube-api-access-7j49g" pod "32e3490e-853c-4d7b-8c67-4423a2b89be3" (UID: "32e3490e-853c-4d7b-8c67-4423a2b89be3"). InnerVolumeSpecName "kube-api-access-7j49g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:30:50.426355 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:50.426324    5510 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7j49g\" (UniqueName: \"kubernetes.io/projected/32e3490e-853c-4d7b-8c67-4423a2b89be3-kube-api-access-7j49g\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:30:50.635021 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:50.634936    5510 generic.go:356] "Generic (PLEG): container finished" podID="32e3490e-853c-4d7b-8c67-4423a2b89be3" containerID="6272f039d4715f19aba5aaf7a13df82dcdc8f52aa8681ded3e7aed98801ba026" exitCode=0
Oct 30 22:30:50.635021 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:50.634978    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-5686cb-6c9b4b9b7f-pt4gf"
Oct 30 22:30:50.635021 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:50.635008    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-5686cb-6c9b4b9b7f-pt4gf" event={"ID":"32e3490e-853c-4d7b-8c67-4423a2b89be3","Type":"ContainerDied","Data":"6272f039d4715f19aba5aaf7a13df82dcdc8f52aa8681ded3e7aed98801ba026"}
Oct 30 22:30:50.635291 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:50.635057    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-5686cb-6c9b4b9b7f-pt4gf" event={"ID":"32e3490e-853c-4d7b-8c67-4423a2b89be3","Type":"ContainerDied","Data":"af36ef95b9da6ef97da1dba38fb05fca90062ab0f8860d107af7b4dece0d4def"}
Oct 30 22:30:50.635291 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:50.635083    5510 scope.go:122] "RemoveContainer" containerID="6272f039d4715f19aba5aaf7a13df82dcdc8f52aa8681ded3e7aed98801ba026"
Oct 30 22:30:50.636555 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:50.636531    5510 generic.go:356] "Generic (PLEG): container finished" podID="bc1e8851-bb44-4e8d-a9da-387642b1c23c" containerID="d06c3b6bf9e2e69f4be3ef8b14048417b6e082f6158eaeeafb57c72f29a354f3" exitCode=0
Oct 30 22:30:50.636712 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:50.636574    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-5686cb-7b8659769-g92j9"
Oct 30 22:30:50.636712 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:50.636582    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-5686cb-7b8659769-g92j9" event={"ID":"bc1e8851-bb44-4e8d-a9da-387642b1c23c","Type":"ContainerDied","Data":"d06c3b6bf9e2e69f4be3ef8b14048417b6e082f6158eaeeafb57c72f29a354f3"}
Oct 30 22:30:50.636712 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:50.636621    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-5686cb-7b8659769-g92j9" event={"ID":"bc1e8851-bb44-4e8d-a9da-387642b1c23c","Type":"ContainerDied","Data":"fa2699b4c3c7a3577630579d61816d853e95ccbca05f67588bb702452f7c636f"}
Oct 30 22:30:50.645576 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:50.645558    5510 scope.go:122] "RemoveContainer" containerID="6272f039d4715f19aba5aaf7a13df82dcdc8f52aa8681ded3e7aed98801ba026"
Oct 30 22:30:50.645910 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:30:50.645884    5510 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6272f039d4715f19aba5aaf7a13df82dcdc8f52aa8681ded3e7aed98801ba026\": container with ID starting with 6272f039d4715f19aba5aaf7a13df82dcdc8f52aa8681ded3e7aed98801ba026 not found: ID does not exist" containerID="6272f039d4715f19aba5aaf7a13df82dcdc8f52aa8681ded3e7aed98801ba026"
Oct 30 22:30:50.645993 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:50.645919    5510 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6272f039d4715f19aba5aaf7a13df82dcdc8f52aa8681ded3e7aed98801ba026"} err="failed to get container status \"6272f039d4715f19aba5aaf7a13df82dcdc8f52aa8681ded3e7aed98801ba026\": rpc error: code = NotFound desc = could not find container \"6272f039d4715f19aba5aaf7a13df82dcdc8f52aa8681ded3e7aed98801ba026\": container with ID starting with 6272f039d4715f19aba5aaf7a13df82dcdc8f52aa8681ded3e7aed98801ba026 not found: ID does not exist"
Oct 30 22:30:50.645993 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:50.645939    5510 scope.go:122] "RemoveContainer" containerID="d06c3b6bf9e2e69f4be3ef8b14048417b6e082f6158eaeeafb57c72f29a354f3"
Oct 30 22:30:50.655255 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:50.655236    5510 scope.go:122] "RemoveContainer" containerID="d06c3b6bf9e2e69f4be3ef8b14048417b6e082f6158eaeeafb57c72f29a354f3"
Oct 30 22:30:50.655521 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:30:50.655503    5510 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d06c3b6bf9e2e69f4be3ef8b14048417b6e082f6158eaeeafb57c72f29a354f3\": container with ID starting with d06c3b6bf9e2e69f4be3ef8b14048417b6e082f6158eaeeafb57c72f29a354f3 not found: ID does not exist" containerID="d06c3b6bf9e2e69f4be3ef8b14048417b6e082f6158eaeeafb57c72f29a354f3"
Oct 30 22:30:50.655574 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:50.655526    5510 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d06c3b6bf9e2e69f4be3ef8b14048417b6e082f6158eaeeafb57c72f29a354f3"} err="failed to get container status \"d06c3b6bf9e2e69f4be3ef8b14048417b6e082f6158eaeeafb57c72f29a354f3\": rpc error: code = NotFound desc = could not find container \"d06c3b6bf9e2e69f4be3ef8b14048417b6e082f6158eaeeafb57c72f29a354f3\": container with ID starting with d06c3b6bf9e2e69f4be3ef8b14048417b6e082f6158eaeeafb57c72f29a354f3 not found: ID does not exist"
Oct 30 22:30:50.800693 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:50.788200    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:30:50.809350 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:50.809311    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-5686cb-7b8659769-g92j9"]
Oct 30 22:30:50.811764 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:50.811737    5510 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-5686cb-7b8659769-g92j9"]
Oct 30 22:30:50.901047 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:50.900967    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-5686cb-6c9b4b9b7f-pt4gf"]
Oct 30 22:30:50.904887 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:50.904859    5510 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-5686cb-6c9b4b9b7f-pt4gf"]
Oct 30 22:30:51.554740 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:51.554700    5510 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="32e3490e-853c-4d7b-8c67-4423a2b89be3" path="/var/lib/kubelet/pods/32e3490e-853c-4d7b-8c67-4423a2b89be3/volumes"
Oct 30 22:30:51.555162 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:51.555148    5510 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="bc1e8851-bb44-4e8d-a9da-387642b1c23c" path="/var/lib/kubelet/pods/bc1e8851-bb44-4e8d-a9da-387642b1c23c/volumes"
Oct 30 22:30:54.977425 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:54.977375    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:30:56.380775 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:56.380738    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:30:56.589857 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:56.589820    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:30:56.804049 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:56.804009    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:30:57.344618 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:30:57.344580    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:31:00.804776 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:00.804684    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:31:02.204333 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:02.204295    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:31:02.409304 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:02.409266    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:31:02.601338 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:02.601307    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:31:03.206729 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:03.206648    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:31:04.410434 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:04.410396    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:31:04.744875 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:04.744761    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:31:04.770731 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:04.770693    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w5-99304b-openshift-default-665c757b9-6t2xl"]
Oct 30 22:31:04.770950 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:04.770929    5510 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w5-99304b-openshift-default-665c757b9-6t2xl" podUID="c4152cef-f9cf-46c1-a9c4-78484ab58efe" containerName="istio-proxy" containerID="cri-o://f4221738b50279b211037184bfdeb5fb11332f527e5d64ebfb3deb31bc033520" gracePeriod=30
Oct 30 22:31:04.981453 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:04.981417    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:31:05.301112 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:05.301075    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:31:06.928268 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:06.928230    5510 patch_prober.go:29] interesting pod/e2e-shared-b-w5-99304b-openshift-default-665c757b9-6t2xl container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 22:31:06.928707 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:06.928290    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-w5-99304b-openshift-default-665c757b9-6t2xl" podUID="c4152cef-f9cf-46c1-a9c4-78484ab58efe" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 22:31:07.687010 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:07.686970    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:31:08.421771 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:31:08.421732    5510 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6gjbf" containerName="extractor"
Oct 30 22:31:09.777990 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:09.777952    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:31:10.043171 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:10.043118    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w5-99304b-openshift-default-665c757b9-6t2xl"
Oct 30 22:31:10.110621 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:10.110586    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c4152cef-f9cf-46c1-a9c4-78484ab58efe-istio-token\" (UniqueName: \"kubernetes.io/projected/c4152cef-f9cf-46c1-a9c4-78484ab58efe-istio-token\") pod \"c4152cef-f9cf-46c1-a9c4-78484ab58efe\" (UID: \"c4152cef-f9cf-46c1-a9c4-78484ab58efe\") "
Oct 30 22:31:10.110825 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:10.110635    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c4152cef-f9cf-46c1-a9c4-78484ab58efe-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c4152cef-f9cf-46c1-a9c4-78484ab58efe-workload-certs\") pod \"c4152cef-f9cf-46c1-a9c4-78484ab58efe\" (UID: \"c4152cef-f9cf-46c1-a9c4-78484ab58efe\") "
Oct 30 22:31:10.110825 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:10.110684    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c4152cef-f9cf-46c1-a9c4-78484ab58efe-kube-api-access-srwlx\" (UniqueName: \"kubernetes.io/projected/c4152cef-f9cf-46c1-a9c4-78484ab58efe-kube-api-access-srwlx\") pod \"c4152cef-f9cf-46c1-a9c4-78484ab58efe\" (UID: \"c4152cef-f9cf-46c1-a9c4-78484ab58efe\") "
Oct 30 22:31:10.110825 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:10.110781    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/c4152cef-f9cf-46c1-a9c4-78484ab58efe-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c4152cef-f9cf-46c1-a9c4-78484ab58efe-istio-podinfo\") pod \"c4152cef-f9cf-46c1-a9c4-78484ab58efe\" (UID: \"c4152cef-f9cf-46c1-a9c4-78484ab58efe\") "
Oct 30 22:31:10.110943 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:10.110910    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c4152cef-f9cf-46c1-a9c4-78484ab58efe-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c4152cef-f9cf-46c1-a9c4-78484ab58efe-istio-data\") pod \"c4152cef-f9cf-46c1-a9c4-78484ab58efe\" (UID: \"c4152cef-f9cf-46c1-a9c4-78484ab58efe\") "
Oct 30 22:31:10.110990 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:10.110973    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c4152cef-f9cf-46c1-a9c4-78484ab58efe-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c4152cef-f9cf-46c1-a9c4-78484ab58efe-istio-envoy\") pod \"c4152cef-f9cf-46c1-a9c4-78484ab58efe\" (UID: \"c4152cef-f9cf-46c1-a9c4-78484ab58efe\") "
Oct 30 22:31:10.111029 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:10.110976    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c4152cef-f9cf-46c1-a9c4-78484ab58efe-workload-certs" pod "c4152cef-f9cf-46c1-a9c4-78484ab58efe" (UID: "c4152cef-f9cf-46c1-a9c4-78484ab58efe"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:31:10.111029 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:10.111017    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c4152cef-f9cf-46c1-a9c4-78484ab58efe-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4152cef-f9cf-46c1-a9c4-78484ab58efe-credential-socket\") pod \"c4152cef-f9cf-46c1-a9c4-78484ab58efe\" (UID: \"c4152cef-f9cf-46c1-a9c4-78484ab58efe\") "
Oct 30 22:31:10.111103 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:10.111057    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c4152cef-f9cf-46c1-a9c4-78484ab58efe-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4152cef-f9cf-46c1-a9c4-78484ab58efe-workload-socket\") pod \"c4152cef-f9cf-46c1-a9c4-78484ab58efe\" (UID: \"c4152cef-f9cf-46c1-a9c4-78484ab58efe\") "
Oct 30 22:31:10.111171 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:10.111108    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c4152cef-f9cf-46c1-a9c4-78484ab58efe-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c4152cef-f9cf-46c1-a9c4-78484ab58efe-istiod-ca-cert\") pod \"c4152cef-f9cf-46c1-a9c4-78484ab58efe\" (UID: \"c4152cef-f9cf-46c1-a9c4-78484ab58efe\") "
Oct 30 22:31:10.111290 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:10.111188    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c4152cef-f9cf-46c1-a9c4-78484ab58efe-istio-data" pod "c4152cef-f9cf-46c1-a9c4-78484ab58efe" (UID: "c4152cef-f9cf-46c1-a9c4-78484ab58efe"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:31:10.111357 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:10.111298    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c4152cef-f9cf-46c1-a9c4-78484ab58efe-credential-socket" pod "c4152cef-f9cf-46c1-a9c4-78484ab58efe" (UID: "c4152cef-f9cf-46c1-a9c4-78484ab58efe"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:31:10.111357 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:10.111337    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c4152cef-f9cf-46c1-a9c4-78484ab58efe-workload-socket" pod "c4152cef-f9cf-46c1-a9c4-78484ab58efe" (UID: "c4152cef-f9cf-46c1-a9c4-78484ab58efe"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:31:10.111617 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:10.111590    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c4152cef-f9cf-46c1-a9c4-78484ab58efe-istiod-ca-cert" pod "c4152cef-f9cf-46c1-a9c4-78484ab58efe" (UID: "c4152cef-f9cf-46c1-a9c4-78484ab58efe"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:31:10.111974 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:10.111949    5510 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c4152cef-f9cf-46c1-a9c4-78484ab58efe-istio-data\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:31:10.112061 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:10.111981    5510 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4152cef-f9cf-46c1-a9c4-78484ab58efe-credential-socket\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:31:10.112061 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:10.111998    5510 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4152cef-f9cf-46c1-a9c4-78484ab58efe-workload-socket\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:31:10.112061 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:10.112013    5510 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c4152cef-f9cf-46c1-a9c4-78484ab58efe-istiod-ca-cert\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:31:10.112061 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:10.112027    5510 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c4152cef-f9cf-46c1-a9c4-78484ab58efe-workload-certs\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:31:10.113110 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:10.113081    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c4152cef-f9cf-46c1-a9c4-78484ab58efe-istio-podinfo" pod "c4152cef-f9cf-46c1-a9c4-78484ab58efe" (UID: "c4152cef-f9cf-46c1-a9c4-78484ab58efe"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:31:10.113507 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:10.113141    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c4152cef-f9cf-46c1-a9c4-78484ab58efe-istio-envoy" pod "c4152cef-f9cf-46c1-a9c4-78484ab58efe" (UID: "c4152cef-f9cf-46c1-a9c4-78484ab58efe"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:31:10.113507 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:10.113149    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c4152cef-f9cf-46c1-a9c4-78484ab58efe-istio-token" pod "c4152cef-f9cf-46c1-a9c4-78484ab58efe" (UID: "c4152cef-f9cf-46c1-a9c4-78484ab58efe"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:31:10.113507 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:10.113381    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c4152cef-f9cf-46c1-a9c4-78484ab58efe-kube-api-access-srwlx" pod "c4152cef-f9cf-46c1-a9c4-78484ab58efe" (UID: "c4152cef-f9cf-46c1-a9c4-78484ab58efe"). InnerVolumeSpecName "kube-api-access-srwlx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:31:10.213282 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:10.213240    5510 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c4152cef-f9cf-46c1-a9c4-78484ab58efe-istio-envoy\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:31:10.213282 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:10.213272    5510 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c4152cef-f9cf-46c1-a9c4-78484ab58efe-istio-token\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:31:10.213282 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:10.213287    5510 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-srwlx\" (UniqueName: \"kubernetes.io/projected/c4152cef-f9cf-46c1-a9c4-78484ab58efe-kube-api-access-srwlx\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:31:10.213581 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:10.213301    5510 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c4152cef-f9cf-46c1-a9c4-78484ab58efe-istio-podinfo\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:31:10.260313 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:10.260277    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w5-c9dbe6/distinct-w5-c9dbe6-kserve-7599fd86d9-bdhxd"]
Oct 30 22:31:10.296650 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:10.296559    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:31:10.504370 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:10.504321    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:31:10.536217 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:10.536181    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-v8m49"]
Oct 30 22:31:10.536513 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:10.536481    5510 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-v8m49" podUID="bd8fbe72-d370-4949-9314-ff91cdab6ff6" containerName="istio-proxy" containerID="cri-o://11632c267999841c4d2731b19a4d82fd58583e965ea9ca58b7dc3ac939125328" gracePeriod=30
Oct 30 22:31:10.705196 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:10.705096    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:31:10.864472 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:10.864439    5510 generic.go:356] "Generic (PLEG): container finished" podID="c4152cef-f9cf-46c1-a9c4-78484ab58efe" containerID="f4221738b50279b211037184bfdeb5fb11332f527e5d64ebfb3deb31bc033520" exitCode=0
Oct 30 22:31:10.864990 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:10.864499    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w5-99304b-openshift-default-665c757b9-6t2xl"
Oct 30 22:31:10.864990 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:10.864525    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w5-99304b-openshift-default-665c757b9-6t2xl" event={"ID":"c4152cef-f9cf-46c1-a9c4-78484ab58efe","Type":"ContainerDied","Data":"f4221738b50279b211037184bfdeb5fb11332f527e5d64ebfb3deb31bc033520"}
Oct 30 22:31:10.864990 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:10.864597    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w5-99304b-openshift-default-665c757b9-6t2xl" event={"ID":"c4152cef-f9cf-46c1-a9c4-78484ab58efe","Type":"ContainerDied","Data":"231c680116133f8e0e387f7e7569c5429921f7f2837a0ec60966df23af2f6e69"}
Oct 30 22:31:10.864990 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:10.864633    5510 scope.go:122] "RemoveContainer" containerID="f4221738b50279b211037184bfdeb5fb11332f527e5d64ebfb3deb31bc033520"
Oct 30 22:31:10.875557 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:10.875539    5510 scope.go:122] "RemoveContainer" containerID="e97bc2ef207378d6ceb74ad4321a4a3e7a692e93e16ed58c50e1d900ac569c18"
Oct 30 22:31:10.884566 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:10.884546    5510 scope.go:122] "RemoveContainer" containerID="f4221738b50279b211037184bfdeb5fb11332f527e5d64ebfb3deb31bc033520"
Oct 30 22:31:10.884964 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:31:10.884945    5510 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f4221738b50279b211037184bfdeb5fb11332f527e5d64ebfb3deb31bc033520\": container with ID starting with f4221738b50279b211037184bfdeb5fb11332f527e5d64ebfb3deb31bc033520 not found: ID does not exist" containerID="f4221738b50279b211037184bfdeb5fb11332f527e5d64ebfb3deb31bc033520"
Oct 30 22:31:10.885057 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:10.884978    5510 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f4221738b50279b211037184bfdeb5fb11332f527e5d64ebfb3deb31bc033520"} err="failed to get container status \"f4221738b50279b211037184bfdeb5fb11332f527e5d64ebfb3deb31bc033520\": rpc error: code = NotFound desc = could not find container \"f4221738b50279b211037184bfdeb5fb11332f527e5d64ebfb3deb31bc033520\": container with ID starting with f4221738b50279b211037184bfdeb5fb11332f527e5d64ebfb3deb31bc033520 not found: ID does not exist"
Oct 30 22:31:10.885057 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:10.885001    5510 scope.go:122] "RemoveContainer" containerID="e97bc2ef207378d6ceb74ad4321a4a3e7a692e93e16ed58c50e1d900ac569c18"
Oct 30 22:31:10.885248 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:31:10.885227    5510 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e97bc2ef207378d6ceb74ad4321a4a3e7a692e93e16ed58c50e1d900ac569c18\": container with ID starting with e97bc2ef207378d6ceb74ad4321a4a3e7a692e93e16ed58c50e1d900ac569c18 not found: ID does not exist" containerID="e97bc2ef207378d6ceb74ad4321a4a3e7a692e93e16ed58c50e1d900ac569c18"
Oct 30 22:31:10.885300 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:10.885253    5510 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e97bc2ef207378d6ceb74ad4321a4a3e7a692e93e16ed58c50e1d900ac569c18"} err="failed to get container status \"e97bc2ef207378d6ceb74ad4321a4a3e7a692e93e16ed58c50e1d900ac569c18\": rpc error: code = NotFound desc = could not find container \"e97bc2ef207378d6ceb74ad4321a4a3e7a692e93e16ed58c50e1d900ac569c18\": container with ID starting with e97bc2ef207378d6ceb74ad4321a4a3e7a692e93e16ed58c50e1d900ac569c18 not found: ID does not exist"
Oct 30 22:31:11.042669 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:11.042620    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w5-99304b-openshift-default-665c757b9-6t2xl"]
Oct 30 22:31:11.076179 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:11.076141    5510 patch_prober.go:29] interesting pod/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-v8m49 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 22:31:11.076377 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:11.076206    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-v8m49" podUID="bd8fbe72-d370-4949-9314-ff91cdab6ff6" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 22:31:11.088758 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:11.088714    5510 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w5-99304b-openshift-default-665c757b9-6t2xl"]
Oct 30 22:31:11.553984 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:11.553953    5510 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c4152cef-f9cf-46c1-a9c4-78484ab58efe" path="/var/lib/kubelet/pods/c4152cef-f9cf-46c1-a9c4-78484ab58efe/volumes"
Oct 30 22:31:12.623992 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:12.623897    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:31:15.808359 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:15.808298    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-v8m49"
Oct 30 22:31:15.818561 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:15.818532    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/bd8fbe72-d370-4949-9314-ff91cdab6ff6-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bd8fbe72-d370-4949-9314-ff91cdab6ff6-istio-podinfo\") pod \"bd8fbe72-d370-4949-9314-ff91cdab6ff6\" (UID: \"bd8fbe72-d370-4949-9314-ff91cdab6ff6\") "
Oct 30 22:31:15.818704 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:15.818583    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bd8fbe72-d370-4949-9314-ff91cdab6ff6-istio-token\" (UniqueName: \"kubernetes.io/projected/bd8fbe72-d370-4949-9314-ff91cdab6ff6-istio-token\") pod \"bd8fbe72-d370-4949-9314-ff91cdab6ff6\" (UID: \"bd8fbe72-d370-4949-9314-ff91cdab6ff6\") "
Oct 30 22:31:15.818704 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:15.818629    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bd8fbe72-d370-4949-9314-ff91cdab6ff6-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd8fbe72-d370-4949-9314-ff91cdab6ff6-workload-socket\") pod \"bd8fbe72-d370-4949-9314-ff91cdab6ff6\" (UID: \"bd8fbe72-d370-4949-9314-ff91cdab6ff6\") "
Oct 30 22:31:15.818704 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:15.818681    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bd8fbe72-d370-4949-9314-ff91cdab6ff6-kube-api-access-mfpzg\" (UniqueName: \"kubernetes.io/projected/bd8fbe72-d370-4949-9314-ff91cdab6ff6-kube-api-access-mfpzg\") pod \"bd8fbe72-d370-4949-9314-ff91cdab6ff6\" (UID: \"bd8fbe72-d370-4949-9314-ff91cdab6ff6\") "
Oct 30 22:31:15.818851 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:15.818704    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bd8fbe72-d370-4949-9314-ff91cdab6ff6-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd8fbe72-d370-4949-9314-ff91cdab6ff6-credential-socket\") pod \"bd8fbe72-d370-4949-9314-ff91cdab6ff6\" (UID: \"bd8fbe72-d370-4949-9314-ff91cdab6ff6\") "
Oct 30 22:31:15.818851 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:15.818769    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bd8fbe72-d370-4949-9314-ff91cdab6ff6-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bd8fbe72-d370-4949-9314-ff91cdab6ff6-istio-data\") pod \"bd8fbe72-d370-4949-9314-ff91cdab6ff6\" (UID: \"bd8fbe72-d370-4949-9314-ff91cdab6ff6\") "
Oct 30 22:31:15.818851 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:15.818820    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/bd8fbe72-d370-4949-9314-ff91cdab6ff6-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bd8fbe72-d370-4949-9314-ff91cdab6ff6-istiod-ca-cert\") pod \"bd8fbe72-d370-4949-9314-ff91cdab6ff6\" (UID: \"bd8fbe72-d370-4949-9314-ff91cdab6ff6\") "
Oct 30 22:31:15.819006 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:15.818932    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bd8fbe72-d370-4949-9314-ff91cdab6ff6-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bd8fbe72-d370-4949-9314-ff91cdab6ff6-istio-envoy\") pod \"bd8fbe72-d370-4949-9314-ff91cdab6ff6\" (UID: \"bd8fbe72-d370-4949-9314-ff91cdab6ff6\") "
Oct 30 22:31:15.819006 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:15.818991    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bd8fbe72-d370-4949-9314-ff91cdab6ff6-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bd8fbe72-d370-4949-9314-ff91cdab6ff6-workload-certs\") pod \"bd8fbe72-d370-4949-9314-ff91cdab6ff6\" (UID: \"bd8fbe72-d370-4949-9314-ff91cdab6ff6\") "
Oct 30 22:31:15.819107 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:15.819060    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd8fbe72-d370-4949-9314-ff91cdab6ff6-credential-socket" pod "bd8fbe72-d370-4949-9314-ff91cdab6ff6" (UID: "bd8fbe72-d370-4949-9314-ff91cdab6ff6"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:31:15.819171 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:15.819113    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd8fbe72-d370-4949-9314-ff91cdab6ff6-istio-data" pod "bd8fbe72-d370-4949-9314-ff91cdab6ff6" (UID: "bd8fbe72-d370-4949-9314-ff91cdab6ff6"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:31:15.819272 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:15.819243    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bd8fbe72-d370-4949-9314-ff91cdab6ff6-istiod-ca-cert" pod "bd8fbe72-d370-4949-9314-ff91cdab6ff6" (UID: "bd8fbe72-d370-4949-9314-ff91cdab6ff6"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:31:15.819432 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:15.819406    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd8fbe72-d370-4949-9314-ff91cdab6ff6-workload-certs" pod "bd8fbe72-d370-4949-9314-ff91cdab6ff6" (UID: "bd8fbe72-d370-4949-9314-ff91cdab6ff6"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:31:15.819889 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:15.819865    5510 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bd8fbe72-d370-4949-9314-ff91cdab6ff6-istio-data\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:31:15.820001 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:15.819894    5510 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bd8fbe72-d370-4949-9314-ff91cdab6ff6-istiod-ca-cert\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:31:15.820001 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:15.819910    5510 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bd8fbe72-d370-4949-9314-ff91cdab6ff6-workload-certs\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:31:15.820001 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:15.819927    5510 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd8fbe72-d370-4949-9314-ff91cdab6ff6-credential-socket\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:31:15.820187 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:15.820016    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd8fbe72-d370-4949-9314-ff91cdab6ff6-workload-socket" pod "bd8fbe72-d370-4949-9314-ff91cdab6ff6" (UID: "bd8fbe72-d370-4949-9314-ff91cdab6ff6"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:31:15.820935 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:15.820910    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/bd8fbe72-d370-4949-9314-ff91cdab6ff6-istio-podinfo" pod "bd8fbe72-d370-4949-9314-ff91cdab6ff6" (UID: "bd8fbe72-d370-4949-9314-ff91cdab6ff6"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:31:15.821025 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:15.820961    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bd8fbe72-d370-4949-9314-ff91cdab6ff6-kube-api-access-mfpzg" pod "bd8fbe72-d370-4949-9314-ff91cdab6ff6" (UID: "bd8fbe72-d370-4949-9314-ff91cdab6ff6"). InnerVolumeSpecName "kube-api-access-mfpzg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:31:15.821025 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:15.820991    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bd8fbe72-d370-4949-9314-ff91cdab6ff6-istio-token" pod "bd8fbe72-d370-4949-9314-ff91cdab6ff6" (UID: "bd8fbe72-d370-4949-9314-ff91cdab6ff6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:31:15.821103 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:15.821081    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd8fbe72-d370-4949-9314-ff91cdab6ff6-istio-envoy" pod "bd8fbe72-d370-4949-9314-ff91cdab6ff6" (UID: "bd8fbe72-d370-4949-9314-ff91cdab6ff6"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:31:15.911919 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:15.911879    5510 generic.go:356] "Generic (PLEG): container finished" podID="bd8fbe72-d370-4949-9314-ff91cdab6ff6" containerID="11632c267999841c4d2731b19a4d82fd58583e965ea9ca58b7dc3ac939125328" exitCode=0
Oct 30 22:31:15.912131 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:15.911937    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-v8m49"
Oct 30 22:31:15.912131 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:15.911975    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-v8m49" event={"ID":"bd8fbe72-d370-4949-9314-ff91cdab6ff6","Type":"ContainerDied","Data":"11632c267999841c4d2731b19a4d82fd58583e965ea9ca58b7dc3ac939125328"}
Oct 30 22:31:15.912131 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:15.912040    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-v8m49" event={"ID":"bd8fbe72-d370-4949-9314-ff91cdab6ff6","Type":"ContainerDied","Data":"8364e2239b982db9f7fb65650f6932d668b8b53a0fa70184b9c0a2c787fad8cc"}
Oct 30 22:31:15.912131 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:15.912071    5510 scope.go:122] "RemoveContainer" containerID="11632c267999841c4d2731b19a4d82fd58583e965ea9ca58b7dc3ac939125328"
Oct 30 22:31:15.921018 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:15.920997    5510 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bd8fbe72-d370-4949-9314-ff91cdab6ff6-istio-podinfo\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:31:15.921018 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:15.921018    5510 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bd8fbe72-d370-4949-9314-ff91cdab6ff6-istio-token\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:31:15.921137 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:15.921028    5510 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd8fbe72-d370-4949-9314-ff91cdab6ff6-workload-socket\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:31:15.921137 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:15.921037    5510 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mfpzg\" (UniqueName: \"kubernetes.io/projected/bd8fbe72-d370-4949-9314-ff91cdab6ff6-kube-api-access-mfpzg\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:31:15.921137 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:15.921047    5510 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bd8fbe72-d370-4949-9314-ff91cdab6ff6-istio-envoy\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:31:15.922569 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:15.922552    5510 scope.go:122] "RemoveContainer" containerID="11632c267999841c4d2731b19a4d82fd58583e965ea9ca58b7dc3ac939125328"
Oct 30 22:31:15.922864 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:31:15.922844    5510 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"11632c267999841c4d2731b19a4d82fd58583e965ea9ca58b7dc3ac939125328\": container with ID starting with 11632c267999841c4d2731b19a4d82fd58583e965ea9ca58b7dc3ac939125328 not found: ID does not exist" containerID="11632c267999841c4d2731b19a4d82fd58583e965ea9ca58b7dc3ac939125328"
Oct 30 22:31:15.922923 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:15.922876    5510 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"11632c267999841c4d2731b19a4d82fd58583e965ea9ca58b7dc3ac939125328"} err="failed to get container status \"11632c267999841c4d2731b19a4d82fd58583e965ea9ca58b7dc3ac939125328\": rpc error: code = NotFound desc = could not find container \"11632c267999841c4d2731b19a4d82fd58583e965ea9ca58b7dc3ac939125328\": container with ID starting with 11632c267999841c4d2731b19a4d82fd58583e965ea9ca58b7dc3ac939125328 not found: ID does not exist"
Oct 30 22:31:16.077737 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:16.077601    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-v8m49"]
Oct 30 22:31:16.080508 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:16.080475    5510 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-65e59f-openshift-default-596d74bb4f-v8m49"]
Oct 30 22:31:16.113912 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:16.113882    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w5-c9dbe6-584f9bc9cf-z2ph8"]
Oct 30 22:31:16.214343 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:16.214302    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w5-c9dbe6-6986bd68b7-2kg6w"]
Oct 30 22:31:16.514026 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:16.513943    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:31:16.708173 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:16.708135    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:31:17.553340 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:17.553307    5510 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="bd8fbe72-d370-4949-9314-ff91cdab6ff6" path="/var/lib/kubelet/pods/bd8fbe72-d370-4949-9314-ff91cdab6ff6/volumes"
Oct 30 22:31:17.595871 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:17.595827    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:31:21.114576 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:21.114534    5510 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w5-c9dbe6-584f9bc9cf-z2ph8" podUID="559d4d98-2fa9-41fe-b159-79a92477294a" containerName="payload-processing" containerID="cri-o://cb168005c0cd05e7265faea26dced6f5dd75278fc17617555e35eed68e0b5f14" gracePeriod=25
Oct 30 22:31:21.214680 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:21.214622    5510 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-c9dbe6-6986bd68b7-2kg6w" podUID="e3ff7b9f-1fe3-4b03-848e-298ac1105742" containerName="payload-pre-processing" containerID="cri-o://a196a415d9dbde8836727c9b005cccefefffd6c4b39b8e434225744bd08e2c32" gracePeriod=25
Oct 30 22:31:22.441133 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:22.440745    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:31:22.976327 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:22.976282    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:31:23.402624 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:23.402586    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:31:23.637802 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:23.637761    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:31:23.892459 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:23.892423    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:31:24.102023 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:24.101975    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:31:24.388942 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:24.388895    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:31:25.281716 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:25.281637    5510 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w5-c9dbe6/distinct-w5-c9dbe6-kserve-7599fd86d9-bdhxd" podUID="e903db90-f2cd-4244-b00b-f99b21873f6e" containerName="main" containerID="cri-o://7be64b84b2721816f365dba73b93a2aa877727429b64ea25017ce45976c2e14e" gracePeriod=45
Oct 30 22:31:25.578450 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:25.578425    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-c9dbe6/distinct-w5-c9dbe6-kserve-7599fd86d9-bdhxd"
Oct 30 22:31:25.687298 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:25.687262    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e903db90-f2cd-4244-b00b-f99b21873f6e-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e903db90-f2cd-4244-b00b-f99b21873f6e-model-cache\") pod \"e903db90-f2cd-4244-b00b-f99b21873f6e\" (UID: \"e903db90-f2cd-4244-b00b-f99b21873f6e\") "
Oct 30 22:31:25.687500 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:25.687345    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e903db90-f2cd-4244-b00b-f99b21873f6e-kube-api-access-vs5hl\" (UniqueName: \"kubernetes.io/projected/e903db90-f2cd-4244-b00b-f99b21873f6e-kube-api-access-vs5hl\") pod \"e903db90-f2cd-4244-b00b-f99b21873f6e\" (UID: \"e903db90-f2cd-4244-b00b-f99b21873f6e\") "
Oct 30 22:31:25.687500 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:25.687458    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e903db90-f2cd-4244-b00b-f99b21873f6e-home\" (UniqueName: \"kubernetes.io/empty-dir/e903db90-f2cd-4244-b00b-f99b21873f6e-home\") pod \"e903db90-f2cd-4244-b00b-f99b21873f6e\" (UID: \"e903db90-f2cd-4244-b00b-f99b21873f6e\") "
Oct 30 22:31:25.687500 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:25.687491    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e903db90-f2cd-4244-b00b-f99b21873f6e-dshm\" (UniqueName: \"kubernetes.io/empty-dir/e903db90-f2cd-4244-b00b-f99b21873f6e-dshm\") pod \"e903db90-f2cd-4244-b00b-f99b21873f6e\" (UID: \"e903db90-f2cd-4244-b00b-f99b21873f6e\") "
Oct 30 22:31:25.687715 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:25.687566    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e903db90-f2cd-4244-b00b-f99b21873f6e-tls-certs\" (UniqueName: \"kubernetes.io/secret/e903db90-f2cd-4244-b00b-f99b21873f6e-tls-certs\") pod \"e903db90-f2cd-4244-b00b-f99b21873f6e\" (UID: \"e903db90-f2cd-4244-b00b-f99b21873f6e\") "
Oct 30 22:31:25.687715 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:25.687574    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e903db90-f2cd-4244-b00b-f99b21873f6e-model-cache" pod "e903db90-f2cd-4244-b00b-f99b21873f6e" (UID: "e903db90-f2cd-4244-b00b-f99b21873f6e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:31:25.687843 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:25.687766    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e903db90-f2cd-4244-b00b-f99b21873f6e-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e903db90-f2cd-4244-b00b-f99b21873f6e-tmp-dir\") pod \"e903db90-f2cd-4244-b00b-f99b21873f6e\" (UID: \"e903db90-f2cd-4244-b00b-f99b21873f6e\") "
Oct 30 22:31:25.687896 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:25.687768    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e903db90-f2cd-4244-b00b-f99b21873f6e-home" pod "e903db90-f2cd-4244-b00b-f99b21873f6e" (UID: "e903db90-f2cd-4244-b00b-f99b21873f6e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:31:25.688030 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:25.688007    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e903db90-f2cd-4244-b00b-f99b21873f6e-tmp-dir" pod "e903db90-f2cd-4244-b00b-f99b21873f6e" (UID: "e903db90-f2cd-4244-b00b-f99b21873f6e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:31:25.688790 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:25.688763    5510 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e903db90-f2cd-4244-b00b-f99b21873f6e-model-cache\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:31:25.688790 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:25.688790    5510 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e903db90-f2cd-4244-b00b-f99b21873f6e-home\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:31:25.688983 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:25.688806    5510 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e903db90-f2cd-4244-b00b-f99b21873f6e-tmp-dir\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:31:25.689889 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:25.689858    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e903db90-f2cd-4244-b00b-f99b21873f6e-tls-certs" pod "e903db90-f2cd-4244-b00b-f99b21873f6e" (UID: "e903db90-f2cd-4244-b00b-f99b21873f6e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:31:25.690000 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:25.689885    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e903db90-f2cd-4244-b00b-f99b21873f6e-dshm" pod "e903db90-f2cd-4244-b00b-f99b21873f6e" (UID: "e903db90-f2cd-4244-b00b-f99b21873f6e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:31:25.690000 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:25.689947    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e903db90-f2cd-4244-b00b-f99b21873f6e-kube-api-access-vs5hl" pod "e903db90-f2cd-4244-b00b-f99b21873f6e" (UID: "e903db90-f2cd-4244-b00b-f99b21873f6e"). InnerVolumeSpecName "kube-api-access-vs5hl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:31:25.790056 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:25.790016    5510 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vs5hl\" (UniqueName: \"kubernetes.io/projected/e903db90-f2cd-4244-b00b-f99b21873f6e-kube-api-access-vs5hl\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:31:25.790056 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:25.790049    5510 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e903db90-f2cd-4244-b00b-f99b21873f6e-dshm\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:31:25.790056 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:25.790058    5510 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e903db90-f2cd-4244-b00b-f99b21873f6e-tls-certs\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:31:25.958375 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:25.958281    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-c9dbe6-6986bd68b7-2kg6w" podUID="e3ff7b9f-1fe3-4b03-848e-298ac1105742" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:31:25.958375 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:25.958311    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w5-c9dbe6-584f9bc9cf-z2ph8" podUID="559d4d98-2fa9-41fe-b159-79a92477294a" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:31:26.029038 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:26.029006    5510 generic.go:356] "Generic (PLEG): container finished" podID="e903db90-f2cd-4244-b00b-f99b21873f6e" containerID="7be64b84b2721816f365dba73b93a2aa877727429b64ea25017ce45976c2e14e" exitCode=0
Oct 30 22:31:26.029203 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:26.029048    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-c9dbe6/distinct-w5-c9dbe6-kserve-7599fd86d9-bdhxd"
Oct 30 22:31:26.029203 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:26.029067    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-c9dbe6/distinct-w5-c9dbe6-kserve-7599fd86d9-bdhxd" event={"ID":"e903db90-f2cd-4244-b00b-f99b21873f6e","Type":"ContainerDied","Data":"7be64b84b2721816f365dba73b93a2aa877727429b64ea25017ce45976c2e14e"}
Oct 30 22:31:26.029203 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:26.029103    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-c9dbe6/distinct-w5-c9dbe6-kserve-7599fd86d9-bdhxd" event={"ID":"e903db90-f2cd-4244-b00b-f99b21873f6e","Type":"ContainerDied","Data":"5f783695670c08a6944643d6667c213eb055e0fa8d5975604f56da6f3bb06f31"}
Oct 30 22:31:26.029203 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:26.029121    5510 scope.go:122] "RemoveContainer" containerID="7be64b84b2721816f365dba73b93a2aa877727429b64ea25017ce45976c2e14e"
Oct 30 22:31:26.044568 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:26.044542    5510 scope.go:122] "RemoveContainer" containerID="7be64b84b2721816f365dba73b93a2aa877727429b64ea25017ce45976c2e14e"
Oct 30 22:31:26.044920 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:31:26.044896    5510 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7be64b84b2721816f365dba73b93a2aa877727429b64ea25017ce45976c2e14e\": container with ID starting with 7be64b84b2721816f365dba73b93a2aa877727429b64ea25017ce45976c2e14e not found: ID does not exist" containerID="7be64b84b2721816f365dba73b93a2aa877727429b64ea25017ce45976c2e14e"
Oct 30 22:31:26.045028 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:26.044927    5510 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7be64b84b2721816f365dba73b93a2aa877727429b64ea25017ce45976c2e14e"} err="failed to get container status \"7be64b84b2721816f365dba73b93a2aa877727429b64ea25017ce45976c2e14e\": rpc error: code = NotFound desc = could not find container \"7be64b84b2721816f365dba73b93a2aa877727429b64ea25017ce45976c2e14e\": container with ID starting with 7be64b84b2721816f365dba73b93a2aa877727429b64ea25017ce45976c2e14e not found: ID does not exist"
Oct 30 22:31:26.185482 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:26.185444    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w5-c9dbe6/distinct-w5-c9dbe6-kserve-7599fd86d9-bdhxd"]
Oct 30 22:31:26.192566 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:26.192537    5510 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w5-c9dbe6/distinct-w5-c9dbe6-kserve-7599fd86d9-bdhxd"]
Oct 30 22:31:27.501088 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:27.501046    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:31:27.553803 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:27.553772    5510 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e903db90-f2cd-4244-b00b-f99b21873f6e" path="/var/lib/kubelet/pods/e903db90-f2cd-4244-b00b-f99b21873f6e/volumes"
Oct 30 22:31:28.154423 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:28.154378    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:31:28.399824 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:28.399791    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:31:31.315512 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:31.315472    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:31:31.561315 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:31.561281    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:31:32.489475 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:32.489447    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:31:32.708583 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:32.708540    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:31:35.958377 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:35.958339    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-c9dbe6-6986bd68b7-2kg6w" podUID="e3ff7b9f-1fe3-4b03-848e-298ac1105742" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:31:35.958858 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:35.958426    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w5-c9dbe6-584f9bc9cf-z2ph8" podUID="559d4d98-2fa9-41fe-b159-79a92477294a" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:31:37.282994 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:37.282951    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:31:38.421324 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:31:38.421287    5510 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6gjbf" containerName="extractor"
Oct 30 22:31:41.240437 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:41.239730    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:31:41.450982 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:41.450954    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-c9dbe6-6986bd68b7-2kg6w"
Oct 30 22:31:41.495463 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:41.495366    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w5-c9dbe6-584f9bc9cf-z2ph8"
Oct 30 22:31:41.559139 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:41.559108    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/559d4d98-2fa9-41fe-b159-79a92477294a-kube-api-access-7q72b\" (UniqueName: \"kubernetes.io/projected/559d4d98-2fa9-41fe-b159-79a92477294a-kube-api-access-7q72b\") pod \"559d4d98-2fa9-41fe-b159-79a92477294a\" (UID: \"559d4d98-2fa9-41fe-b159-79a92477294a\") "
Oct 30 22:31:41.559336 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:41.559325    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e3ff7b9f-1fe3-4b03-848e-298ac1105742-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e3ff7b9f-1fe3-4b03-848e-298ac1105742-plugins-config-volume\") pod \"e3ff7b9f-1fe3-4b03-848e-298ac1105742\" (UID: \"e3ff7b9f-1fe3-4b03-848e-298ac1105742\") "
Oct 30 22:31:41.559409 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:41.559387    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e3ff7b9f-1fe3-4b03-848e-298ac1105742-kube-api-access-zjll7\" (UniqueName: \"kubernetes.io/projected/e3ff7b9f-1fe3-4b03-848e-298ac1105742-kube-api-access-zjll7\") pod \"e3ff7b9f-1fe3-4b03-848e-298ac1105742\" (UID: \"e3ff7b9f-1fe3-4b03-848e-298ac1105742\") "
Oct 30 22:31:41.559477 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:41.559421    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/559d4d98-2fa9-41fe-b159-79a92477294a-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/559d4d98-2fa9-41fe-b159-79a92477294a-plugins-config-volume\") pod \"559d4d98-2fa9-41fe-b159-79a92477294a\" (UID: \"559d4d98-2fa9-41fe-b159-79a92477294a\") "
Oct 30 22:31:41.559817 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:41.559780    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e3ff7b9f-1fe3-4b03-848e-298ac1105742-plugins-config-volume" pod "e3ff7b9f-1fe3-4b03-848e-298ac1105742" (UID: "e3ff7b9f-1fe3-4b03-848e-298ac1105742"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:31:41.559951 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:41.559899    5510 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e3ff7b9f-1fe3-4b03-848e-298ac1105742-plugins-config-volume\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:31:41.559951 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:41.559893    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/559d4d98-2fa9-41fe-b159-79a92477294a-plugins-config-volume" pod "559d4d98-2fa9-41fe-b159-79a92477294a" (UID: "559d4d98-2fa9-41fe-b159-79a92477294a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:31:41.561315 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:41.561274    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/559d4d98-2fa9-41fe-b159-79a92477294a-kube-api-access-7q72b" pod "559d4d98-2fa9-41fe-b159-79a92477294a" (UID: "559d4d98-2fa9-41fe-b159-79a92477294a"). InnerVolumeSpecName "kube-api-access-7q72b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:31:41.561931 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:41.561907    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e3ff7b9f-1fe3-4b03-848e-298ac1105742-kube-api-access-zjll7" pod "e3ff7b9f-1fe3-4b03-848e-298ac1105742" (UID: "e3ff7b9f-1fe3-4b03-848e-298ac1105742"). InnerVolumeSpecName "kube-api-access-zjll7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:31:41.669091 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:41.666782    5510 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zjll7\" (UniqueName: \"kubernetes.io/projected/e3ff7b9f-1fe3-4b03-848e-298ac1105742-kube-api-access-zjll7\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:31:41.669091 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:41.666819    5510 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/559d4d98-2fa9-41fe-b159-79a92477294a-plugins-config-volume\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:31:41.669091 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:41.666835    5510 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7q72b\" (UniqueName: \"kubernetes.io/projected/559d4d98-2fa9-41fe-b159-79a92477294a-kube-api-access-7q72b\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:31:42.178349 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:42.178310    5510 generic.go:356] "Generic (PLEG): container finished" podID="e3ff7b9f-1fe3-4b03-848e-298ac1105742" containerID="a196a415d9dbde8836727c9b005cccefefffd6c4b39b8e434225744bd08e2c32" exitCode=0
Oct 30 22:31:42.178527 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:42.178359    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-c9dbe6-6986bd68b7-2kg6w"
Oct 30 22:31:42.178527 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:42.178392    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-c9dbe6-6986bd68b7-2kg6w" event={"ID":"e3ff7b9f-1fe3-4b03-848e-298ac1105742","Type":"ContainerDied","Data":"a196a415d9dbde8836727c9b005cccefefffd6c4b39b8e434225744bd08e2c32"}
Oct 30 22:31:42.178527 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:42.178444    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-c9dbe6-6986bd68b7-2kg6w" event={"ID":"e3ff7b9f-1fe3-4b03-848e-298ac1105742","Type":"ContainerDied","Data":"4808189a81b1eb93da96bf71101c6a912e16940a52bc5edbaf992c48ab0b6f1a"}
Oct 30 22:31:42.178527 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:42.178465    5510 scope.go:122] "RemoveContainer" containerID="a196a415d9dbde8836727c9b005cccefefffd6c4b39b8e434225744bd08e2c32"
Oct 30 22:31:42.180035 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:42.180014    5510 generic.go:356] "Generic (PLEG): container finished" podID="559d4d98-2fa9-41fe-b159-79a92477294a" containerID="cb168005c0cd05e7265faea26dced6f5dd75278fc17617555e35eed68e0b5f14" exitCode=0
Oct 30 22:31:42.180158 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:42.180045    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w5-c9dbe6-584f9bc9cf-z2ph8" event={"ID":"559d4d98-2fa9-41fe-b159-79a92477294a","Type":"ContainerDied","Data":"cb168005c0cd05e7265faea26dced6f5dd75278fc17617555e35eed68e0b5f14"}
Oct 30 22:31:42.180158 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:42.180065    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w5-c9dbe6-584f9bc9cf-z2ph8"
Oct 30 22:31:42.180158 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:42.180097    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w5-c9dbe6-584f9bc9cf-z2ph8" event={"ID":"559d4d98-2fa9-41fe-b159-79a92477294a","Type":"ContainerDied","Data":"4cd5c883fe296218a63ec6e6df97546d73aa08966f2c2d98e99d6b13506f4802"}
Oct 30 22:31:42.189774 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:42.189706    5510 scope.go:122] "RemoveContainer" containerID="a196a415d9dbde8836727c9b005cccefefffd6c4b39b8e434225744bd08e2c32"
Oct 30 22:31:42.190031 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:31:42.189996    5510 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a196a415d9dbde8836727c9b005cccefefffd6c4b39b8e434225744bd08e2c32\": container with ID starting with a196a415d9dbde8836727c9b005cccefefffd6c4b39b8e434225744bd08e2c32 not found: ID does not exist" containerID="a196a415d9dbde8836727c9b005cccefefffd6c4b39b8e434225744bd08e2c32"
Oct 30 22:31:42.190159 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:42.190039    5510 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a196a415d9dbde8836727c9b005cccefefffd6c4b39b8e434225744bd08e2c32"} err="failed to get container status \"a196a415d9dbde8836727c9b005cccefefffd6c4b39b8e434225744bd08e2c32\": rpc error: code = NotFound desc = could not find container \"a196a415d9dbde8836727c9b005cccefefffd6c4b39b8e434225744bd08e2c32\": container with ID starting with a196a415d9dbde8836727c9b005cccefefffd6c4b39b8e434225744bd08e2c32 not found: ID does not exist"
Oct 30 22:31:42.190159 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:42.190065    5510 scope.go:122] "RemoveContainer" containerID="cb168005c0cd05e7265faea26dced6f5dd75278fc17617555e35eed68e0b5f14"
Oct 30 22:31:42.198981 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:42.198959    5510 scope.go:122] "RemoveContainer" containerID="cb168005c0cd05e7265faea26dced6f5dd75278fc17617555e35eed68e0b5f14"
Oct 30 22:31:42.199235 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:31:42.199214    5510 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cb168005c0cd05e7265faea26dced6f5dd75278fc17617555e35eed68e0b5f14\": container with ID starting with cb168005c0cd05e7265faea26dced6f5dd75278fc17617555e35eed68e0b5f14 not found: ID does not exist" containerID="cb168005c0cd05e7265faea26dced6f5dd75278fc17617555e35eed68e0b5f14"
Oct 30 22:31:42.199295 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:42.199241    5510 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cb168005c0cd05e7265faea26dced6f5dd75278fc17617555e35eed68e0b5f14"} err="failed to get container status \"cb168005c0cd05e7265faea26dced6f5dd75278fc17617555e35eed68e0b5f14\": rpc error: code = NotFound desc = could not find container \"cb168005c0cd05e7265faea26dced6f5dd75278fc17617555e35eed68e0b5f14\": container with ID starting with cb168005c0cd05e7265faea26dced6f5dd75278fc17617555e35eed68e0b5f14 not found: ID does not exist"
Oct 30 22:31:42.337050 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:42.337014    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w5-c9dbe6-6986bd68b7-2kg6w"]
Oct 30 22:31:42.383466 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:42.383381    5510 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w5-c9dbe6-6986bd68b7-2kg6w"]
Oct 30 22:31:42.465379 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:42.465340    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w5-c9dbe6-584f9bc9cf-z2ph8"]
Oct 30 22:31:42.470995 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:42.470963    5510 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w5-c9dbe6-584f9bc9cf-z2ph8"]
Oct 30 22:31:42.780316 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:42.780286    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:31:43.130245 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:43.130156    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:31:43.330627 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:43.330591    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:31:43.554126 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:43.554095    5510 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="559d4d98-2fa9-41fe-b159-79a92477294a" path="/var/lib/kubelet/pods/559d4d98-2fa9-41fe-b159-79a92477294a/volumes"
Oct 30 22:31:43.554520 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:43.554514    5510 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e3ff7b9f-1fe3-4b03-848e-298ac1105742" path="/var/lib/kubelet/pods/e3ff7b9f-1fe3-4b03-848e-298ac1105742/volumes"
Oct 30 22:31:43.926396 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:43.926303    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:31:46.359691 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:46.359612    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:31:46.919457 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:46.919424    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:31:47.134107 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:47.134074    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:31:47.613095 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:47.613058    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:31:47.987373 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:47.987293    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:31:48.466081 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:48.466046    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:31:48.686338 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:48.686296    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:31:50.089752 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:50.089713    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:31:51.315461 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:51.315416    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:31:51.554938 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:51.554896    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:31:51.776505 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:51.776470    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:31:52.377705 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:52.377668    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:31:52.621595 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:52.621561    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:31:53.969684 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:53.968781    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:31:55.475176 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:55.475132    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:31:55.667194 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:55.667154    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:31:55.948688 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:55.945685    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:31:56.013578 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:56.013538    5510 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-506964-56d6b85d7f-2lbbz"]
Oct 30 22:31:56.014218 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:56.014200    5510 cpu_manager.go:414] "containerMap: removing container" podUID="eb812de4-a46c-41c5-adc7-7b325b1de62b" containerName="istio-proxy"
Oct 30 22:31:56.014218 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:56.014218    5510 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="eb812de4-a46c-41c5-adc7-7b325b1de62b" containerName="istio-proxy"
Oct 30 22:31:56.014366 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:56.014240    5510 cpu_manager.go:414] "containerMap: removing container" podUID="743f9a1e-c695-454d-823d-7939bfbb6aff" containerName="maas-api"
Oct 30 22:31:56.014366 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:56.014246    5510 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="743f9a1e-c695-454d-823d-7939bfbb6aff" containerName="maas-api"
Oct 30 22:31:56.014366 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:56.014258    5510 cpu_manager.go:414] "containerMap: removing container" podUID="11bc04c5-986f-47f9-93d1-0f803b775eb5" containerName="istio-proxy"
Oct 30 22:31:56.014366 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:56.014263    5510 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="11bc04c5-986f-47f9-93d1-0f803b775eb5" containerName="istio-proxy"
Oct 30 22:31:56.014366 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:56.014271    5510 cpu_manager.go:414] "containerMap: removing container" podUID="bd8fbe72-d370-4949-9314-ff91cdab6ff6" containerName="istio-proxy"
Oct 30 22:31:56.014366 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:56.014275    5510 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bd8fbe72-d370-4949-9314-ff91cdab6ff6" containerName="istio-proxy"
Oct 30 22:31:56.014366 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:56.014282    5510 cpu_manager.go:414] "containerMap: removing container" podUID="bc1e8851-bb44-4e8d-a9da-387642b1c23c" containerName="payload-processing"
Oct 30 22:31:56.014366 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:56.014286    5510 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bc1e8851-bb44-4e8d-a9da-387642b1c23c" containerName="payload-processing"
Oct 30 22:31:56.014366 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:56.014295    5510 cpu_manager.go:414] "containerMap: removing container" podUID="11bc04c5-986f-47f9-93d1-0f803b775eb5" containerName="istio-proxy"
Oct 30 22:31:56.014366 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:56.014300    5510 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="11bc04c5-986f-47f9-93d1-0f803b775eb5" containerName="istio-proxy"
Oct 30 22:31:56.014366 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:56.014307    5510 cpu_manager.go:414] "containerMap: removing container" podUID="c4152cef-f9cf-46c1-a9c4-78484ab58efe" containerName="istio-proxy"
Oct 30 22:31:56.014366 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:56.014312    5510 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c4152cef-f9cf-46c1-a9c4-78484ab58efe" containerName="istio-proxy"
Oct 30 22:31:56.014366 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:56.014320    5510 cpu_manager.go:414] "containerMap: removing container" podUID="e3ff7b9f-1fe3-4b03-848e-298ac1105742" containerName="payload-pre-processing"
Oct 30 22:31:56.014366 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:56.014325    5510 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e3ff7b9f-1fe3-4b03-848e-298ac1105742" containerName="payload-pre-processing"
Oct 30 22:31:56.014366 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:56.014335    5510 cpu_manager.go:414] "containerMap: removing container" podUID="e903db90-f2cd-4244-b00b-f99b21873f6e" containerName="main"
Oct 30 22:31:56.014366 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:56.014339    5510 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e903db90-f2cd-4244-b00b-f99b21873f6e" containerName="main"
Oct 30 22:31:56.014366 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:56.014348    5510 cpu_manager.go:414] "containerMap: removing container" podUID="abac65c3-1048-47a8-b307-c2e002e6f7d5" containerName="cleanup"
Oct 30 22:31:56.014366 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:56.014352    5510 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="abac65c3-1048-47a8-b307-c2e002e6f7d5" containerName="cleanup"
Oct 30 22:31:56.014366 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:56.014366    5510 cpu_manager.go:414] "containerMap: removing container" podUID="abac65c3-1048-47a8-b307-c2e002e6f7d5" containerName="cleanup"
Oct 30 22:31:56.014366 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:56.014370    5510 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="abac65c3-1048-47a8-b307-c2e002e6f7d5" containerName="cleanup"
Oct 30 22:31:56.014366 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:56.014376    5510 cpu_manager.go:414] "containerMap: removing container" podUID="32e3490e-853c-4d7b-8c67-4423a2b89be3" containerName="payload-pre-processing"
Oct 30 22:31:56.015329 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:56.014381    5510 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="32e3490e-853c-4d7b-8c67-4423a2b89be3" containerName="payload-pre-processing"
Oct 30 22:31:56.015329 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:56.014389    5510 cpu_manager.go:414] "containerMap: removing container" podUID="abac65c3-1048-47a8-b307-c2e002e6f7d5" containerName="cleanup"
Oct 30 22:31:56.015329 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:56.014394    5510 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="abac65c3-1048-47a8-b307-c2e002e6f7d5" containerName="cleanup"
Oct 30 22:31:56.015329 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:56.014402    5510 cpu_manager.go:414] "containerMap: removing container" podUID="559d4d98-2fa9-41fe-b159-79a92477294a" containerName="payload-processing"
Oct 30 22:31:56.015329 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:56.014406    5510 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="559d4d98-2fa9-41fe-b159-79a92477294a" containerName="payload-processing"
Oct 30 22:31:56.015329 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:56.014482    5510 memory_manager.go:365] "RemoveStaleState removing state" podUID="bd8fbe72-d370-4949-9314-ff91cdab6ff6" containerName="istio-proxy"
Oct 30 22:31:56.015329 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:56.014491    5510 memory_manager.go:365] "RemoveStaleState removing state" podUID="bc1e8851-bb44-4e8d-a9da-387642b1c23c" containerName="payload-processing"
Oct 30 22:31:56.015329 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:56.014500    5510 memory_manager.go:365] "RemoveStaleState removing state" podUID="11bc04c5-986f-47f9-93d1-0f803b775eb5" containerName="istio-proxy"
Oct 30 22:31:56.015329 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:56.014506    5510 memory_manager.go:365] "RemoveStaleState removing state" podUID="c4152cef-f9cf-46c1-a9c4-78484ab58efe" containerName="istio-proxy"
Oct 30 22:31:56.015329 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:56.014511    5510 memory_manager.go:365] "RemoveStaleState removing state" podUID="e3ff7b9f-1fe3-4b03-848e-298ac1105742" containerName="payload-pre-processing"
Oct 30 22:31:56.015329 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:56.014518    5510 memory_manager.go:365] "RemoveStaleState removing state" podUID="e903db90-f2cd-4244-b00b-f99b21873f6e" containerName="main"
Oct 30 22:31:56.015329 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:56.014524    5510 memory_manager.go:365] "RemoveStaleState removing state" podUID="abac65c3-1048-47a8-b307-c2e002e6f7d5" containerName="cleanup"
Oct 30 22:31:56.015329 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:56.014533    5510 memory_manager.go:365] "RemoveStaleState removing state" podUID="abac65c3-1048-47a8-b307-c2e002e6f7d5" containerName="cleanup"
Oct 30 22:31:56.015329 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:56.014538    5510 memory_manager.go:365] "RemoveStaleState removing state" podUID="c4152cef-f9cf-46c1-a9c4-78484ab58efe" containerName="istio-proxy"
Oct 30 22:31:56.015329 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:56.014544    5510 memory_manager.go:365] "RemoveStaleState removing state" podUID="32e3490e-853c-4d7b-8c67-4423a2b89be3" containerName="payload-pre-processing"
Oct 30 22:31:56.015329 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:56.014549    5510 memory_manager.go:365] "RemoveStaleState removing state" podUID="abac65c3-1048-47a8-b307-c2e002e6f7d5" containerName="cleanup"
Oct 30 22:31:56.015329 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:56.014556    5510 memory_manager.go:365] "RemoveStaleState removing state" podUID="559d4d98-2fa9-41fe-b159-79a92477294a" containerName="payload-processing"
Oct 30 22:31:56.015329 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:56.014562    5510 memory_manager.go:365] "RemoveStaleState removing state" podUID="eb812de4-a46c-41c5-adc7-7b325b1de62b" containerName="istio-proxy"
Oct 30 22:31:56.015329 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:56.014569    5510 memory_manager.go:365] "RemoveStaleState removing state" podUID="743f9a1e-c695-454d-823d-7939bfbb6aff" containerName="maas-api"
Oct 30 22:31:56.015329 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:56.014576    5510 memory_manager.go:365] "RemoveStaleState removing state" podUID="13760c22-8ba0-45ed-b2e4-4e23d5273901" containerName="istio-proxy"
Oct 30 22:31:56.025683 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:56.024006    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-506964-56d6b85d7f-2lbbz"
Oct 30 22:31:56.059770 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:56.059624    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d29d64dd-be94-4d3b-9888-3196c0c8b5f0-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-506964-56d6b85d7f-2lbbz\" (UID: \"d29d64dd-be94-4d3b-9888-3196c0c8b5f0\") " pod="openshift-ingress/payload-processing-e2e-inf-b-506964-56d6b85d7f-2lbbz"
Oct 30 22:31:56.059770 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:56.059757    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ddbjk\" (UniqueName: \"kubernetes.io/projected/d29d64dd-be94-4d3b-9888-3196c0c8b5f0-kube-api-access-ddbjk\") pod \"payload-processing-e2e-inf-b-506964-56d6b85d7f-2lbbz\" (UID: \"d29d64dd-be94-4d3b-9888-3196c0c8b5f0\") " pod="openshift-ingress/payload-processing-e2e-inf-b-506964-56d6b85d7f-2lbbz"
Oct 30 22:31:56.065392 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:56.065357    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-506964-56d6b85d7f-2lbbz"]
Oct 30 22:31:56.108362 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:56.108332    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-506964-dockercfg-6gjjr\""
Oct 30 22:31:56.108362 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:56.108332    5510 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-506964\""
Oct 30 22:31:56.161712 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:56.161648    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ddbjk\" (UniqueName: \"kubernetes.io/projected/d29d64dd-be94-4d3b-9888-3196c0c8b5f0-kube-api-access-ddbjk\") pod \"payload-processing-e2e-inf-b-506964-56d6b85d7f-2lbbz\" (UID: \"d29d64dd-be94-4d3b-9888-3196c0c8b5f0\") " pod="openshift-ingress/payload-processing-e2e-inf-b-506964-56d6b85d7f-2lbbz"
Oct 30 22:31:56.161898 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:56.161864    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d29d64dd-be94-4d3b-9888-3196c0c8b5f0-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-506964-56d6b85d7f-2lbbz\" (UID: \"d29d64dd-be94-4d3b-9888-3196c0c8b5f0\") " pod="openshift-ingress/payload-processing-e2e-inf-b-506964-56d6b85d7f-2lbbz"
Oct 30 22:31:56.162544 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:56.162524    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d29d64dd-be94-4d3b-9888-3196c0c8b5f0-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-506964-56d6b85d7f-2lbbz\" (UID: \"d29d64dd-be94-4d3b-9888-3196c0c8b5f0\") " pod="openshift-ingress/payload-processing-e2e-inf-b-506964-56d6b85d7f-2lbbz"
Oct 30 22:31:56.220434 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:56.220357    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ddbjk\" (UniqueName: \"kubernetes.io/projected/d29d64dd-be94-4d3b-9888-3196c0c8b5f0-kube-api-access-ddbjk\") pod \"payload-processing-e2e-inf-b-506964-56d6b85d7f-2lbbz\" (UID: \"d29d64dd-be94-4d3b-9888-3196c0c8b5f0\") " pod="openshift-ingress/payload-processing-e2e-inf-b-506964-56d6b85d7f-2lbbz"
Oct 30 22:31:56.354007 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:56.353969    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-506964-56d6b85d7f-2lbbz"
Oct 30 22:31:56.696985 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:56.696946    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:31:56.728073 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:56.728023    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-506964-56d6b85d7f-2lbbz"]
Oct 30 22:31:56.893251 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:56.893209    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:31:57.277630 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:57.277594    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-506964-56d6b85d7f-2lbbz" event={"ID":"d29d64dd-be94-4d3b-9888-3196c0c8b5f0","Type":"ContainerStarted","Data":"7f9881c346cbeb848a99b083ddd523952bf71f18ef64f1eeeee7af9061b6a6d0"}
Oct 30 22:31:57.277821 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:57.277645    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-506964-56d6b85d7f-2lbbz" event={"ID":"d29d64dd-be94-4d3b-9888-3196c0c8b5f0","Type":"ContainerStarted","Data":"97477b55143a11454f026c4dfe3210c2cffea005c349ce6982b15f8cdad34546"}
Oct 30 22:31:57.277821 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:57.277701    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-b-506964-56d6b85d7f-2lbbz"
Oct 30 22:31:57.389184 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:57.389134    5510 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-b-506964-56d6b85d7f-2lbbz" podStartSLOduration=2.3891185999999998 podStartE2EDuration="2.3891186s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:31:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:31:57.387313371 +0000 UTC m=+2497.948123376" watchObservedRunningTime="2026-10-30 22:31:57.3891186 +0000 UTC m=+2497.949928636"
Oct 30 22:31:58.716676 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:31:58.714444    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:32:03.292032 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:32:03.291995    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:32:03.684525 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:32:03.684384    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:32:03.878298 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:32:03.878257    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:32:04.744814 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:32:04.744774    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:32:04.984891 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:32:04.984855    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:32:07.897539 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:32:07.897500    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:32:08.297427 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:32:08.297396    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-b-506964-56d6b85d7f-2lbbz"
Oct 30 22:32:08.421807 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:32:08.421771    5510 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6gjbf" containerName="extractor"
Oct 30 22:32:12.507394 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:32:12.507357    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:32:16.462999 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:32:16.462964    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:32:17.914277 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:32:17.914237    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:32:18.258158 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:32:18.258071    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:32:18.452081 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:32:18.452048    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:32:18.870632 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:32:18.870588    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:32:22.925618 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:32:22.925583    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:32:23.160626 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:32:23.160594    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:32:23.361461 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:32:23.361421    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:32:25.333560 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:32:25.333521    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:32:25.624733 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:32:25.613684    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:32:25.661101 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:32:25.661067    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-clqsv"]
Oct 30 22:32:25.661348 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:32:25.661317    5510 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-clqsv" podUID="21557bdf-26e4-4a0c-ad9f-50dcc391616b" containerName="istio-proxy" containerID="cri-o://283c8194c84b50b132cb851bab4cd1a702243d948d2414f149b1ae16ce4ffbed" gracePeriod=30
Oct 30 22:32:25.810703 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:32:25.810639    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:32:25.935413 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:32:25.935332    5510 patch_prober.go:29] interesting pod/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-clqsv container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 22:32:25.935575 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:32:25.935405    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-clqsv" podUID="21557bdf-26e4-4a0c-ad9f-50dcc391616b" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 22:32:26.703571 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:32:26.703535    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:32:26.893013 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:32:26.892974    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:32:27.759383 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:32:27.759346    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:32:30.921125 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:32:30.921097    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-clqsv"
Oct 30 22:32:31.068536 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:32:31.068502    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/21557bdf-26e4-4a0c-ad9f-50dcc391616b-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/21557bdf-26e4-4a0c-ad9f-50dcc391616b-istio-data\") pod \"21557bdf-26e4-4a0c-ad9f-50dcc391616b\" (UID: \"21557bdf-26e4-4a0c-ad9f-50dcc391616b\") "
Oct 30 22:32:31.068755 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:32:31.068585    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/21557bdf-26e4-4a0c-ad9f-50dcc391616b-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/21557bdf-26e4-4a0c-ad9f-50dcc391616b-istio-envoy\") pod \"21557bdf-26e4-4a0c-ad9f-50dcc391616b\" (UID: \"21557bdf-26e4-4a0c-ad9f-50dcc391616b\") "
Oct 30 22:32:31.068755 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:32:31.068641    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/21557bdf-26e4-4a0c-ad9f-50dcc391616b-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/21557bdf-26e4-4a0c-ad9f-50dcc391616b-workload-certs\") pod \"21557bdf-26e4-4a0c-ad9f-50dcc391616b\" (UID: \"21557bdf-26e4-4a0c-ad9f-50dcc391616b\") "
Oct 30 22:32:31.068755 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:32:31.068718    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/21557bdf-26e4-4a0c-ad9f-50dcc391616b-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/21557bdf-26e4-4a0c-ad9f-50dcc391616b-credential-socket\") pod \"21557bdf-26e4-4a0c-ad9f-50dcc391616b\" (UID: \"21557bdf-26e4-4a0c-ad9f-50dcc391616b\") "
Oct 30 22:32:31.069020 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:32:31.068759    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/21557bdf-26e4-4a0c-ad9f-50dcc391616b-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/21557bdf-26e4-4a0c-ad9f-50dcc391616b-istiod-ca-cert\") pod \"21557bdf-26e4-4a0c-ad9f-50dcc391616b\" (UID: \"21557bdf-26e4-4a0c-ad9f-50dcc391616b\") "
Oct 30 22:32:31.069020 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:32:31.068791    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/21557bdf-26e4-4a0c-ad9f-50dcc391616b-istio-token\" (UniqueName: \"kubernetes.io/projected/21557bdf-26e4-4a0c-ad9f-50dcc391616b-istio-token\") pod \"21557bdf-26e4-4a0c-ad9f-50dcc391616b\" (UID: \"21557bdf-26e4-4a0c-ad9f-50dcc391616b\") "
Oct 30 22:32:31.069020 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:32:31.068809    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/21557bdf-26e4-4a0c-ad9f-50dcc391616b-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/21557bdf-26e4-4a0c-ad9f-50dcc391616b-workload-socket\") pod \"21557bdf-26e4-4a0c-ad9f-50dcc391616b\" (UID: \"21557bdf-26e4-4a0c-ad9f-50dcc391616b\") "
Oct 30 22:32:31.069020 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:32:31.068839    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/21557bdf-26e4-4a0c-ad9f-50dcc391616b-istio-data" pod "21557bdf-26e4-4a0c-ad9f-50dcc391616b" (UID: "21557bdf-26e4-4a0c-ad9f-50dcc391616b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:32:31.069020 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:32:31.068901    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/21557bdf-26e4-4a0c-ad9f-50dcc391616b-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/21557bdf-26e4-4a0c-ad9f-50dcc391616b-istio-podinfo\") pod \"21557bdf-26e4-4a0c-ad9f-50dcc391616b\" (UID: \"21557bdf-26e4-4a0c-ad9f-50dcc391616b\") "
Oct 30 22:32:31.069020 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:32:31.068962    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/21557bdf-26e4-4a0c-ad9f-50dcc391616b-kube-api-access-kzxw8\" (UniqueName: \"kubernetes.io/projected/21557bdf-26e4-4a0c-ad9f-50dcc391616b-kube-api-access-kzxw8\") pod \"21557bdf-26e4-4a0c-ad9f-50dcc391616b\" (UID: \"21557bdf-26e4-4a0c-ad9f-50dcc391616b\") "
Oct 30 22:32:31.069333 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:32:31.069020    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/21557bdf-26e4-4a0c-ad9f-50dcc391616b-workload-certs" pod "21557bdf-26e4-4a0c-ad9f-50dcc391616b" (UID: "21557bdf-26e4-4a0c-ad9f-50dcc391616b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:32:31.069333 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:32:31.069035    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/21557bdf-26e4-4a0c-ad9f-50dcc391616b-credential-socket" pod "21557bdf-26e4-4a0c-ad9f-50dcc391616b" (UID: "21557bdf-26e4-4a0c-ad9f-50dcc391616b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:32:31.069333 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:32:31.069249    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/21557bdf-26e4-4a0c-ad9f-50dcc391616b-istiod-ca-cert" pod "21557bdf-26e4-4a0c-ad9f-50dcc391616b" (UID: "21557bdf-26e4-4a0c-ad9f-50dcc391616b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:32:31.069333 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:32:31.069255    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/21557bdf-26e4-4a0c-ad9f-50dcc391616b-workload-socket" pod "21557bdf-26e4-4a0c-ad9f-50dcc391616b" (UID: "21557bdf-26e4-4a0c-ad9f-50dcc391616b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:32:31.069549 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:32:31.069481    5510 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/21557bdf-26e4-4a0c-ad9f-50dcc391616b-istio-data\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:32:31.069549 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:32:31.069510    5510 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/21557bdf-26e4-4a0c-ad9f-50dcc391616b-workload-certs\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:32:31.069549 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:32:31.069527    5510 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/21557bdf-26e4-4a0c-ad9f-50dcc391616b-credential-socket\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:32:31.069549 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:32:31.069540    5510 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/21557bdf-26e4-4a0c-ad9f-50dcc391616b-istiod-ca-cert\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:32:31.069809 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:32:31.069554    5510 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/21557bdf-26e4-4a0c-ad9f-50dcc391616b-workload-socket\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:32:31.071081 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:32:31.071056    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/21557bdf-26e4-4a0c-ad9f-50dcc391616b-istio-envoy" pod "21557bdf-26e4-4a0c-ad9f-50dcc391616b" (UID: "21557bdf-26e4-4a0c-ad9f-50dcc391616b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:32:31.071196 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:32:31.071089    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/21557bdf-26e4-4a0c-ad9f-50dcc391616b-istio-token" pod "21557bdf-26e4-4a0c-ad9f-50dcc391616b" (UID: "21557bdf-26e4-4a0c-ad9f-50dcc391616b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:32:31.071240 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:32:31.071190    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/21557bdf-26e4-4a0c-ad9f-50dcc391616b-istio-podinfo" pod "21557bdf-26e4-4a0c-ad9f-50dcc391616b" (UID: "21557bdf-26e4-4a0c-ad9f-50dcc391616b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:32:31.071373 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:32:31.071353    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/21557bdf-26e4-4a0c-ad9f-50dcc391616b-kube-api-access-kzxw8" pod "21557bdf-26e4-4a0c-ad9f-50dcc391616b" (UID: "21557bdf-26e4-4a0c-ad9f-50dcc391616b"). InnerVolumeSpecName "kube-api-access-kzxw8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:32:31.170491 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:32:31.170448    5510 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/21557bdf-26e4-4a0c-ad9f-50dcc391616b-istio-token\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:32:31.170491 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:32:31.170482    5510 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/21557bdf-26e4-4a0c-ad9f-50dcc391616b-istio-podinfo\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:32:31.170491 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:32:31.170496    5510 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kzxw8\" (UniqueName: \"kubernetes.io/projected/21557bdf-26e4-4a0c-ad9f-50dcc391616b-kube-api-access-kzxw8\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:32:31.170755 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:32:31.170510    5510 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/21557bdf-26e4-4a0c-ad9f-50dcc391616b-istio-envoy\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:32:31.510533 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:32:31.510445    5510 generic.go:356] "Generic (PLEG): container finished" podID="21557bdf-26e4-4a0c-ad9f-50dcc391616b" containerID="283c8194c84b50b132cb851bab4cd1a702243d948d2414f149b1ae16ce4ffbed" exitCode=0
Oct 30 22:32:31.510533 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:32:31.510502    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-clqsv"
Oct 30 22:32:31.510814 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:32:31.510538    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-clqsv" event={"ID":"21557bdf-26e4-4a0c-ad9f-50dcc391616b","Type":"ContainerDied","Data":"283c8194c84b50b132cb851bab4cd1a702243d948d2414f149b1ae16ce4ffbed"}
Oct 30 22:32:31.510814 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:32:31.510598    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-clqsv" event={"ID":"21557bdf-26e4-4a0c-ad9f-50dcc391616b","Type":"ContainerDied","Data":"7388e36da4856c4db671cfdab01bd92ca1782277e174b908d17b435149bbdbc2"}
Oct 30 22:32:31.510814 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:32:31.510624    5510 scope.go:122] "RemoveContainer" containerID="283c8194c84b50b132cb851bab4cd1a702243d948d2414f149b1ae16ce4ffbed"
Oct 30 22:32:31.521851 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:32:31.521832    5510 scope.go:122] "RemoveContainer" containerID="f3a4ecf277b974e0f83bff4dbe835374e4b4ebe28697c46ac83f94aba5fa0924"
Oct 30 22:32:31.530665 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:32:31.530638    5510 scope.go:122] "RemoveContainer" containerID="283c8194c84b50b132cb851bab4cd1a702243d948d2414f149b1ae16ce4ffbed"
Oct 30 22:32:31.530931 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:32:31.530910    5510 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"283c8194c84b50b132cb851bab4cd1a702243d948d2414f149b1ae16ce4ffbed\": container with ID starting with 283c8194c84b50b132cb851bab4cd1a702243d948d2414f149b1ae16ce4ffbed not found: ID does not exist" containerID="283c8194c84b50b132cb851bab4cd1a702243d948d2414f149b1ae16ce4ffbed"
Oct 30 22:32:31.531010 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:32:31.530946    5510 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"283c8194c84b50b132cb851bab4cd1a702243d948d2414f149b1ae16ce4ffbed"} err="failed to get container status \"283c8194c84b50b132cb851bab4cd1a702243d948d2414f149b1ae16ce4ffbed\": rpc error: code = NotFound desc = could not find container \"283c8194c84b50b132cb851bab4cd1a702243d948d2414f149b1ae16ce4ffbed\": container with ID starting with 283c8194c84b50b132cb851bab4cd1a702243d948d2414f149b1ae16ce4ffbed not found: ID does not exist"
Oct 30 22:32:31.531010 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:32:31.530974    5510 scope.go:122] "RemoveContainer" containerID="f3a4ecf277b974e0f83bff4dbe835374e4b4ebe28697c46ac83f94aba5fa0924"
Oct 30 22:32:31.531251 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:32:31.531236    5510 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f3a4ecf277b974e0f83bff4dbe835374e4b4ebe28697c46ac83f94aba5fa0924\": container with ID starting with f3a4ecf277b974e0f83bff4dbe835374e4b4ebe28697c46ac83f94aba5fa0924 not found: ID does not exist" containerID="f3a4ecf277b974e0f83bff4dbe835374e4b4ebe28697c46ac83f94aba5fa0924"
Oct 30 22:32:31.531295 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:32:31.531257    5510 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f3a4ecf277b974e0f83bff4dbe835374e4b4ebe28697c46ac83f94aba5fa0924"} err="failed to get container status \"f3a4ecf277b974e0f83bff4dbe835374e4b4ebe28697c46ac83f94aba5fa0924\": rpc error: code = NotFound desc = could not find container \"f3a4ecf277b974e0f83bff4dbe835374e4b4ebe28697c46ac83f94aba5fa0924\": container with ID starting with f3a4ecf277b974e0f83bff4dbe835374e4b4ebe28697c46ac83f94aba5fa0924 not found: ID does not exist"
Oct 30 22:32:31.680358 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:32:31.680327    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-clqsv"]
Oct 30 22:32:31.726626 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:32:31.726586    5510 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-c9dbe6-openshift-default-59bd48d5d5-clqsv"]
Oct 30 22:32:32.294535 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:32:32.294503    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:32:32.550905 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:32:32.550820    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:32:32.866460 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:32:32.866367    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:32:33.558770 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:32:33.558733    5510 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="21557bdf-26e4-4a0c-ad9f-50dcc391616b" path="/var/lib/kubelet/pods/21557bdf-26e4-4a0c-ad9f-50dcc391616b/volumes"
Oct 30 22:32:37.293636 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:32:37.293597    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-bc714d/distinct-w2-bc714d-kserve-66cfb968dd-jvgqj"]
Oct 30 22:32:37.470281 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:32:37.470242    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:32:37.681914 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:32:37.681823    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:32:38.420874 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:32:38.420837    5510 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6gjbf" containerName="extractor"
Oct 30 22:32:42.423107 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:32:42.423070    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:32:43.809909 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:32:43.809871    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:32:44.081768 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:32:44.081673    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:32:47.227439 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:32:47.227399    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:32:51.897217 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:32:51.897183    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:32:52.173159 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:32:52.173075    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:32:52.314734 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:32:52.314682    5510 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w2-bc714d/distinct-w2-bc714d-kserve-66cfb968dd-jvgqj" podUID="9e9b11b8-1cc9-4fc8-8097-1afcffa8c422" containerName="main" containerID="cri-o://f70ce9298fd0c8b101c2375c907e006433e5cd428a64b0ac937ffc099299bf84" gracePeriod=45
Oct 30 22:32:52.342559 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:32:52.342525    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-bc714d/distinct-w2-bc714d-kserve-66cfb968dd-jvgqj" podUID="9e9b11b8-1cc9-4fc8-8097-1afcffa8c422" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": dial tcp 10.133.0.64:8000: connect: connection refused"
Oct 30 22:32:52.584193 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:32:52.584163    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-bc714d/distinct-w2-bc714d-kserve-66cfb968dd-jvgqj"
Oct 30 22:32:52.647302 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:32:52.647237    5510 generic.go:356] "Generic (PLEG): container finished" podID="9e9b11b8-1cc9-4fc8-8097-1afcffa8c422" containerID="f70ce9298fd0c8b101c2375c907e006433e5cd428a64b0ac937ffc099299bf84" exitCode=0
Oct 30 22:32:52.647302 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:32:52.647288    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-bc714d/distinct-w2-bc714d-kserve-66cfb968dd-jvgqj"
Oct 30 22:32:52.647519 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:32:52.647317    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-bc714d/distinct-w2-bc714d-kserve-66cfb968dd-jvgqj" event={"ID":"9e9b11b8-1cc9-4fc8-8097-1afcffa8c422","Type":"ContainerDied","Data":"f70ce9298fd0c8b101c2375c907e006433e5cd428a64b0ac937ffc099299bf84"}
Oct 30 22:32:52.647519 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:32:52.647370    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-bc714d/distinct-w2-bc714d-kserve-66cfb968dd-jvgqj" event={"ID":"9e9b11b8-1cc9-4fc8-8097-1afcffa8c422","Type":"ContainerDied","Data":"cddc7762e6edd87a2f3cbf01401782f9d43711dfe340c1d7e34b311acc293fb6"}
Oct 30 22:32:52.647519 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:32:52.647388    5510 scope.go:122] "RemoveContainer" containerID="f70ce9298fd0c8b101c2375c907e006433e5cd428a64b0ac937ffc099299bf84"
Oct 30 22:32:52.656743 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:32:52.656723    5510 scope.go:122] "RemoveContainer" containerID="f70ce9298fd0c8b101c2375c907e006433e5cd428a64b0ac937ffc099299bf84"
Oct 30 22:32:52.657032 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:32:52.657004    5510 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f70ce9298fd0c8b101c2375c907e006433e5cd428a64b0ac937ffc099299bf84\": container with ID starting with f70ce9298fd0c8b101c2375c907e006433e5cd428a64b0ac937ffc099299bf84 not found: ID does not exist" containerID="f70ce9298fd0c8b101c2375c907e006433e5cd428a64b0ac937ffc099299bf84"
Oct 30 22:32:52.657115 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:32:52.657041    5510 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f70ce9298fd0c8b101c2375c907e006433e5cd428a64b0ac937ffc099299bf84"} err="failed to get container status \"f70ce9298fd0c8b101c2375c907e006433e5cd428a64b0ac937ffc099299bf84\": rpc error: code = NotFound desc = could not find container \"f70ce9298fd0c8b101c2375c907e006433e5cd428a64b0ac937ffc099299bf84\": container with ID starting with f70ce9298fd0c8b101c2375c907e006433e5cd428a64b0ac937ffc099299bf84 not found: ID does not exist"
Oct 30 22:32:52.662422 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:32:52.662398    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9e9b11b8-1cc9-4fc8-8097-1afcffa8c422-home\" (UniqueName: \"kubernetes.io/empty-dir/9e9b11b8-1cc9-4fc8-8097-1afcffa8c422-home\") pod \"9e9b11b8-1cc9-4fc8-8097-1afcffa8c422\" (UID: \"9e9b11b8-1cc9-4fc8-8097-1afcffa8c422\") "
Oct 30 22:32:52.662529 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:32:52.662502    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9e9b11b8-1cc9-4fc8-8097-1afcffa8c422-dshm\" (UniqueName: \"kubernetes.io/empty-dir/9e9b11b8-1cc9-4fc8-8097-1afcffa8c422-dshm\") pod \"9e9b11b8-1cc9-4fc8-8097-1afcffa8c422\" (UID: \"9e9b11b8-1cc9-4fc8-8097-1afcffa8c422\") "
Oct 30 22:32:52.662646 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:32:52.662627    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9e9b11b8-1cc9-4fc8-8097-1afcffa8c422-home" pod "9e9b11b8-1cc9-4fc8-8097-1afcffa8c422" (UID: "9e9b11b8-1cc9-4fc8-8097-1afcffa8c422"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:32:52.662746 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:32:52.662729    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9e9b11b8-1cc9-4fc8-8097-1afcffa8c422-kube-api-access-g5h9s\" (UniqueName: \"kubernetes.io/projected/9e9b11b8-1cc9-4fc8-8097-1afcffa8c422-kube-api-access-g5h9s\") pod \"9e9b11b8-1cc9-4fc8-8097-1afcffa8c422\" (UID: \"9e9b11b8-1cc9-4fc8-8097-1afcffa8c422\") "
Oct 30 22:32:52.662811 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:32:52.662783    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9e9b11b8-1cc9-4fc8-8097-1afcffa8c422-tls-certs\" (UniqueName: \"kubernetes.io/secret/9e9b11b8-1cc9-4fc8-8097-1afcffa8c422-tls-certs\") pod \"9e9b11b8-1cc9-4fc8-8097-1afcffa8c422\" (UID: \"9e9b11b8-1cc9-4fc8-8097-1afcffa8c422\") "
Oct 30 22:32:52.662870 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:32:52.662848    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9e9b11b8-1cc9-4fc8-8097-1afcffa8c422-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9e9b11b8-1cc9-4fc8-8097-1afcffa8c422-tmp-dir\") pod \"9e9b11b8-1cc9-4fc8-8097-1afcffa8c422\" (UID: \"9e9b11b8-1cc9-4fc8-8097-1afcffa8c422\") "
Oct 30 22:32:52.662939 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:32:52.662918    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9e9b11b8-1cc9-4fc8-8097-1afcffa8c422-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9e9b11b8-1cc9-4fc8-8097-1afcffa8c422-model-cache\") pod \"9e9b11b8-1cc9-4fc8-8097-1afcffa8c422\" (UID: \"9e9b11b8-1cc9-4fc8-8097-1afcffa8c422\") "
Oct 30 22:32:52.663217 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:32:52.663176    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9e9b11b8-1cc9-4fc8-8097-1afcffa8c422-tmp-dir" pod "9e9b11b8-1cc9-4fc8-8097-1afcffa8c422" (UID: "9e9b11b8-1cc9-4fc8-8097-1afcffa8c422"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:32:52.663305 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:32:52.663242    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9e9b11b8-1cc9-4fc8-8097-1afcffa8c422-model-cache" pod "9e9b11b8-1cc9-4fc8-8097-1afcffa8c422" (UID: "9e9b11b8-1cc9-4fc8-8097-1afcffa8c422"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:32:52.663488 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:32:52.663470    5510 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9e9b11b8-1cc9-4fc8-8097-1afcffa8c422-tmp-dir\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:32:52.663540 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:32:52.663499    5510 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9e9b11b8-1cc9-4fc8-8097-1afcffa8c422-model-cache\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:32:52.663540 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:32:52.663516    5510 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9e9b11b8-1cc9-4fc8-8097-1afcffa8c422-home\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:32:52.664901 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:32:52.664872    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9e9b11b8-1cc9-4fc8-8097-1afcffa8c422-tls-certs" pod "9e9b11b8-1cc9-4fc8-8097-1afcffa8c422" (UID: "9e9b11b8-1cc9-4fc8-8097-1afcffa8c422"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:32:52.664987 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:32:52.664912    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9e9b11b8-1cc9-4fc8-8097-1afcffa8c422-dshm" pod "9e9b11b8-1cc9-4fc8-8097-1afcffa8c422" (UID: "9e9b11b8-1cc9-4fc8-8097-1afcffa8c422"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:32:52.664987 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:32:52.664914    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9e9b11b8-1cc9-4fc8-8097-1afcffa8c422-kube-api-access-g5h9s" pod "9e9b11b8-1cc9-4fc8-8097-1afcffa8c422" (UID: "9e9b11b8-1cc9-4fc8-8097-1afcffa8c422"). InnerVolumeSpecName "kube-api-access-g5h9s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:32:52.765032 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:32:52.764940    5510 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g5h9s\" (UniqueName: \"kubernetes.io/projected/9e9b11b8-1cc9-4fc8-8097-1afcffa8c422-kube-api-access-g5h9s\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:32:52.765032 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:32:52.764983    5510 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9e9b11b8-1cc9-4fc8-8097-1afcffa8c422-tls-certs\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:32:52.765032 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:32:52.765000    5510 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9e9b11b8-1cc9-4fc8-8097-1afcffa8c422-dshm\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:32:53.074829 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:32:53.074790    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-bc714d/distinct-w2-bc714d-kserve-66cfb968dd-jvgqj"]
Oct 30 22:32:53.079284 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:32:53.079257    5510 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w2-bc714d/distinct-w2-bc714d-kserve-66cfb968dd-jvgqj"]
Oct 30 22:32:53.554531 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:32:53.554497    5510 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9e9b11b8-1cc9-4fc8-8097-1afcffa8c422" path="/var/lib/kubelet/pods/9e9b11b8-1cc9-4fc8-8097-1afcffa8c422/volumes"
Oct 30 22:32:55.936650 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:32:55.936612    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-c42c0a-8545457dcf-ghrwb"]
Oct 30 22:32:55.937156 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:32:55.936852    5510 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-c42c0a-8545457dcf-ghrwb" podUID="42db3ad0-7d90-4073-9ad9-19b6b5590551" containerName="maas-api" containerID="cri-o://20947ec152a579120ffc74a77d841e88e7990da4c29221f4bfe0994ac2ef78bf" gracePeriod=30
Oct 30 22:32:56.086471 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:32:56.086434    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:32:56.236620 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:32:56.236590    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-c42c0a-8545457dcf-ghrwb"
Oct 30 22:32:56.423268 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:32:56.423235    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/42db3ad0-7d90-4073-9ad9-19b6b5590551-metrics-certs\" (UniqueName: \"kubernetes.io/secret/42db3ad0-7d90-4073-9ad9-19b6b5590551-metrics-certs\") pod \"42db3ad0-7d90-4073-9ad9-19b6b5590551\" (UID: \"42db3ad0-7d90-4073-9ad9-19b6b5590551\") "
Oct 30 22:32:56.423440 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:32:56.423373    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/42db3ad0-7d90-4073-9ad9-19b6b5590551-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/42db3ad0-7d90-4073-9ad9-19b6b5590551-openshift-service-ca\") pod \"42db3ad0-7d90-4073-9ad9-19b6b5590551\" (UID: \"42db3ad0-7d90-4073-9ad9-19b6b5590551\") "
Oct 30 22:32:56.423537 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:32:56.423522    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/42db3ad0-7d90-4073-9ad9-19b6b5590551-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/42db3ad0-7d90-4073-9ad9-19b6b5590551-maas-api-tls\") pod \"42db3ad0-7d90-4073-9ad9-19b6b5590551\" (UID: \"42db3ad0-7d90-4073-9ad9-19b6b5590551\") "
Oct 30 22:32:56.423575 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:32:56.423558    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/42db3ad0-7d90-4073-9ad9-19b6b5590551-kube-api-access-bqsfn\" (UniqueName: \"kubernetes.io/projected/42db3ad0-7d90-4073-9ad9-19b6b5590551-kube-api-access-bqsfn\") pod \"42db3ad0-7d90-4073-9ad9-19b6b5590551\" (UID: \"42db3ad0-7d90-4073-9ad9-19b6b5590551\") "
Oct 30 22:32:56.423781 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:32:56.423749    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/42db3ad0-7d90-4073-9ad9-19b6b5590551-openshift-service-ca" pod "42db3ad0-7d90-4073-9ad9-19b6b5590551" (UID: "42db3ad0-7d90-4073-9ad9-19b6b5590551"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:32:56.424096 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:32:56.424075    5510 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/42db3ad0-7d90-4073-9ad9-19b6b5590551-openshift-service-ca\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:32:56.425539 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:32:56.425508    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/42db3ad0-7d90-4073-9ad9-19b6b5590551-metrics-certs" pod "42db3ad0-7d90-4073-9ad9-19b6b5590551" (UID: "42db3ad0-7d90-4073-9ad9-19b6b5590551"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:32:56.425698 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:32:56.425648    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/42db3ad0-7d90-4073-9ad9-19b6b5590551-maas-api-tls" pod "42db3ad0-7d90-4073-9ad9-19b6b5590551" (UID: "42db3ad0-7d90-4073-9ad9-19b6b5590551"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:32:56.425752 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:32:56.425720    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/42db3ad0-7d90-4073-9ad9-19b6b5590551-kube-api-access-bqsfn" pod "42db3ad0-7d90-4073-9ad9-19b6b5590551" (UID: "42db3ad0-7d90-4073-9ad9-19b6b5590551"). InnerVolumeSpecName "kube-api-access-bqsfn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:32:56.525472 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:32:56.525438    5510 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/42db3ad0-7d90-4073-9ad9-19b6b5590551-maas-api-tls\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:32:56.525472 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:32:56.525471    5510 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bqsfn\" (UniqueName: \"kubernetes.io/projected/42db3ad0-7d90-4073-9ad9-19b6b5590551-kube-api-access-bqsfn\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:32:56.525723 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:32:56.525485    5510 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/42db3ad0-7d90-4073-9ad9-19b6b5590551-metrics-certs\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:32:56.673310 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:32:56.673269    5510 generic.go:356] "Generic (PLEG): container finished" podID="42db3ad0-7d90-4073-9ad9-19b6b5590551" containerID="20947ec152a579120ffc74a77d841e88e7990da4c29221f4bfe0994ac2ef78bf" exitCode=0
Oct 30 22:32:56.673454 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:32:56.673319    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-c42c0a-8545457dcf-ghrwb"
Oct 30 22:32:56.673454 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:32:56.673342    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-c42c0a-8545457dcf-ghrwb" event={"ID":"42db3ad0-7d90-4073-9ad9-19b6b5590551","Type":"ContainerDied","Data":"20947ec152a579120ffc74a77d841e88e7990da4c29221f4bfe0994ac2ef78bf"}
Oct 30 22:32:56.673454 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:32:56.673388    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-c42c0a-8545457dcf-ghrwb" event={"ID":"42db3ad0-7d90-4073-9ad9-19b6b5590551","Type":"ContainerDied","Data":"66bb191d3d5035ced3affb6aaacb96d2dd24840adfe561a620eff06240ff4d06"}
Oct 30 22:32:56.673454 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:32:56.673411    5510 scope.go:122] "RemoveContainer" containerID="20947ec152a579120ffc74a77d841e88e7990da4c29221f4bfe0994ac2ef78bf"
Oct 30 22:32:56.684379 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:32:56.684353    5510 scope.go:122] "RemoveContainer" containerID="20947ec152a579120ffc74a77d841e88e7990da4c29221f4bfe0994ac2ef78bf"
Oct 30 22:32:56.684639 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:32:56.684619    5510 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"20947ec152a579120ffc74a77d841e88e7990da4c29221f4bfe0994ac2ef78bf\": container with ID starting with 20947ec152a579120ffc74a77d841e88e7990da4c29221f4bfe0994ac2ef78bf not found: ID does not exist" containerID="20947ec152a579120ffc74a77d841e88e7990da4c29221f4bfe0994ac2ef78bf"
Oct 30 22:32:56.684731 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:32:56.684647    5510 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"20947ec152a579120ffc74a77d841e88e7990da4c29221f4bfe0994ac2ef78bf"} err="failed to get container status \"20947ec152a579120ffc74a77d841e88e7990da4c29221f4bfe0994ac2ef78bf\": rpc error: code = NotFound desc = could not find container \"20947ec152a579120ffc74a77d841e88e7990da4c29221f4bfe0994ac2ef78bf\": container with ID starting with 20947ec152a579120ffc74a77d841e88e7990da4c29221f4bfe0994ac2ef78bf not found: ID does not exist"
Oct 30 22:32:56.857734 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:32:56.854043    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-c42c0a-8545457dcf-ghrwb"]
Oct 30 22:32:56.889191 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:32:56.889151    5510 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-c42c0a-8545457dcf-ghrwb"]
Oct 30 22:32:57.553286 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:32:57.553256    5510 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="42db3ad0-7d90-4073-9ad9-19b6b5590551" path="/var/lib/kubelet/pods/42db3ad0-7d90-4073-9ad9-19b6b5590551/volumes"
Oct 30 22:32:57.567265 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:32:57.567233    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:32:57.763348 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:32:57.763301    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:32:57.976358 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:32:57.976264    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:32:58.512501 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:32:58.512464    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:33:02.503795 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:33:02.503755    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:33:04.705339 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:33:04.705304    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:33:04.949735 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:33:04.949650    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:33:05.016008 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:33:05.015923    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w0-c42c0a-openshift-default-558b6f6d6f-7rcsl"]
Oct 30 22:33:05.016193 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:33:05.016146    5510 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w0-c42c0a-openshift-default-558b6f6d6f-7rcsl" podUID="3a76b5b9-88cf-4f7f-9d14-fd725017b62f" containerName="istio-proxy" containerID="cri-o://a0c3f8fb005cc45c5fc150779ea05a86b4117cb7ce1c3d7425174d06a8652a66" gracePeriod=30
Oct 30 22:33:05.146799 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:33:05.146745    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:33:07.592063 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:33:07.592027    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:33:08.419860 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:33:08.419809    5510 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6gjbf" containerName="extractor"
Oct 30 22:33:10.282979 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:33:10.282918    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w0-c42c0a-openshift-default-558b6f6d6f-7rcsl"
Oct 30 22:33:10.427145 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:33:10.427110    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3a76b5b9-88cf-4f7f-9d14-fd725017b62f-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a76b5b9-88cf-4f7f-9d14-fd725017b62f-credential-socket\") pod \"3a76b5b9-88cf-4f7f-9d14-fd725017b62f\" (UID: \"3a76b5b9-88cf-4f7f-9d14-fd725017b62f\") "
Oct 30 22:33:10.427374 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:33:10.427172    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3a76b5b9-88cf-4f7f-9d14-fd725017b62f-istio-token\" (UniqueName: \"kubernetes.io/projected/3a76b5b9-88cf-4f7f-9d14-fd725017b62f-istio-token\") pod \"3a76b5b9-88cf-4f7f-9d14-fd725017b62f\" (UID: \"3a76b5b9-88cf-4f7f-9d14-fd725017b62f\") "
Oct 30 22:33:10.427374 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:33:10.427193    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3a76b5b9-88cf-4f7f-9d14-fd725017b62f-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3a76b5b9-88cf-4f7f-9d14-fd725017b62f-istio-data\") pod \"3a76b5b9-88cf-4f7f-9d14-fd725017b62f\" (UID: \"3a76b5b9-88cf-4f7f-9d14-fd725017b62f\") "
Oct 30 22:33:10.427374 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:33:10.427298    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3a76b5b9-88cf-4f7f-9d14-fd725017b62f-kube-api-access-fpzw4\" (UniqueName: \"kubernetes.io/projected/3a76b5b9-88cf-4f7f-9d14-fd725017b62f-kube-api-access-fpzw4\") pod \"3a76b5b9-88cf-4f7f-9d14-fd725017b62f\" (UID: \"3a76b5b9-88cf-4f7f-9d14-fd725017b62f\") "
Oct 30 22:33:10.427374 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:33:10.427356    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3a76b5b9-88cf-4f7f-9d14-fd725017b62f-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3a76b5b9-88cf-4f7f-9d14-fd725017b62f-workload-certs\") pod \"3a76b5b9-88cf-4f7f-9d14-fd725017b62f\" (UID: \"3a76b5b9-88cf-4f7f-9d14-fd725017b62f\") "
Oct 30 22:33:10.427611 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:33:10.427383    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3a76b5b9-88cf-4f7f-9d14-fd725017b62f-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3a76b5b9-88cf-4f7f-9d14-fd725017b62f-istio-envoy\") pod \"3a76b5b9-88cf-4f7f-9d14-fd725017b62f\" (UID: \"3a76b5b9-88cf-4f7f-9d14-fd725017b62f\") "
Oct 30 22:33:10.427611 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:33:10.427467    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3a76b5b9-88cf-4f7f-9d14-fd725017b62f-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a76b5b9-88cf-4f7f-9d14-fd725017b62f-workload-socket\") pod \"3a76b5b9-88cf-4f7f-9d14-fd725017b62f\" (UID: \"3a76b5b9-88cf-4f7f-9d14-fd725017b62f\") "
Oct 30 22:33:10.427611 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:33:10.427494    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/3a76b5b9-88cf-4f7f-9d14-fd725017b62f-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3a76b5b9-88cf-4f7f-9d14-fd725017b62f-istio-podinfo\") pod \"3a76b5b9-88cf-4f7f-9d14-fd725017b62f\" (UID: \"3a76b5b9-88cf-4f7f-9d14-fd725017b62f\") "
Oct 30 22:33:10.427611 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:33:10.427496    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3a76b5b9-88cf-4f7f-9d14-fd725017b62f-credential-socket" pod "3a76b5b9-88cf-4f7f-9d14-fd725017b62f" (UID: "3a76b5b9-88cf-4f7f-9d14-fd725017b62f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:33:10.427611 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:33:10.427587    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3a76b5b9-88cf-4f7f-9d14-fd725017b62f-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3a76b5b9-88cf-4f7f-9d14-fd725017b62f-istiod-ca-cert\") pod \"3a76b5b9-88cf-4f7f-9d14-fd725017b62f\" (UID: \"3a76b5b9-88cf-4f7f-9d14-fd725017b62f\") "
Oct 30 22:33:10.427917 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:33:10.427617    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3a76b5b9-88cf-4f7f-9d14-fd725017b62f-istio-data" pod "3a76b5b9-88cf-4f7f-9d14-fd725017b62f" (UID: "3a76b5b9-88cf-4f7f-9d14-fd725017b62f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:33:10.427917 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:33:10.427640    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3a76b5b9-88cf-4f7f-9d14-fd725017b62f-workload-certs" pod "3a76b5b9-88cf-4f7f-9d14-fd725017b62f" (UID: "3a76b5b9-88cf-4f7f-9d14-fd725017b62f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:33:10.427917 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:33:10.427883    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3a76b5b9-88cf-4f7f-9d14-fd725017b62f-workload-socket" pod "3a76b5b9-88cf-4f7f-9d14-fd725017b62f" (UID: "3a76b5b9-88cf-4f7f-9d14-fd725017b62f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:33:10.428211 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:33:10.428184    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3a76b5b9-88cf-4f7f-9d14-fd725017b62f-istiod-ca-cert" pod "3a76b5b9-88cf-4f7f-9d14-fd725017b62f" (UID: "3a76b5b9-88cf-4f7f-9d14-fd725017b62f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:33:10.428285 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:33:10.428222    5510 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a76b5b9-88cf-4f7f-9d14-fd725017b62f-credential-socket\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:33:10.428285 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:33:10.428263    5510 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3a76b5b9-88cf-4f7f-9d14-fd725017b62f-istio-data\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:33:10.428285 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:33:10.428279    5510 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3a76b5b9-88cf-4f7f-9d14-fd725017b62f-workload-certs\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:33:10.428429 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:33:10.428294    5510 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a76b5b9-88cf-4f7f-9d14-fd725017b62f-workload-socket\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:33:10.429793 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:33:10.429762    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3a76b5b9-88cf-4f7f-9d14-fd725017b62f-istio-envoy" pod "3a76b5b9-88cf-4f7f-9d14-fd725017b62f" (UID: "3a76b5b9-88cf-4f7f-9d14-fd725017b62f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:33:10.429937 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:33:10.429801    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3a76b5b9-88cf-4f7f-9d14-fd725017b62f-istio-token" pod "3a76b5b9-88cf-4f7f-9d14-fd725017b62f" (UID: "3a76b5b9-88cf-4f7f-9d14-fd725017b62f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:33:10.430033 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:33:10.430017    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3a76b5b9-88cf-4f7f-9d14-fd725017b62f-istio-podinfo" pod "3a76b5b9-88cf-4f7f-9d14-fd725017b62f" (UID: "3a76b5b9-88cf-4f7f-9d14-fd725017b62f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:33:10.430121 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:33:10.430107    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3a76b5b9-88cf-4f7f-9d14-fd725017b62f-kube-api-access-fpzw4" pod "3a76b5b9-88cf-4f7f-9d14-fd725017b62f" (UID: "3a76b5b9-88cf-4f7f-9d14-fd725017b62f"). InnerVolumeSpecName "kube-api-access-fpzw4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:33:10.529894 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:33:10.529858    5510 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3a76b5b9-88cf-4f7f-9d14-fd725017b62f-istio-podinfo\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:33:10.529894 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:33:10.529888    5510 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3a76b5b9-88cf-4f7f-9d14-fd725017b62f-istiod-ca-cert\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:33:10.529894 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:33:10.529898    5510 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3a76b5b9-88cf-4f7f-9d14-fd725017b62f-istio-token\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:33:10.530139 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:33:10.529907    5510 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fpzw4\" (UniqueName: \"kubernetes.io/projected/3a76b5b9-88cf-4f7f-9d14-fd725017b62f-kube-api-access-fpzw4\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:33:10.530139 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:33:10.529918    5510 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3a76b5b9-88cf-4f7f-9d14-fd725017b62f-istio-envoy\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:33:10.769037 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:33:10.769004    5510 generic.go:356] "Generic (PLEG): container finished" podID="3a76b5b9-88cf-4f7f-9d14-fd725017b62f" containerID="a0c3f8fb005cc45c5fc150779ea05a86b4117cb7ce1c3d7425174d06a8652a66" exitCode=0
Oct 30 22:33:10.769221 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:33:10.769051    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w0-c42c0a-openshift-default-558b6f6d6f-7rcsl" event={"ID":"3a76b5b9-88cf-4f7f-9d14-fd725017b62f","Type":"ContainerDied","Data":"a0c3f8fb005cc45c5fc150779ea05a86b4117cb7ce1c3d7425174d06a8652a66"}
Oct 30 22:33:10.769221 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:33:10.769077    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w0-c42c0a-openshift-default-558b6f6d6f-7rcsl"
Oct 30 22:33:10.769221 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:33:10.769093    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w0-c42c0a-openshift-default-558b6f6d6f-7rcsl" event={"ID":"3a76b5b9-88cf-4f7f-9d14-fd725017b62f","Type":"ContainerDied","Data":"f7c15ebc7bdeb825ba84162e90afe48dcfbe5a9d923930cecd984d28c41ddf15"}
Oct 30 22:33:10.769221 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:33:10.769115    5510 scope.go:122] "RemoveContainer" containerID="a0c3f8fb005cc45c5fc150779ea05a86b4117cb7ce1c3d7425174d06a8652a66"
Oct 30 22:33:10.779695 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:33:10.779676    5510 scope.go:122] "RemoveContainer" containerID="f9f8ec7c0b142468c3f7862ce226aa9aa72fbaeb489447aefd7e17bb2921aae2"
Oct 30 22:33:10.788438 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:33:10.788422    5510 scope.go:122] "RemoveContainer" containerID="a0c3f8fb005cc45c5fc150779ea05a86b4117cb7ce1c3d7425174d06a8652a66"
Oct 30 22:33:10.788712 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:33:10.788692    5510 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a0c3f8fb005cc45c5fc150779ea05a86b4117cb7ce1c3d7425174d06a8652a66\": container with ID starting with a0c3f8fb005cc45c5fc150779ea05a86b4117cb7ce1c3d7425174d06a8652a66 not found: ID does not exist" containerID="a0c3f8fb005cc45c5fc150779ea05a86b4117cb7ce1c3d7425174d06a8652a66"
Oct 30 22:33:10.788782 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:33:10.788722    5510 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a0c3f8fb005cc45c5fc150779ea05a86b4117cb7ce1c3d7425174d06a8652a66"} err="failed to get container status \"a0c3f8fb005cc45c5fc150779ea05a86b4117cb7ce1c3d7425174d06a8652a66\": rpc error: code = NotFound desc = could not find container \"a0c3f8fb005cc45c5fc150779ea05a86b4117cb7ce1c3d7425174d06a8652a66\": container with ID starting with a0c3f8fb005cc45c5fc150779ea05a86b4117cb7ce1c3d7425174d06a8652a66 not found: ID does not exist"
Oct 30 22:33:10.788782 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:33:10.788741    5510 scope.go:122] "RemoveContainer" containerID="f9f8ec7c0b142468c3f7862ce226aa9aa72fbaeb489447aefd7e17bb2921aae2"
Oct 30 22:33:10.788989 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:33:10.788966    5510 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f9f8ec7c0b142468c3f7862ce226aa9aa72fbaeb489447aefd7e17bb2921aae2\": container with ID starting with f9f8ec7c0b142468c3f7862ce226aa9aa72fbaeb489447aefd7e17bb2921aae2 not found: ID does not exist" containerID="f9f8ec7c0b142468c3f7862ce226aa9aa72fbaeb489447aefd7e17bb2921aae2"
Oct 30 22:33:10.789034 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:33:10.788993    5510 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f9f8ec7c0b142468c3f7862ce226aa9aa72fbaeb489447aefd7e17bb2921aae2"} err="failed to get container status \"f9f8ec7c0b142468c3f7862ce226aa9aa72fbaeb489447aefd7e17bb2921aae2\": rpc error: code = NotFound desc = could not find container \"f9f8ec7c0b142468c3f7862ce226aa9aa72fbaeb489447aefd7e17bb2921aae2\": container with ID starting with f9f8ec7c0b142468c3f7862ce226aa9aa72fbaeb489447aefd7e17bb2921aae2 not found: ID does not exist"
Oct 30 22:33:10.925695 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:33:10.925645    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w0-c42c0a-openshift-default-558b6f6d6f-7rcsl"]
Oct 30 22:33:10.931583 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:33:10.931551    5510 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w0-c42c0a-openshift-default-558b6f6d6f-7rcsl"]
Oct 30 22:33:11.553872 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:33:11.553831    5510 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3a76b5b9-88cf-4f7f-9d14-fd725017b62f" path="/var/lib/kubelet/pods/3a76b5b9-88cf-4f7f-9d14-fd725017b62f/volumes"
Oct 30 22:33:12.470031 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:33:12.469987    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:33:14.426681 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:33:14.421137    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:33:14.831965 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:33:14.831927    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:33:15.035711 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:33:15.035646    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:33:16.008548 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:33:16.008504    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:33:16.667400 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:33:16.665000    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:33:16.865013 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:33:16.864963    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:33:17.833378 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:33:17.833332    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:33:22.436777 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:33:22.436732    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:33:22.691505 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:33:22.691415    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:33:22.925036 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:33:22.924996    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:33:25.687756 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:33:25.687718    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:33:27.619301 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:33:27.619258    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:33:32.333061 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:33:32.333016    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:33:33.155791 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:33:33.155751    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:33:36.888285 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:33:36.888246    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:33:38.419680 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:33:38.419629    5510 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6gjbf" containerName="extractor"
Oct 30 22:33:41.433100 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:33:41.433059    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:33:46.088752 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:33:46.088705    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:33:50.808387 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:33:50.808344    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:33:55.315004 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:33:55.314964    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:33:55.609045 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:33:55.608963    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:33:59.535412 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:33:59.535367    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-ed44da-9b6f6d88c-jz6p9"]
Oct 30 22:33:59.535964 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:33:59.535620    5510 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-ed44da-9b6f6d88c-jz6p9" podUID="f8bd673e-4938-421a-bb40-5eb3e7f7802f" containerName="maas-api" containerID="cri-o://0bef431ef8a859e9eb41393ac8a3f93b9b6541c0f05e20fdee02003361424a70" gracePeriod=30
Oct 30 22:33:59.821318 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:33:59.821289    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-ed44da-9b6f6d88c-jz6p9"
Oct 30 22:33:59.906982 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:33:59.904596    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:33:59.918309 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:33:59.918278    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f8bd673e-4938-421a-bb40-5eb3e7f7802f-metrics-certs\" (UniqueName: \"kubernetes.io/secret/f8bd673e-4938-421a-bb40-5eb3e7f7802f-metrics-certs\") pod \"f8bd673e-4938-421a-bb40-5eb3e7f7802f\" (UID: \"f8bd673e-4938-421a-bb40-5eb3e7f7802f\") "
Oct 30 22:33:59.918462 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:33:59.918341    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f8bd673e-4938-421a-bb40-5eb3e7f7802f-kube-api-access-8r65x\" (UniqueName: \"kubernetes.io/projected/f8bd673e-4938-421a-bb40-5eb3e7f7802f-kube-api-access-8r65x\") pod \"f8bd673e-4938-421a-bb40-5eb3e7f7802f\" (UID: \"f8bd673e-4938-421a-bb40-5eb3e7f7802f\") "
Oct 30 22:33:59.918462 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:33:59.918385    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f8bd673e-4938-421a-bb40-5eb3e7f7802f-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f8bd673e-4938-421a-bb40-5eb3e7f7802f-openshift-service-ca\") pod \"f8bd673e-4938-421a-bb40-5eb3e7f7802f\" (UID: \"f8bd673e-4938-421a-bb40-5eb3e7f7802f\") "
Oct 30 22:33:59.918462 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:33:59.918436    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f8bd673e-4938-421a-bb40-5eb3e7f7802f-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f8bd673e-4938-421a-bb40-5eb3e7f7802f-maas-api-tls\") pod \"f8bd673e-4938-421a-bb40-5eb3e7f7802f\" (UID: \"f8bd673e-4938-421a-bb40-5eb3e7f7802f\") "
Oct 30 22:33:59.918781 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:33:59.918755    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f8bd673e-4938-421a-bb40-5eb3e7f7802f-openshift-service-ca" pod "f8bd673e-4938-421a-bb40-5eb3e7f7802f" (UID: "f8bd673e-4938-421a-bb40-5eb3e7f7802f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:33:59.918979 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:33:59.918958    5510 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f8bd673e-4938-421a-bb40-5eb3e7f7802f-openshift-service-ca\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:33:59.920539 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:33:59.920516    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f8bd673e-4938-421a-bb40-5eb3e7f7802f-maas-api-tls" pod "f8bd673e-4938-421a-bb40-5eb3e7f7802f" (UID: "f8bd673e-4938-421a-bb40-5eb3e7f7802f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:33:59.920645 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:33:59.920556    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f8bd673e-4938-421a-bb40-5eb3e7f7802f-metrics-certs" pod "f8bd673e-4938-421a-bb40-5eb3e7f7802f" (UID: "f8bd673e-4938-421a-bb40-5eb3e7f7802f"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:33:59.920645 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:33:59.920567    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f8bd673e-4938-421a-bb40-5eb3e7f7802f-kube-api-access-8r65x" pod "f8bd673e-4938-421a-bb40-5eb3e7f7802f" (UID: "f8bd673e-4938-421a-bb40-5eb3e7f7802f"). InnerVolumeSpecName "kube-api-access-8r65x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:34:00.019831 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:00.019754    5510 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f8bd673e-4938-421a-bb40-5eb3e7f7802f-maas-api-tls\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:34:00.019831 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:00.019782    5510 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f8bd673e-4938-421a-bb40-5eb3e7f7802f-metrics-certs\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:34:00.019831 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:00.019793    5510 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8r65x\" (UniqueName: \"kubernetes.io/projected/f8bd673e-4938-421a-bb40-5eb3e7f7802f-kube-api-access-8r65x\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:34:00.060101 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:00.060022    5510 generic.go:356] "Generic (PLEG): container finished" podID="f8bd673e-4938-421a-bb40-5eb3e7f7802f" containerID="0bef431ef8a859e9eb41393ac8a3f93b9b6541c0f05e20fdee02003361424a70" exitCode=0
Oct 30 22:34:00.060101 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:00.060072    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-ed44da-9b6f6d88c-jz6p9"
Oct 30 22:34:00.060342 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:00.060114    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-ed44da-9b6f6d88c-jz6p9" event={"ID":"f8bd673e-4938-421a-bb40-5eb3e7f7802f","Type":"ContainerDied","Data":"0bef431ef8a859e9eb41393ac8a3f93b9b6541c0f05e20fdee02003361424a70"}
Oct 30 22:34:00.060342 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:00.060172    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-ed44da-9b6f6d88c-jz6p9" event={"ID":"f8bd673e-4938-421a-bb40-5eb3e7f7802f","Type":"ContainerDied","Data":"42b5d9c2b748bacc5e36c5e1a35f17e39fb8b31d37c93d86bc58b8a7c83baa83"}
Oct 30 22:34:00.060342 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:00.060202    5510 scope.go:122] "RemoveContainer" containerID="0bef431ef8a859e9eb41393ac8a3f93b9b6541c0f05e20fdee02003361424a70"
Oct 30 22:34:00.070774 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:00.070748    5510 scope.go:122] "RemoveContainer" containerID="0bef431ef8a859e9eb41393ac8a3f93b9b6541c0f05e20fdee02003361424a70"
Oct 30 22:34:00.071104 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:34:00.071077    5510 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0bef431ef8a859e9eb41393ac8a3f93b9b6541c0f05e20fdee02003361424a70\": container with ID starting with 0bef431ef8a859e9eb41393ac8a3f93b9b6541c0f05e20fdee02003361424a70 not found: ID does not exist" containerID="0bef431ef8a859e9eb41393ac8a3f93b9b6541c0f05e20fdee02003361424a70"
Oct 30 22:34:00.071174 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:00.071116    5510 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0bef431ef8a859e9eb41393ac8a3f93b9b6541c0f05e20fdee02003361424a70"} err="failed to get container status \"0bef431ef8a859e9eb41393ac8a3f93b9b6541c0f05e20fdee02003361424a70\": rpc error: code = NotFound desc = could not find container \"0bef431ef8a859e9eb41393ac8a3f93b9b6541c0f05e20fdee02003361424a70\": container with ID starting with 0bef431ef8a859e9eb41393ac8a3f93b9b6541c0f05e20fdee02003361424a70 not found: ID does not exist"
Oct 30 22:34:00.138494 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:00.138455    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:34:00.233708 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:00.232030    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-ed44da-9b6f6d88c-jz6p9"]
Oct 30 22:34:00.252218 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:00.252183    5510 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-ed44da-9b6f6d88c-jz6p9"]
Oct 30 22:34:01.317955 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:01.317921    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:34:01.529353 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:01.529318    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:34:01.553265 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:01.553235    5510 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f8bd673e-4938-421a-bb40-5eb3e7f7802f" path="/var/lib/kubelet/pods/f8bd673e-4938-421a-bb40-5eb3e7f7802f/volumes"
Oct 30 22:34:01.733775 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:01.733684    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:34:02.257229 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:02.257194    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:34:05.729593 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:05.729553    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-bc714d-58cbbc478d-xsqpn"]
Oct 30 22:34:05.857639 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:05.857599    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:34:06.131445 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:06.131405    5510 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-bc714d-58cbbc478d-xsqpn" podUID="afc34d05-1306-4b04-8c81-f123a51f682c" containerName="maas-api" containerID="cri-o://7c5f6e92fc61553d960f5e6364d7393271237a87f58a0c3e0a1ceb00ec19b79e" gracePeriod=30
Oct 30 22:34:06.390251 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:06.390189    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-bc714d-58cbbc478d-xsqpn"
Oct 30 22:34:06.418784 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:06.418752    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/afc34d05-1306-4b04-8c81-f123a51f682c-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/afc34d05-1306-4b04-8c81-f123a51f682c-maas-api-tls\") pod \"afc34d05-1306-4b04-8c81-f123a51f682c\" (UID: \"afc34d05-1306-4b04-8c81-f123a51f682c\") "
Oct 30 22:34:06.418962 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:06.418821    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/afc34d05-1306-4b04-8c81-f123a51f682c-kube-api-access-tv88v\" (UniqueName: \"kubernetes.io/projected/afc34d05-1306-4b04-8c81-f123a51f682c-kube-api-access-tv88v\") pod \"afc34d05-1306-4b04-8c81-f123a51f682c\" (UID: \"afc34d05-1306-4b04-8c81-f123a51f682c\") "
Oct 30 22:34:06.418962 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:06.418840    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/afc34d05-1306-4b04-8c81-f123a51f682c-metrics-certs\" (UniqueName: \"kubernetes.io/secret/afc34d05-1306-4b04-8c81-f123a51f682c-metrics-certs\") pod \"afc34d05-1306-4b04-8c81-f123a51f682c\" (UID: \"afc34d05-1306-4b04-8c81-f123a51f682c\") "
Oct 30 22:34:06.418962 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:06.418882    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/afc34d05-1306-4b04-8c81-f123a51f682c-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/afc34d05-1306-4b04-8c81-f123a51f682c-openshift-service-ca\") pod \"afc34d05-1306-4b04-8c81-f123a51f682c\" (UID: \"afc34d05-1306-4b04-8c81-f123a51f682c\") "
Oct 30 22:34:06.419362 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:06.419334    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/afc34d05-1306-4b04-8c81-f123a51f682c-openshift-service-ca" pod "afc34d05-1306-4b04-8c81-f123a51f682c" (UID: "afc34d05-1306-4b04-8c81-f123a51f682c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:34:06.420972 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:06.420951    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/afc34d05-1306-4b04-8c81-f123a51f682c-maas-api-tls" pod "afc34d05-1306-4b04-8c81-f123a51f682c" (UID: "afc34d05-1306-4b04-8c81-f123a51f682c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:34:06.421180 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:06.421164    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/afc34d05-1306-4b04-8c81-f123a51f682c-kube-api-access-tv88v" pod "afc34d05-1306-4b04-8c81-f123a51f682c" (UID: "afc34d05-1306-4b04-8c81-f123a51f682c"). InnerVolumeSpecName "kube-api-access-tv88v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:34:06.421266 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:06.421246    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/afc34d05-1306-4b04-8c81-f123a51f682c-metrics-certs" pod "afc34d05-1306-4b04-8c81-f123a51f682c" (UID: "afc34d05-1306-4b04-8c81-f123a51f682c"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:34:06.520343 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:06.520311    5510 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/afc34d05-1306-4b04-8c81-f123a51f682c-maas-api-tls\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:34:06.520343 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:06.520341    5510 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tv88v\" (UniqueName: \"kubernetes.io/projected/afc34d05-1306-4b04-8c81-f123a51f682c-kube-api-access-tv88v\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:34:06.520343 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:06.520352    5510 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/afc34d05-1306-4b04-8c81-f123a51f682c-metrics-certs\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:34:06.520614 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:06.520360    5510 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/afc34d05-1306-4b04-8c81-f123a51f682c-openshift-service-ca\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:34:07.085043 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:07.085005    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-xv7c7"]
Oct 30 22:34:07.085474 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:07.085199    5510 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-xv7c7" podUID="21c0fef2-49cd-4734-b402-f1cdeb031ac8" containerName="istio-proxy" containerID="cri-o://4a7ab40fdddf291e6c6022b41eb55c5dcb262680b18dc93ef9a1447cde8a7616" gracePeriod=30
Oct 30 22:34:07.141724 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:07.141687    5510 generic.go:356] "Generic (PLEG): container finished" podID="afc34d05-1306-4b04-8c81-f123a51f682c" containerID="7c5f6e92fc61553d960f5e6364d7393271237a87f58a0c3e0a1ceb00ec19b79e" exitCode=0
Oct 30 22:34:07.141909 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:07.141732    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-bc714d-58cbbc478d-xsqpn" event={"ID":"afc34d05-1306-4b04-8c81-f123a51f682c","Type":"ContainerDied","Data":"7c5f6e92fc61553d960f5e6364d7393271237a87f58a0c3e0a1ceb00ec19b79e"}
Oct 30 22:34:07.141909 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:07.141789    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-bc714d-58cbbc478d-xsqpn" event={"ID":"afc34d05-1306-4b04-8c81-f123a51f682c","Type":"ContainerDied","Data":"6bd84cb5a0ce22cd6d35c436f92ec19a37a02a299bf623d3c00cfd94b9d3ca05"}
Oct 30 22:34:07.141909 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:07.141804    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-bc714d-58cbbc478d-xsqpn"
Oct 30 22:34:07.141909 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:07.141820    5510 scope.go:122] "RemoveContainer" containerID="7c5f6e92fc61553d960f5e6364d7393271237a87f58a0c3e0a1ceb00ec19b79e"
Oct 30 22:34:07.153411 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:07.153301    5510 scope.go:122] "RemoveContainer" containerID="7c5f6e92fc61553d960f5e6364d7393271237a87f58a0c3e0a1ceb00ec19b79e"
Oct 30 22:34:07.153638 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:34:07.153610    5510 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7c5f6e92fc61553d960f5e6364d7393271237a87f58a0c3e0a1ceb00ec19b79e\": container with ID starting with 7c5f6e92fc61553d960f5e6364d7393271237a87f58a0c3e0a1ceb00ec19b79e not found: ID does not exist" containerID="7c5f6e92fc61553d960f5e6364d7393271237a87f58a0c3e0a1ceb00ec19b79e"
Oct 30 22:34:07.153789 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:07.153646    5510 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7c5f6e92fc61553d960f5e6364d7393271237a87f58a0c3e0a1ceb00ec19b79e"} err="failed to get container status \"7c5f6e92fc61553d960f5e6364d7393271237a87f58a0c3e0a1ceb00ec19b79e\": rpc error: code = NotFound desc = could not find container \"7c5f6e92fc61553d960f5e6364d7393271237a87f58a0c3e0a1ceb00ec19b79e\": container with ID starting with 7c5f6e92fc61553d960f5e6364d7393271237a87f58a0c3e0a1ceb00ec19b79e not found: ID does not exist"
Oct 30 22:34:07.314142 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:07.314112    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-bc714d-58cbbc478d-xsqpn"]
Oct 30 22:34:07.317675 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:07.317628    5510 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-bc714d-58cbbc478d-xsqpn"]
Oct 30 22:34:07.347241 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:07.347153    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:34:07.553751 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:07.553720    5510 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="afc34d05-1306-4b04-8c81-f123a51f682c" path="/var/lib/kubelet/pods/afc34d05-1306-4b04-8c81-f123a51f682c/volumes"
Oct 30 22:34:07.679750 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:07.679646    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:34:08.251164 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:08.251124    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:34:08.419628 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:34:08.419594    5510 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6gjbf" containerName="extractor"
Oct 30 22:34:09.642571 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:09.642528    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:34:09.868503 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:09.868470    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:34:10.102042 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:10.102002    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:34:10.566832 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:10.566788    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:34:10.902004 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:10.901927    5510 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-85544d4cd7-xv7c7 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 22:34:10.902004 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:10.901990    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-xv7c7" podUID="21c0fef2-49cd-4734-b402-f1cdeb031ac8" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 22:34:12.347328 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:12.347301    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-xv7c7"
Oct 30 22:34:12.414741 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:12.414642    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/21c0fef2-49cd-4734-b402-f1cdeb031ac8-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/21c0fef2-49cd-4734-b402-f1cdeb031ac8-workload-certs\") pod \"21c0fef2-49cd-4734-b402-f1cdeb031ac8\" (UID: \"21c0fef2-49cd-4734-b402-f1cdeb031ac8\") "
Oct 30 22:34:12.414888 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:12.414749    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/21c0fef2-49cd-4734-b402-f1cdeb031ac8-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/21c0fef2-49cd-4734-b402-f1cdeb031ac8-istio-data\") pod \"21c0fef2-49cd-4734-b402-f1cdeb031ac8\" (UID: \"21c0fef2-49cd-4734-b402-f1cdeb031ac8\") "
Oct 30 22:34:12.414888 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:12.414776    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/21c0fef2-49cd-4734-b402-f1cdeb031ac8-kube-api-access-ltb4x\" (UniqueName: \"kubernetes.io/projected/21c0fef2-49cd-4734-b402-f1cdeb031ac8-kube-api-access-ltb4x\") pod \"21c0fef2-49cd-4734-b402-f1cdeb031ac8\" (UID: \"21c0fef2-49cd-4734-b402-f1cdeb031ac8\") "
Oct 30 22:34:12.414888 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:12.414801    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/21c0fef2-49cd-4734-b402-f1cdeb031ac8-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/21c0fef2-49cd-4734-b402-f1cdeb031ac8-istio-podinfo\") pod \"21c0fef2-49cd-4734-b402-f1cdeb031ac8\" (UID: \"21c0fef2-49cd-4734-b402-f1cdeb031ac8\") "
Oct 30 22:34:12.414888 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:12.414859    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/21c0fef2-49cd-4734-b402-f1cdeb031ac8-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/21c0fef2-49cd-4734-b402-f1cdeb031ac8-istio-envoy\") pod \"21c0fef2-49cd-4734-b402-f1cdeb031ac8\" (UID: \"21c0fef2-49cd-4734-b402-f1cdeb031ac8\") "
Oct 30 22:34:12.414888 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:12.414878    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/21c0fef2-49cd-4734-b402-f1cdeb031ac8-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/21c0fef2-49cd-4734-b402-f1cdeb031ac8-credential-socket\") pod \"21c0fef2-49cd-4734-b402-f1cdeb031ac8\" (UID: \"21c0fef2-49cd-4734-b402-f1cdeb031ac8\") "
Oct 30 22:34:12.415164 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:12.414891    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/21c0fef2-49cd-4734-b402-f1cdeb031ac8-workload-certs" pod "21c0fef2-49cd-4734-b402-f1cdeb031ac8" (UID: "21c0fef2-49cd-4734-b402-f1cdeb031ac8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:34:12.415164 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:12.414925    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/21c0fef2-49cd-4734-b402-f1cdeb031ac8-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/21c0fef2-49cd-4734-b402-f1cdeb031ac8-istiod-ca-cert\") pod \"21c0fef2-49cd-4734-b402-f1cdeb031ac8\" (UID: \"21c0fef2-49cd-4734-b402-f1cdeb031ac8\") "
Oct 30 22:34:12.415164 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:12.414953    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/21c0fef2-49cd-4734-b402-f1cdeb031ac8-istio-token\" (UniqueName: \"kubernetes.io/projected/21c0fef2-49cd-4734-b402-f1cdeb031ac8-istio-token\") pod \"21c0fef2-49cd-4734-b402-f1cdeb031ac8\" (UID: \"21c0fef2-49cd-4734-b402-f1cdeb031ac8\") "
Oct 30 22:34:12.415164 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:12.414994    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/21c0fef2-49cd-4734-b402-f1cdeb031ac8-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/21c0fef2-49cd-4734-b402-f1cdeb031ac8-workload-socket\") pod \"21c0fef2-49cd-4734-b402-f1cdeb031ac8\" (UID: \"21c0fef2-49cd-4734-b402-f1cdeb031ac8\") "
Oct 30 22:34:12.415164 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:12.415004    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/21c0fef2-49cd-4734-b402-f1cdeb031ac8-istio-data" pod "21c0fef2-49cd-4734-b402-f1cdeb031ac8" (UID: "21c0fef2-49cd-4734-b402-f1cdeb031ac8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:34:12.415439 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:12.415287    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/21c0fef2-49cd-4734-b402-f1cdeb031ac8-credential-socket" pod "21c0fef2-49cd-4734-b402-f1cdeb031ac8" (UID: "21c0fef2-49cd-4734-b402-f1cdeb031ac8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:34:12.415439 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:12.415348    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/21c0fef2-49cd-4734-b402-f1cdeb031ac8-istiod-ca-cert" pod "21c0fef2-49cd-4734-b402-f1cdeb031ac8" (UID: "21c0fef2-49cd-4734-b402-f1cdeb031ac8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:34:12.415545 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:12.415517    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/21c0fef2-49cd-4734-b402-f1cdeb031ac8-workload-socket" pod "21c0fef2-49cd-4734-b402-f1cdeb031ac8" (UID: "21c0fef2-49cd-4734-b402-f1cdeb031ac8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:34:12.415722 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:12.415698    5510 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/21c0fef2-49cd-4734-b402-f1cdeb031ac8-workload-certs\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:34:12.415809 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:12.415727    5510 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/21c0fef2-49cd-4734-b402-f1cdeb031ac8-istio-data\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:34:12.415809 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:12.415743    5510 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/21c0fef2-49cd-4734-b402-f1cdeb031ac8-credential-socket\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:34:12.415809 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:12.415757    5510 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/21c0fef2-49cd-4734-b402-f1cdeb031ac8-istiod-ca-cert\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:34:12.415809 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:12.415771    5510 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/21c0fef2-49cd-4734-b402-f1cdeb031ac8-workload-socket\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:34:12.417285 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:12.417257    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/21c0fef2-49cd-4734-b402-f1cdeb031ac8-istio-envoy" pod "21c0fef2-49cd-4734-b402-f1cdeb031ac8" (UID: "21c0fef2-49cd-4734-b402-f1cdeb031ac8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:34:12.417398 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:12.417356    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/21c0fef2-49cd-4734-b402-f1cdeb031ac8-istio-podinfo" pod "21c0fef2-49cd-4734-b402-f1cdeb031ac8" (UID: "21c0fef2-49cd-4734-b402-f1cdeb031ac8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:34:12.417541 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:12.417520    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/21c0fef2-49cd-4734-b402-f1cdeb031ac8-istio-token" pod "21c0fef2-49cd-4734-b402-f1cdeb031ac8" (UID: "21c0fef2-49cd-4734-b402-f1cdeb031ac8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:34:12.417597 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:12.417551    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/21c0fef2-49cd-4734-b402-f1cdeb031ac8-kube-api-access-ltb4x" pod "21c0fef2-49cd-4734-b402-f1cdeb031ac8" (UID: "21c0fef2-49cd-4734-b402-f1cdeb031ac8"). InnerVolumeSpecName "kube-api-access-ltb4x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:34:12.516698 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:12.516648    5510 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ltb4x\" (UniqueName: \"kubernetes.io/projected/21c0fef2-49cd-4734-b402-f1cdeb031ac8-kube-api-access-ltb4x\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:34:12.516698 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:12.516697    5510 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/21c0fef2-49cd-4734-b402-f1cdeb031ac8-istio-podinfo\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:34:12.516698 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:12.516708    5510 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/21c0fef2-49cd-4734-b402-f1cdeb031ac8-istio-envoy\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:34:12.516982 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:12.516718    5510 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/21c0fef2-49cd-4734-b402-f1cdeb031ac8-istio-token\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:34:12.791627 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:12.791587    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:34:13.179455 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:13.179363    5510 generic.go:356] "Generic (PLEG): container finished" podID="21c0fef2-49cd-4734-b402-f1cdeb031ac8" containerID="4a7ab40fdddf291e6c6022b41eb55c5dcb262680b18dc93ef9a1447cde8a7616" exitCode=0
Oct 30 22:34:13.179455 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:13.179414    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-xv7c7"
Oct 30 22:34:13.179669 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:13.179447    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-xv7c7" event={"ID":"21c0fef2-49cd-4734-b402-f1cdeb031ac8","Type":"ContainerDied","Data":"4a7ab40fdddf291e6c6022b41eb55c5dcb262680b18dc93ef9a1447cde8a7616"}
Oct 30 22:34:13.179669 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:13.179509    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-xv7c7" event={"ID":"21c0fef2-49cd-4734-b402-f1cdeb031ac8","Type":"ContainerDied","Data":"5912920fc39403e38e45c4d02ebe937e09ff944960005edc4b3aa9eb8bb73f45"}
Oct 30 22:34:13.179669 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:13.179523    5510 scope.go:122] "RemoveContainer" containerID="4a7ab40fdddf291e6c6022b41eb55c5dcb262680b18dc93ef9a1447cde8a7616"
Oct 30 22:34:13.190593 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:13.190568    5510 scope.go:122] "RemoveContainer" containerID="4a7ab40fdddf291e6c6022b41eb55c5dcb262680b18dc93ef9a1447cde8a7616"
Oct 30 22:34:13.190926 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:34:13.190899    5510 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4a7ab40fdddf291e6c6022b41eb55c5dcb262680b18dc93ef9a1447cde8a7616\": container with ID starting with 4a7ab40fdddf291e6c6022b41eb55c5dcb262680b18dc93ef9a1447cde8a7616 not found: ID does not exist" containerID="4a7ab40fdddf291e6c6022b41eb55c5dcb262680b18dc93ef9a1447cde8a7616"
Oct 30 22:34:13.191013 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:13.190937    5510 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4a7ab40fdddf291e6c6022b41eb55c5dcb262680b18dc93ef9a1447cde8a7616"} err="failed to get container status \"4a7ab40fdddf291e6c6022b41eb55c5dcb262680b18dc93ef9a1447cde8a7616\": rpc error: code = NotFound desc = could not find container \"4a7ab40fdddf291e6c6022b41eb55c5dcb262680b18dc93ef9a1447cde8a7616\": container with ID starting with 4a7ab40fdddf291e6c6022b41eb55c5dcb262680b18dc93ef9a1447cde8a7616 not found: ID does not exist"
Oct 30 22:34:13.337379 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:13.337347    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-xv7c7"]
Oct 30 22:34:13.383499 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:13.383468    5510 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-xv7c7"]
Oct 30 22:34:13.553641 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:13.553594    5510 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="21c0fef2-49cd-4734-b402-f1cdeb031ac8" path="/var/lib/kubelet/pods/21c0fef2-49cd-4734-b402-f1cdeb031ac8/volumes"
Oct 30 22:34:15.014209 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:15.014155    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:34:15.210987 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:15.210952    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:34:15.258792 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:15.258756    5510 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-07e9c8/test-model-07e9c8-kserve-9c9cbf4bc-b88r4"]
Oct 30 22:34:15.259532 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:15.259510    5510 cpu_manager.go:414] "containerMap: removing container" podUID="afc34d05-1306-4b04-8c81-f123a51f682c" containerName="maas-api"
Oct 30 22:34:15.259532 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:15.259531    5510 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="afc34d05-1306-4b04-8c81-f123a51f682c" containerName="maas-api"
Oct 30 22:34:15.259712 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:15.259543    5510 cpu_manager.go:414] "containerMap: removing container" podUID="3a76b5b9-88cf-4f7f-9d14-fd725017b62f" containerName="istio-proxy"
Oct 30 22:34:15.259712 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:15.259548    5510 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3a76b5b9-88cf-4f7f-9d14-fd725017b62f" containerName="istio-proxy"
Oct 30 22:34:15.259712 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:15.259566    5510 cpu_manager.go:414] "containerMap: removing container" podUID="f8bd673e-4938-421a-bb40-5eb3e7f7802f" containerName="maas-api"
Oct 30 22:34:15.259712 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:15.259572    5510 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f8bd673e-4938-421a-bb40-5eb3e7f7802f" containerName="maas-api"
Oct 30 22:34:15.259712 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:15.259595    5510 cpu_manager.go:414] "containerMap: removing container" podUID="21557bdf-26e4-4a0c-ad9f-50dcc391616b" containerName="istio-proxy"
Oct 30 22:34:15.259712 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:15.259600    5510 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="21557bdf-26e4-4a0c-ad9f-50dcc391616b" containerName="istio-proxy"
Oct 30 22:34:15.259712 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:15.259607    5510 cpu_manager.go:414] "containerMap: removing container" podUID="3a76b5b9-88cf-4f7f-9d14-fd725017b62f" containerName="istio-proxy"
Oct 30 22:34:15.259712 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:15.259612    5510 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3a76b5b9-88cf-4f7f-9d14-fd725017b62f" containerName="istio-proxy"
Oct 30 22:34:15.259712 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:15.259622    5510 cpu_manager.go:414] "containerMap: removing container" podUID="42db3ad0-7d90-4073-9ad9-19b6b5590551" containerName="maas-api"
Oct 30 22:34:15.259712 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:15.259627    5510 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="42db3ad0-7d90-4073-9ad9-19b6b5590551" containerName="maas-api"
Oct 30 22:34:15.259712 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:15.259634    5510 cpu_manager.go:414] "containerMap: removing container" podUID="9e9b11b8-1cc9-4fc8-8097-1afcffa8c422" containerName="main"
Oct 30 22:34:15.259712 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:15.259639    5510 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9e9b11b8-1cc9-4fc8-8097-1afcffa8c422" containerName="main"
Oct 30 22:34:15.259712 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:15.259648    5510 cpu_manager.go:414] "containerMap: removing container" podUID="c4152cef-f9cf-46c1-a9c4-78484ab58efe" containerName="istio-proxy"
Oct 30 22:34:15.259712 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:15.259664    5510 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c4152cef-f9cf-46c1-a9c4-78484ab58efe" containerName="istio-proxy"
Oct 30 22:34:15.259712 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:15.259673    5510 cpu_manager.go:414] "containerMap: removing container" podUID="21557bdf-26e4-4a0c-ad9f-50dcc391616b" containerName="istio-proxy"
Oct 30 22:34:15.259712 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:15.259677    5510 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="21557bdf-26e4-4a0c-ad9f-50dcc391616b" containerName="istio-proxy"
Oct 30 22:34:15.259712 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:15.259689    5510 cpu_manager.go:414] "containerMap: removing container" podUID="21c0fef2-49cd-4734-b402-f1cdeb031ac8" containerName="istio-proxy"
Oct 30 22:34:15.259712 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:15.259693    5510 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="21c0fef2-49cd-4734-b402-f1cdeb031ac8" containerName="istio-proxy"
Oct 30 22:34:15.260245 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:15.259793    5510 memory_manager.go:365] "RemoveStaleState removing state" podUID="21557bdf-26e4-4a0c-ad9f-50dcc391616b" containerName="istio-proxy"
Oct 30 22:34:15.260245 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:15.259801    5510 memory_manager.go:365] "RemoveStaleState removing state" podUID="21c0fef2-49cd-4734-b402-f1cdeb031ac8" containerName="istio-proxy"
Oct 30 22:34:15.260245 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:15.259809    5510 memory_manager.go:365] "RemoveStaleState removing state" podUID="afc34d05-1306-4b04-8c81-f123a51f682c" containerName="maas-api"
Oct 30 22:34:15.260245 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:15.259816    5510 memory_manager.go:365] "RemoveStaleState removing state" podUID="3a76b5b9-88cf-4f7f-9d14-fd725017b62f" containerName="istio-proxy"
Oct 30 22:34:15.260245 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:15.259823    5510 memory_manager.go:365] "RemoveStaleState removing state" podUID="f8bd673e-4938-421a-bb40-5eb3e7f7802f" containerName="maas-api"
Oct 30 22:34:15.260245 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:15.259834    5510 memory_manager.go:365] "RemoveStaleState removing state" podUID="3a76b5b9-88cf-4f7f-9d14-fd725017b62f" containerName="istio-proxy"
Oct 30 22:34:15.260245 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:15.259841    5510 memory_manager.go:365] "RemoveStaleState removing state" podUID="42db3ad0-7d90-4073-9ad9-19b6b5590551" containerName="maas-api"
Oct 30 22:34:15.260245 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:15.259846    5510 memory_manager.go:365] "RemoveStaleState removing state" podUID="9e9b11b8-1cc9-4fc8-8097-1afcffa8c422" containerName="main"
Oct 30 22:34:15.260245 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:15.259853    5510 memory_manager.go:365] "RemoveStaleState removing state" podUID="11bc04c5-986f-47f9-93d1-0f803b775eb5" containerName="istio-proxy"
Oct 30 22:34:15.264901 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:15.264625    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-07e9c8/test-model-07e9c8-kserve-9c9cbf4bc-b88r4"
Oct 30 22:34:15.274933 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:15.274906    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-rd56v"]
Oct 30 22:34:15.275142 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:15.275108    5510 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-rd56v" podUID="7e6ab5c7-cadf-4bb4-9dda-68b17a855729" containerName="istio-proxy" containerID="cri-o://16d6941c50b495e8850bd41acef908fbe8e0aea410be304a0a0dbddf41f1fa55" gracePeriod=30
Oct 30 22:34:15.319611 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:15.319573    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-07e9c8\"/\"test-model-07e9c8-kserve-self-signed-certs\""
Oct 30 22:34:15.319791 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:15.319681    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-07e9c8\"/\"default-dockercfg-6952t\""
Oct 30 22:34:15.323083 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:15.323056    5510 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-07e9c8\"/\"openshift-service-ca.crt\""
Oct 30 22:34:15.323310 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:15.323187    5510 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-07e9c8\"/\"kube-root-ca.crt\""
Oct 30 22:34:15.325879 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:15.325854    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-07e9c8/test-model-07e9c8-kserve-9c9cbf4bc-b88r4"]
Oct 30 22:34:15.349117 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:15.349078    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6824a63b-432d-4967-9b83-742599c9cd94-home\") pod \"test-model-07e9c8-kserve-9c9cbf4bc-b88r4\" (UID: \"6824a63b-432d-4967-9b83-742599c9cd94\") " pod="ai-tenant-e2e-inf-b-07e9c8/test-model-07e9c8-kserve-9c9cbf4bc-b88r4"
Oct 30 22:34:15.349263 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:15.349229    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6824a63b-432d-4967-9b83-742599c9cd94-model-cache\") pod \"test-model-07e9c8-kserve-9c9cbf4bc-b88r4\" (UID: \"6824a63b-432d-4967-9b83-742599c9cd94\") " pod="ai-tenant-e2e-inf-b-07e9c8/test-model-07e9c8-kserve-9c9cbf4bc-b88r4"
Oct 30 22:34:15.349330 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:15.349310    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6824a63b-432d-4967-9b83-742599c9cd94-tmp-dir\") pod \"test-model-07e9c8-kserve-9c9cbf4bc-b88r4\" (UID: \"6824a63b-432d-4967-9b83-742599c9cd94\") " pod="ai-tenant-e2e-inf-b-07e9c8/test-model-07e9c8-kserve-9c9cbf4bc-b88r4"
Oct 30 22:34:15.349454 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:15.349428    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6824a63b-432d-4967-9b83-742599c9cd94-dshm\") pod \"test-model-07e9c8-kserve-9c9cbf4bc-b88r4\" (UID: \"6824a63b-432d-4967-9b83-742599c9cd94\") " pod="ai-tenant-e2e-inf-b-07e9c8/test-model-07e9c8-kserve-9c9cbf4bc-b88r4"
Oct 30 22:34:15.349577 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:15.349476    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6824a63b-432d-4967-9b83-742599c9cd94-tls-certs\") pod \"test-model-07e9c8-kserve-9c9cbf4bc-b88r4\" (UID: \"6824a63b-432d-4967-9b83-742599c9cd94\") " pod="ai-tenant-e2e-inf-b-07e9c8/test-model-07e9c8-kserve-9c9cbf4bc-b88r4"
Oct 30 22:34:15.349627 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:15.349613    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p4qt8\" (UniqueName: \"kubernetes.io/projected/6824a63b-432d-4967-9b83-742599c9cd94-kube-api-access-p4qt8\") pod \"test-model-07e9c8-kserve-9c9cbf4bc-b88r4\" (UID: \"6824a63b-432d-4967-9b83-742599c9cd94\") " pod="ai-tenant-e2e-inf-b-07e9c8/test-model-07e9c8-kserve-9c9cbf4bc-b88r4"
Oct 30 22:34:15.391417 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:15.391376    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-779f32/distinct-2-w0-779f32-kserve-857ffdf778-gk2h9"]
Oct 30 22:34:15.399588 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:15.399551    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-779f32/distinct-w0-779f32-kserve-6dcbc948c7-2vzdf"]
Oct 30 22:34:15.445584 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:15.445552    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:34:15.450946 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:15.450919    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6824a63b-432d-4967-9b83-742599c9cd94-home\") pod \"test-model-07e9c8-kserve-9c9cbf4bc-b88r4\" (UID: \"6824a63b-432d-4967-9b83-742599c9cd94\") " pod="ai-tenant-e2e-inf-b-07e9c8/test-model-07e9c8-kserve-9c9cbf4bc-b88r4"
Oct 30 22:34:15.451081 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:15.450971    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6824a63b-432d-4967-9b83-742599c9cd94-model-cache\") pod \"test-model-07e9c8-kserve-9c9cbf4bc-b88r4\" (UID: \"6824a63b-432d-4967-9b83-742599c9cd94\") " pod="ai-tenant-e2e-inf-b-07e9c8/test-model-07e9c8-kserve-9c9cbf4bc-b88r4"
Oct 30 22:34:15.451081 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:15.451011    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6824a63b-432d-4967-9b83-742599c9cd94-tmp-dir\") pod \"test-model-07e9c8-kserve-9c9cbf4bc-b88r4\" (UID: \"6824a63b-432d-4967-9b83-742599c9cd94\") " pod="ai-tenant-e2e-inf-b-07e9c8/test-model-07e9c8-kserve-9c9cbf4bc-b88r4"
Oct 30 22:34:15.451081 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:15.451040    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6824a63b-432d-4967-9b83-742599c9cd94-dshm\") pod \"test-model-07e9c8-kserve-9c9cbf4bc-b88r4\" (UID: \"6824a63b-432d-4967-9b83-742599c9cd94\") " pod="ai-tenant-e2e-inf-b-07e9c8/test-model-07e9c8-kserve-9c9cbf4bc-b88r4"
Oct 30 22:34:15.451278 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:15.451171    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6824a63b-432d-4967-9b83-742599c9cd94-tls-certs\") pod \"test-model-07e9c8-kserve-9c9cbf4bc-b88r4\" (UID: \"6824a63b-432d-4967-9b83-742599c9cd94\") " pod="ai-tenant-e2e-inf-b-07e9c8/test-model-07e9c8-kserve-9c9cbf4bc-b88r4"
Oct 30 22:34:15.451336 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:15.451301    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p4qt8\" (UniqueName: \"kubernetes.io/projected/6824a63b-432d-4967-9b83-742599c9cd94-kube-api-access-p4qt8\") pod \"test-model-07e9c8-kserve-9c9cbf4bc-b88r4\" (UID: \"6824a63b-432d-4967-9b83-742599c9cd94\") " pod="ai-tenant-e2e-inf-b-07e9c8/test-model-07e9c8-kserve-9c9cbf4bc-b88r4"
Oct 30 22:34:15.451398 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:15.451373    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6824a63b-432d-4967-9b83-742599c9cd94-home\") pod \"test-model-07e9c8-kserve-9c9cbf4bc-b88r4\" (UID: \"6824a63b-432d-4967-9b83-742599c9cd94\") " pod="ai-tenant-e2e-inf-b-07e9c8/test-model-07e9c8-kserve-9c9cbf4bc-b88r4"
Oct 30 22:34:15.451467 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:15.451405    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6824a63b-432d-4967-9b83-742599c9cd94-tmp-dir\") pod \"test-model-07e9c8-kserve-9c9cbf4bc-b88r4\" (UID: \"6824a63b-432d-4967-9b83-742599c9cd94\") " pod="ai-tenant-e2e-inf-b-07e9c8/test-model-07e9c8-kserve-9c9cbf4bc-b88r4"
Oct 30 22:34:15.451522 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:15.451467    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6824a63b-432d-4967-9b83-742599c9cd94-model-cache\") pod \"test-model-07e9c8-kserve-9c9cbf4bc-b88r4\" (UID: \"6824a63b-432d-4967-9b83-742599c9cd94\") " pod="ai-tenant-e2e-inf-b-07e9c8/test-model-07e9c8-kserve-9c9cbf4bc-b88r4"
Oct 30 22:34:15.453386 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:15.453358    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6824a63b-432d-4967-9b83-742599c9cd94-dshm\") pod \"test-model-07e9c8-kserve-9c9cbf4bc-b88r4\" (UID: \"6824a63b-432d-4967-9b83-742599c9cd94\") " pod="ai-tenant-e2e-inf-b-07e9c8/test-model-07e9c8-kserve-9c9cbf4bc-b88r4"
Oct 30 22:34:15.453642 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:15.453624    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6824a63b-432d-4967-9b83-742599c9cd94-tls-certs\") pod \"test-model-07e9c8-kserve-9c9cbf4bc-b88r4\" (UID: \"6824a63b-432d-4967-9b83-742599c9cd94\") " pod="ai-tenant-e2e-inf-b-07e9c8/test-model-07e9c8-kserve-9c9cbf4bc-b88r4"
Oct 30 22:34:15.510404 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:15.510368    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p4qt8\" (UniqueName: \"kubernetes.io/projected/6824a63b-432d-4967-9b83-742599c9cd94-kube-api-access-p4qt8\") pod \"test-model-07e9c8-kserve-9c9cbf4bc-b88r4\" (UID: \"6824a63b-432d-4967-9b83-742599c9cd94\") " pod="ai-tenant-e2e-inf-b-07e9c8/test-model-07e9c8-kserve-9c9cbf4bc-b88r4"
Oct 30 22:34:15.577851 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:15.577817    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-07e9c8/test-model-07e9c8-kserve-9c9cbf4bc-b88r4"
Oct 30 22:34:15.670997 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:15.670958    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:34:15.957522 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:15.957494    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-07e9c8/test-model-07e9c8-kserve-9c9cbf4bc-b88r4"]
Oct 30 22:34:15.959837 ip-10-0-136-210 kubenswrapper[5510]: W1030 22:34:15.959809    5510 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6824a63b_432d_4967_9b83_742599c9cd94.slice/crio-49398718e4308e147ff5582677d084ded61a71967339a97760725ba46917c3e3 WatchSource:0}: Error finding container 49398718e4308e147ff5582677d084ded61a71967339a97760725ba46917c3e3: Status 404 returned error can't find the container with id 49398718e4308e147ff5582677d084ded61a71967339a97760725ba46917c3e3
Oct 30 22:34:16.202165 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:16.202124    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-07e9c8/test-model-07e9c8-kserve-9c9cbf4bc-b88r4" event={"ID":"6824a63b-432d-4967-9b83-742599c9cd94","Type":"ContainerStarted","Data":"ff16cfb1db3109775a5076d569bd19166470c44b7ef63f2d8bc5828f6cada460"}
Oct 30 22:34:16.202595 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:16.202180    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-07e9c8/test-model-07e9c8-kserve-9c9cbf4bc-b88r4" event={"ID":"6824a63b-432d-4967-9b83-742599c9cd94","Type":"ContainerStarted","Data":"49398718e4308e147ff5582677d084ded61a71967339a97760725ba46917c3e3"}
Oct 30 22:34:16.315870 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:16.315812    5510 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-07e9c8/test-model-07e9c8-kserve-9c9cbf4bc-b88r4" podStartSLOduration=1.315798722 podStartE2EDuration="1.315798722s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:34:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:34:16.314041315 +0000 UTC m=+2636.874851322" watchObservedRunningTime="2026-10-30 22:34:16.315798722 +0000 UTC m=+2636.876608750"
Oct 30 22:34:16.578061 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:16.577978    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-07e9c8/test-model-07e9c8-kserve-9c9cbf4bc-b88r4"
Oct 30 22:34:17.655062 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:17.655026    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:34:18.824685 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:18.824624    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:34:20.529792 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:20.529763    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-rd56v"
Oct 30 22:34:20.625937 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:20.625839    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7e6ab5c7-cadf-4bb4-9dda-68b17a855729-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e6ab5c7-cadf-4bb4-9dda-68b17a855729-workload-socket\") pod \"7e6ab5c7-cadf-4bb4-9dda-68b17a855729\" (UID: \"7e6ab5c7-cadf-4bb4-9dda-68b17a855729\") "
Oct 30 22:34:20.625937 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:20.625923    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7e6ab5c7-cadf-4bb4-9dda-68b17a855729-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e6ab5c7-cadf-4bb4-9dda-68b17a855729-credential-socket\") pod \"7e6ab5c7-cadf-4bb4-9dda-68b17a855729\" (UID: \"7e6ab5c7-cadf-4bb4-9dda-68b17a855729\") "
Oct 30 22:34:20.626188 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:20.625960    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7e6ab5c7-cadf-4bb4-9dda-68b17a855729-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7e6ab5c7-cadf-4bb4-9dda-68b17a855729-istio-data\") pod \"7e6ab5c7-cadf-4bb4-9dda-68b17a855729\" (UID: \"7e6ab5c7-cadf-4bb4-9dda-68b17a855729\") "
Oct 30 22:34:20.626188 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:20.626000    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7e6ab5c7-cadf-4bb4-9dda-68b17a855729-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7e6ab5c7-cadf-4bb4-9dda-68b17a855729-istiod-ca-cert\") pod \"7e6ab5c7-cadf-4bb4-9dda-68b17a855729\" (UID: \"7e6ab5c7-cadf-4bb4-9dda-68b17a855729\") "
Oct 30 22:34:20.626188 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:20.626091    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/7e6ab5c7-cadf-4bb4-9dda-68b17a855729-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7e6ab5c7-cadf-4bb4-9dda-68b17a855729-istio-podinfo\") pod \"7e6ab5c7-cadf-4bb4-9dda-68b17a855729\" (UID: \"7e6ab5c7-cadf-4bb4-9dda-68b17a855729\") "
Oct 30 22:34:20.626188 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:20.626129    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7e6ab5c7-cadf-4bb4-9dda-68b17a855729-istio-token\" (UniqueName: \"kubernetes.io/projected/7e6ab5c7-cadf-4bb4-9dda-68b17a855729-istio-token\") pod \"7e6ab5c7-cadf-4bb4-9dda-68b17a855729\" (UID: \"7e6ab5c7-cadf-4bb4-9dda-68b17a855729\") "
Oct 30 22:34:20.626188 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:20.626162    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7e6ab5c7-cadf-4bb4-9dda-68b17a855729-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7e6ab5c7-cadf-4bb4-9dda-68b17a855729-workload-certs\") pod \"7e6ab5c7-cadf-4bb4-9dda-68b17a855729\" (UID: \"7e6ab5c7-cadf-4bb4-9dda-68b17a855729\") "
Oct 30 22:34:20.626443 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:20.626196    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7e6ab5c7-cadf-4bb4-9dda-68b17a855729-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7e6ab5c7-cadf-4bb4-9dda-68b17a855729-istio-envoy\") pod \"7e6ab5c7-cadf-4bb4-9dda-68b17a855729\" (UID: \"7e6ab5c7-cadf-4bb4-9dda-68b17a855729\") "
Oct 30 22:34:20.626443 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:20.626127    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e6ab5c7-cadf-4bb4-9dda-68b17a855729-workload-socket" pod "7e6ab5c7-cadf-4bb4-9dda-68b17a855729" (UID: "7e6ab5c7-cadf-4bb4-9dda-68b17a855729"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:34:20.626443 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:20.626214    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e6ab5c7-cadf-4bb4-9dda-68b17a855729-istio-data" pod "7e6ab5c7-cadf-4bb4-9dda-68b17a855729" (UID: "7e6ab5c7-cadf-4bb4-9dda-68b17a855729"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:34:20.626443 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:20.626298    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e6ab5c7-cadf-4bb4-9dda-68b17a855729-credential-socket" pod "7e6ab5c7-cadf-4bb4-9dda-68b17a855729" (UID: "7e6ab5c7-cadf-4bb4-9dda-68b17a855729"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:34:20.626443 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:20.626319    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7e6ab5c7-cadf-4bb4-9dda-68b17a855729-kube-api-access-dxqkw\" (UniqueName: \"kubernetes.io/projected/7e6ab5c7-cadf-4bb4-9dda-68b17a855729-kube-api-access-dxqkw\") pod \"7e6ab5c7-cadf-4bb4-9dda-68b17a855729\" (UID: \"7e6ab5c7-cadf-4bb4-9dda-68b17a855729\") "
Oct 30 22:34:20.626443 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:20.626408    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e6ab5c7-cadf-4bb4-9dda-68b17a855729-workload-certs" pod "7e6ab5c7-cadf-4bb4-9dda-68b17a855729" (UID: "7e6ab5c7-cadf-4bb4-9dda-68b17a855729"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:34:20.626863 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:20.626842    5510 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e6ab5c7-cadf-4bb4-9dda-68b17a855729-workload-socket\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:34:20.626943 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:20.626868    5510 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e6ab5c7-cadf-4bb4-9dda-68b17a855729-credential-socket\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:34:20.626943 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:20.626883    5510 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7e6ab5c7-cadf-4bb4-9dda-68b17a855729-istio-data\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:34:20.626943 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:20.626887    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7e6ab5c7-cadf-4bb4-9dda-68b17a855729-istiod-ca-cert" pod "7e6ab5c7-cadf-4bb4-9dda-68b17a855729" (UID: "7e6ab5c7-cadf-4bb4-9dda-68b17a855729"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:34:20.626943 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:20.626899    5510 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7e6ab5c7-cadf-4bb4-9dda-68b17a855729-workload-certs\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:34:20.628482 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:20.628454    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7e6ab5c7-cadf-4bb4-9dda-68b17a855729-istio-token" pod "7e6ab5c7-cadf-4bb4-9dda-68b17a855729" (UID: "7e6ab5c7-cadf-4bb4-9dda-68b17a855729"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:34:20.628805 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:20.628787    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7e6ab5c7-cadf-4bb4-9dda-68b17a855729-kube-api-access-dxqkw" pod "7e6ab5c7-cadf-4bb4-9dda-68b17a855729" (UID: "7e6ab5c7-cadf-4bb4-9dda-68b17a855729"). InnerVolumeSpecName "kube-api-access-dxqkw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:34:20.628911 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:20.628893    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e6ab5c7-cadf-4bb4-9dda-68b17a855729-istio-envoy" pod "7e6ab5c7-cadf-4bb4-9dda-68b17a855729" (UID: "7e6ab5c7-cadf-4bb4-9dda-68b17a855729"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:34:20.629268 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:20.629251    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7e6ab5c7-cadf-4bb4-9dda-68b17a855729-istio-podinfo" pod "7e6ab5c7-cadf-4bb4-9dda-68b17a855729" (UID: "7e6ab5c7-cadf-4bb4-9dda-68b17a855729"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:34:20.728673 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:20.728622    5510 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7e6ab5c7-cadf-4bb4-9dda-68b17a855729-istiod-ca-cert\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:34:20.728673 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:20.728678    5510 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7e6ab5c7-cadf-4bb4-9dda-68b17a855729-istio-podinfo\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:34:20.728916 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:20.728690    5510 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7e6ab5c7-cadf-4bb4-9dda-68b17a855729-istio-token\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:34:20.728916 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:20.728699    5510 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7e6ab5c7-cadf-4bb4-9dda-68b17a855729-istio-envoy\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:34:20.728916 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:20.728709    5510 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dxqkw\" (UniqueName: \"kubernetes.io/projected/7e6ab5c7-cadf-4bb4-9dda-68b17a855729-kube-api-access-dxqkw\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:34:21.233878 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:21.233841    5510 generic.go:356] "Generic (PLEG): container finished" podID="7e6ab5c7-cadf-4bb4-9dda-68b17a855729" containerID="16d6941c50b495e8850bd41acef908fbe8e0aea410be304a0a0dbddf41f1fa55" exitCode=0
Oct 30 22:34:21.234049 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:21.233890    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-rd56v"
Oct 30 22:34:21.234049 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:21.233935    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-rd56v" event={"ID":"7e6ab5c7-cadf-4bb4-9dda-68b17a855729","Type":"ContainerDied","Data":"16d6941c50b495e8850bd41acef908fbe8e0aea410be304a0a0dbddf41f1fa55"}
Oct 30 22:34:21.234049 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:21.233988    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-rd56v" event={"ID":"7e6ab5c7-cadf-4bb4-9dda-68b17a855729","Type":"ContainerDied","Data":"bab49d066a60965f765409d587bce7c2868d6a057479a235a7c04a5b284ec2a6"}
Oct 30 22:34:21.234049 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:21.234009    5510 scope.go:122] "RemoveContainer" containerID="16d6941c50b495e8850bd41acef908fbe8e0aea410be304a0a0dbddf41f1fa55"
Oct 30 22:34:21.247128 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:21.247103    5510 scope.go:122] "RemoveContainer" containerID="16d6941c50b495e8850bd41acef908fbe8e0aea410be304a0a0dbddf41f1fa55"
Oct 30 22:34:21.247412 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:34:21.247392    5510 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"16d6941c50b495e8850bd41acef908fbe8e0aea410be304a0a0dbddf41f1fa55\": container with ID starting with 16d6941c50b495e8850bd41acef908fbe8e0aea410be304a0a0dbddf41f1fa55 not found: ID does not exist" containerID="16d6941c50b495e8850bd41acef908fbe8e0aea410be304a0a0dbddf41f1fa55"
Oct 30 22:34:21.247484 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:21.247424    5510 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"16d6941c50b495e8850bd41acef908fbe8e0aea410be304a0a0dbddf41f1fa55"} err="failed to get container status \"16d6941c50b495e8850bd41acef908fbe8e0aea410be304a0a0dbddf41f1fa55\": rpc error: code = NotFound desc = could not find container \"16d6941c50b495e8850bd41acef908fbe8e0aea410be304a0a0dbddf41f1fa55\": container with ID starting with 16d6941c50b495e8850bd41acef908fbe8e0aea410be304a0a0dbddf41f1fa55 not found: ID does not exist"
Oct 30 22:34:21.439566 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:21.439534    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-rd56v"]
Oct 30 22:34:21.486209 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:21.486121    5510 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-bc714d-openshift-default-67558b95cf-rd56v"]
Oct 30 22:34:21.554058 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:21.554026    5510 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7e6ab5c7-cadf-4bb4-9dda-68b17a855729" path="/var/lib/kubelet/pods/7e6ab5c7-cadf-4bb4-9dda-68b17a855729/volumes"
Oct 30 22:34:22.051203 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:22.051162    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-779f32-67bbb7bf95-sjnl9"]
Oct 30 22:34:22.283165 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:22.283126    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:34:22.503319 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:22.503206    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:34:25.578172 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:25.578131    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-07e9c8/test-model-07e9c8-kserve-9c9cbf4bc-b88r4"
Oct 30 22:34:25.591084 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:25.591059    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-07e9c8/test-model-07e9c8-kserve-9c9cbf4bc-b88r4"
Oct 30 22:34:26.273747 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:26.273717    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-07e9c8/test-model-07e9c8-kserve-9c9cbf4bc-b88r4"
Oct 30 22:34:26.981022 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:26.980988    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:34:27.051691 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:27.051633    5510 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-779f32-67bbb7bf95-sjnl9" podUID="4fd056ae-943a-43c1-8682-8edb3fe3b5ed" containerName="payload-pre-processing" containerID="cri-o://a46ab953c0354b4a66225f2364b7f14057e5153277721e749c2479e31b77ee2f" gracePeriod=25
Oct 30 22:34:30.413761 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:30.413722    5510 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-779f32/distinct-2-w0-779f32-kserve-857ffdf778-gk2h9" podUID="ef500cdd-cc52-45b0-8dd7-1896897acc46" containerName="main" containerID="cri-o://9767c12917b25a2f77eb31d59b982a4c5b1b851c6d51350581ef02f47249de27" gracePeriod=45
Oct 30 22:34:30.419340 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:30.419305    5510 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-779f32/distinct-w0-779f32-kserve-6dcbc948c7-2vzdf" podUID="e17cedd4-7d48-42b3-b127-b6192c1b3795" containerName="main" containerID="cri-o://f7de5feaffd118764ebef90ea7f126c3c8b7974d6a32da832873599fffae0343" gracePeriod=45
Oct 30 22:34:30.697680 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:30.697638    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-779f32/distinct-2-w0-779f32-kserve-857ffdf778-gk2h9"
Oct 30 22:34:30.727671 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:30.727635    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-779f32/distinct-w0-779f32-kserve-6dcbc948c7-2vzdf"
Oct 30 22:34:30.855413 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:30.855373    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e17cedd4-7d48-42b3-b127-b6192c1b3795-tls-certs\" (UniqueName: \"kubernetes.io/secret/e17cedd4-7d48-42b3-b127-b6192c1b3795-tls-certs\") pod \"e17cedd4-7d48-42b3-b127-b6192c1b3795\" (UID: \"e17cedd4-7d48-42b3-b127-b6192c1b3795\") "
Oct 30 22:34:30.855604 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:30.855420    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ef500cdd-cc52-45b0-8dd7-1896897acc46-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ef500cdd-cc52-45b0-8dd7-1896897acc46-tmp-dir\") pod \"ef500cdd-cc52-45b0-8dd7-1896897acc46\" (UID: \"ef500cdd-cc52-45b0-8dd7-1896897acc46\") "
Oct 30 22:34:30.855604 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:30.855465    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e17cedd4-7d48-42b3-b127-b6192c1b3795-home\" (UniqueName: \"kubernetes.io/empty-dir/e17cedd4-7d48-42b3-b127-b6192c1b3795-home\") pod \"e17cedd4-7d48-42b3-b127-b6192c1b3795\" (UID: \"e17cedd4-7d48-42b3-b127-b6192c1b3795\") "
Oct 30 22:34:30.855604 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:30.855516    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e17cedd4-7d48-42b3-b127-b6192c1b3795-dshm\" (UniqueName: \"kubernetes.io/empty-dir/e17cedd4-7d48-42b3-b127-b6192c1b3795-dshm\") pod \"e17cedd4-7d48-42b3-b127-b6192c1b3795\" (UID: \"e17cedd4-7d48-42b3-b127-b6192c1b3795\") "
Oct 30 22:34:30.855604 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:30.855566    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ef500cdd-cc52-45b0-8dd7-1896897acc46-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ef500cdd-cc52-45b0-8dd7-1896897acc46-model-cache\") pod \"ef500cdd-cc52-45b0-8dd7-1896897acc46\" (UID: \"ef500cdd-cc52-45b0-8dd7-1896897acc46\") "
Oct 30 22:34:30.855604 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:30.855598    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e17cedd4-7d48-42b3-b127-b6192c1b3795-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e17cedd4-7d48-42b3-b127-b6192c1b3795-model-cache\") pod \"e17cedd4-7d48-42b3-b127-b6192c1b3795\" (UID: \"e17cedd4-7d48-42b3-b127-b6192c1b3795\") "
Oct 30 22:34:30.855911 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:30.855680    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e17cedd4-7d48-42b3-b127-b6192c1b3795-kube-api-access-6rmpw\" (UniqueName: \"kubernetes.io/projected/e17cedd4-7d48-42b3-b127-b6192c1b3795-kube-api-access-6rmpw\") pod \"e17cedd4-7d48-42b3-b127-b6192c1b3795\" (UID: \"e17cedd4-7d48-42b3-b127-b6192c1b3795\") "
Oct 30 22:34:30.855911 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:30.855713    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ef500cdd-cc52-45b0-8dd7-1896897acc46-dshm\" (UniqueName: \"kubernetes.io/empty-dir/ef500cdd-cc52-45b0-8dd7-1896897acc46-dshm\") pod \"ef500cdd-cc52-45b0-8dd7-1896897acc46\" (UID: \"ef500cdd-cc52-45b0-8dd7-1896897acc46\") "
Oct 30 22:34:30.855911 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:30.855754    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ef500cdd-cc52-45b0-8dd7-1896897acc46-kube-api-access-q8dnw\" (UniqueName: \"kubernetes.io/projected/ef500cdd-cc52-45b0-8dd7-1896897acc46-kube-api-access-q8dnw\") pod \"ef500cdd-cc52-45b0-8dd7-1896897acc46\" (UID: \"ef500cdd-cc52-45b0-8dd7-1896897acc46\") "
Oct 30 22:34:30.855911 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:30.855768    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ef500cdd-cc52-45b0-8dd7-1896897acc46-tmp-dir" pod "ef500cdd-cc52-45b0-8dd7-1896897acc46" (UID: "ef500cdd-cc52-45b0-8dd7-1896897acc46"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:34:30.855911 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:30.855767    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e17cedd4-7d48-42b3-b127-b6192c1b3795-home" pod "e17cedd4-7d48-42b3-b127-b6192c1b3795" (UID: "e17cedd4-7d48-42b3-b127-b6192c1b3795"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:34:30.855911 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:30.855895    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ef500cdd-cc52-45b0-8dd7-1896897acc46-home\" (UniqueName: \"kubernetes.io/empty-dir/ef500cdd-cc52-45b0-8dd7-1896897acc46-home\") pod \"ef500cdd-cc52-45b0-8dd7-1896897acc46\" (UID: \"ef500cdd-cc52-45b0-8dd7-1896897acc46\") "
Oct 30 22:34:30.855911 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:30.855889    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ef500cdd-cc52-45b0-8dd7-1896897acc46-model-cache" pod "ef500cdd-cc52-45b0-8dd7-1896897acc46" (UID: "ef500cdd-cc52-45b0-8dd7-1896897acc46"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:34:30.856251 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:30.855931    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ef500cdd-cc52-45b0-8dd7-1896897acc46-tls-certs\" (UniqueName: \"kubernetes.io/secret/ef500cdd-cc52-45b0-8dd7-1896897acc46-tls-certs\") pod \"ef500cdd-cc52-45b0-8dd7-1896897acc46\" (UID: \"ef500cdd-cc52-45b0-8dd7-1896897acc46\") "
Oct 30 22:34:30.856251 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:30.855995    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e17cedd4-7d48-42b3-b127-b6192c1b3795-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e17cedd4-7d48-42b3-b127-b6192c1b3795-tmp-dir\") pod \"e17cedd4-7d48-42b3-b127-b6192c1b3795\" (UID: \"e17cedd4-7d48-42b3-b127-b6192c1b3795\") "
Oct 30 22:34:30.856251 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:30.856010    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e17cedd4-7d48-42b3-b127-b6192c1b3795-model-cache" pod "e17cedd4-7d48-42b3-b127-b6192c1b3795" (UID: "e17cedd4-7d48-42b3-b127-b6192c1b3795"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:34:30.856251 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:30.856158    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ef500cdd-cc52-45b0-8dd7-1896897acc46-home" pod "ef500cdd-cc52-45b0-8dd7-1896897acc46" (UID: "ef500cdd-cc52-45b0-8dd7-1896897acc46"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:34:30.856540 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:30.856513    5510 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ef500cdd-cc52-45b0-8dd7-1896897acc46-home\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:34:30.856540 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:30.856538    5510 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ef500cdd-cc52-45b0-8dd7-1896897acc46-tmp-dir\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:34:30.856706 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:30.856552    5510 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e17cedd4-7d48-42b3-b127-b6192c1b3795-home\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:34:30.856706 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:30.856564    5510 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ef500cdd-cc52-45b0-8dd7-1896897acc46-model-cache\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:34:30.856706 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:30.856576    5510 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e17cedd4-7d48-42b3-b127-b6192c1b3795-model-cache\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:34:30.856706 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:30.856586    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e17cedd4-7d48-42b3-b127-b6192c1b3795-tmp-dir" pod "e17cedd4-7d48-42b3-b127-b6192c1b3795" (UID: "e17cedd4-7d48-42b3-b127-b6192c1b3795"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:34:30.858436 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:30.858413    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e17cedd4-7d48-42b3-b127-b6192c1b3795-dshm" pod "e17cedd4-7d48-42b3-b127-b6192c1b3795" (UID: "e17cedd4-7d48-42b3-b127-b6192c1b3795"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:34:30.858554 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:30.858436    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e17cedd4-7d48-42b3-b127-b6192c1b3795-kube-api-access-6rmpw" pod "e17cedd4-7d48-42b3-b127-b6192c1b3795" (UID: "e17cedd4-7d48-42b3-b127-b6192c1b3795"). InnerVolumeSpecName "kube-api-access-6rmpw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:34:30.858554 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:30.858486    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ef500cdd-cc52-45b0-8dd7-1896897acc46-tls-certs" pod "ef500cdd-cc52-45b0-8dd7-1896897acc46" (UID: "ef500cdd-cc52-45b0-8dd7-1896897acc46"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:34:30.858959 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:30.858905    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ef500cdd-cc52-45b0-8dd7-1896897acc46-kube-api-access-q8dnw" pod "ef500cdd-cc52-45b0-8dd7-1896897acc46" (UID: "ef500cdd-cc52-45b0-8dd7-1896897acc46"). InnerVolumeSpecName "kube-api-access-q8dnw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:34:30.859346 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:30.859319    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e17cedd4-7d48-42b3-b127-b6192c1b3795-tls-certs" pod "e17cedd4-7d48-42b3-b127-b6192c1b3795" (UID: "e17cedd4-7d48-42b3-b127-b6192c1b3795"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:34:30.859459 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:30.859316    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ef500cdd-cc52-45b0-8dd7-1896897acc46-dshm" pod "ef500cdd-cc52-45b0-8dd7-1896897acc46" (UID: "ef500cdd-cc52-45b0-8dd7-1896897acc46"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:34:30.958186 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:30.958115    5510 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e17cedd4-7d48-42b3-b127-b6192c1b3795-tls-certs\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:34:30.958186 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:30.958144    5510 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e17cedd4-7d48-42b3-b127-b6192c1b3795-dshm\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:34:30.958186 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:30.958157    5510 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6rmpw\" (UniqueName: \"kubernetes.io/projected/e17cedd4-7d48-42b3-b127-b6192c1b3795-kube-api-access-6rmpw\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:34:30.958186 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:30.958167    5510 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ef500cdd-cc52-45b0-8dd7-1896897acc46-dshm\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:34:30.958186 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:30.958176    5510 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q8dnw\" (UniqueName: \"kubernetes.io/projected/ef500cdd-cc52-45b0-8dd7-1896897acc46-kube-api-access-q8dnw\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:34:30.958186 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:30.958185    5510 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ef500cdd-cc52-45b0-8dd7-1896897acc46-tls-certs\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:34:30.958564 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:30.958193    5510 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e17cedd4-7d48-42b3-b127-b6192c1b3795-tmp-dir\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:34:30.992995 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:30.992963    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-779f32-67bbb7bf95-sjnl9" podUID="4fd056ae-943a-43c1-8682-8edb3fe3b5ed" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:34:31.294235 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:31.294198    5510 generic.go:356] "Generic (PLEG): container finished" podID="ef500cdd-cc52-45b0-8dd7-1896897acc46" containerID="9767c12917b25a2f77eb31d59b982a4c5b1b851c6d51350581ef02f47249de27" exitCode=0
Oct 30 22:34:31.294441 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:31.294250    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-779f32/distinct-2-w0-779f32-kserve-857ffdf778-gk2h9"
Oct 30 22:34:31.294441 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:31.294283    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-779f32/distinct-2-w0-779f32-kserve-857ffdf778-gk2h9" event={"ID":"ef500cdd-cc52-45b0-8dd7-1896897acc46","Type":"ContainerDied","Data":"9767c12917b25a2f77eb31d59b982a4c5b1b851c6d51350581ef02f47249de27"}
Oct 30 22:34:31.294441 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:31.294326    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-779f32/distinct-2-w0-779f32-kserve-857ffdf778-gk2h9" event={"ID":"ef500cdd-cc52-45b0-8dd7-1896897acc46","Type":"ContainerDied","Data":"a919144f88571a79b3353edeeb74e17fe00197b109946f8c4aea2a6eddabe57f"}
Oct 30 22:34:31.294441 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:31.294347    5510 scope.go:122] "RemoveContainer" containerID="9767c12917b25a2f77eb31d59b982a4c5b1b851c6d51350581ef02f47249de27"
Oct 30 22:34:31.296511 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:31.296483    5510 generic.go:356] "Generic (PLEG): container finished" podID="e17cedd4-7d48-42b3-b127-b6192c1b3795" containerID="f7de5feaffd118764ebef90ea7f126c3c8b7974d6a32da832873599fffae0343" exitCode=0
Oct 30 22:34:31.296637 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:31.296541    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-779f32/distinct-w0-779f32-kserve-6dcbc948c7-2vzdf"
Oct 30 22:34:31.296637 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:31.296539    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-779f32/distinct-w0-779f32-kserve-6dcbc948c7-2vzdf" event={"ID":"e17cedd4-7d48-42b3-b127-b6192c1b3795","Type":"ContainerDied","Data":"f7de5feaffd118764ebef90ea7f126c3c8b7974d6a32da832873599fffae0343"}
Oct 30 22:34:31.296785 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:31.296686    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-779f32/distinct-w0-779f32-kserve-6dcbc948c7-2vzdf" event={"ID":"e17cedd4-7d48-42b3-b127-b6192c1b3795","Type":"ContainerDied","Data":"58a9e3cfb38d27bbd767479bde3692688592ed2c5076102fd005a3bfabd2ae9c"}
Oct 30 22:34:31.305989 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:31.305972    5510 scope.go:122] "RemoveContainer" containerID="9767c12917b25a2f77eb31d59b982a4c5b1b851c6d51350581ef02f47249de27"
Oct 30 22:34:31.306282 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:34:31.306262    5510 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9767c12917b25a2f77eb31d59b982a4c5b1b851c6d51350581ef02f47249de27\": container with ID starting with 9767c12917b25a2f77eb31d59b982a4c5b1b851c6d51350581ef02f47249de27 not found: ID does not exist" containerID="9767c12917b25a2f77eb31d59b982a4c5b1b851c6d51350581ef02f47249de27"
Oct 30 22:34:31.306373 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:31.306290    5510 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9767c12917b25a2f77eb31d59b982a4c5b1b851c6d51350581ef02f47249de27"} err="failed to get container status \"9767c12917b25a2f77eb31d59b982a4c5b1b851c6d51350581ef02f47249de27\": rpc error: code = NotFound desc = could not find container \"9767c12917b25a2f77eb31d59b982a4c5b1b851c6d51350581ef02f47249de27\": container with ID starting with 9767c12917b25a2f77eb31d59b982a4c5b1b851c6d51350581ef02f47249de27 not found: ID does not exist"
Oct 30 22:34:31.306373 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:31.306307    5510 scope.go:122] "RemoveContainer" containerID="f7de5feaffd118764ebef90ea7f126c3c8b7974d6a32da832873599fffae0343"
Oct 30 22:34:31.314953 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:31.314936    5510 scope.go:122] "RemoveContainer" containerID="f7de5feaffd118764ebef90ea7f126c3c8b7974d6a32da832873599fffae0343"
Oct 30 22:34:31.315241 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:34:31.315221    5510 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f7de5feaffd118764ebef90ea7f126c3c8b7974d6a32da832873599fffae0343\": container with ID starting with f7de5feaffd118764ebef90ea7f126c3c8b7974d6a32da832873599fffae0343 not found: ID does not exist" containerID="f7de5feaffd118764ebef90ea7f126c3c8b7974d6a32da832873599fffae0343"
Oct 30 22:34:31.315316 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:31.315251    5510 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f7de5feaffd118764ebef90ea7f126c3c8b7974d6a32da832873599fffae0343"} err="failed to get container status \"f7de5feaffd118764ebef90ea7f126c3c8b7974d6a32da832873599fffae0343\": rpc error: code = NotFound desc = could not find container \"f7de5feaffd118764ebef90ea7f126c3c8b7974d6a32da832873599fffae0343\": container with ID starting with f7de5feaffd118764ebef90ea7f126c3c8b7974d6a32da832873599fffae0343 not found: ID does not exist"
Oct 30 22:34:31.455768 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:31.455734    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-779f32/distinct-w0-779f32-kserve-6dcbc948c7-2vzdf"]
Oct 30 22:34:31.458046 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:31.458022    5510 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-779f32/distinct-w0-779f32-kserve-6dcbc948c7-2vzdf"]
Oct 30 22:34:31.514756 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:31.514699    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:34:31.552838 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:31.552680    5510 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e17cedd4-7d48-42b3-b127-b6192c1b3795" path="/var/lib/kubelet/pods/e17cedd4-7d48-42b3-b127-b6192c1b3795/volumes"
Oct 30 22:34:31.562675 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:31.562638    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-779f32/distinct-2-w0-779f32-kserve-857ffdf778-gk2h9"]
Oct 30 22:34:31.567472 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:31.567440    5510 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-779f32/distinct-2-w0-779f32-kserve-857ffdf778-gk2h9"]
Oct 30 22:34:31.623080 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:31.623043    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-779f32/distinct-2-w0-779f32-kserve-857ffdf778-gk2h9" podUID="ef500cdd-cc52-45b0-8dd7-1896897acc46" containerName="main" probeResult="failure" output="Get \"https://10.133.0.74:8000/ready\": context deadline exceeded"
Oct 30 22:34:33.501486 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:33.501454    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:34:33.553349 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:33.553317    5510 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ef500cdd-cc52-45b0-8dd7-1896897acc46" path="/var/lib/kubelet/pods/ef500cdd-cc52-45b0-8dd7-1896897acc46/volumes"
Oct 30 22:34:36.141189 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:36.141158    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:34:38.420272 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:34:38.420234    5510 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6gjbf" containerName="extractor"
Oct 30 22:34:40.896883 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:40.896844    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:34:40.993408 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:40.993364    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-779f32-67bbb7bf95-sjnl9" podUID="4fd056ae-943a-43c1-8682-8edb3fe3b5ed" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:34:43.561383 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:43.561351    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-lggs2"]
Oct 30 22:34:43.561778 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:43.561536    5510 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-lggs2" podUID="a3a8e626-b7fa-4fe1-8d74-0d8bef87fb63" containerName="istio-proxy" containerID="cri-o://5540c53ae933ff5b3eb69e9f0fb8a7ac5ed9a36279ae4079ea91a6f84f93b83c" gracePeriod=30
Oct 30 22:34:43.685702 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:43.685669    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-f68tv"]
Oct 30 22:34:43.685896 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:43.685875    5510 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-f68tv" podUID="4407af9b-8cf7-4552-9dfc-8324d9832655" containerName="discovery" containerID="cri-o://63bdd7c736f92dca64eb88441088f5d389f31638f5b377962b6df007730106be" gracePeriod=30
Oct 30 22:34:43.951138 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:43.951105    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-f68tv"
Oct 30 22:34:44.103329 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:44.103282    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4407af9b-8cf7-4552-9dfc-8324d9832655-istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/4407af9b-8cf7-4552-9dfc-8324d9832655-istio-csr-dns-cert\") pod \"4407af9b-8cf7-4552-9dfc-8324d9832655\" (UID: \"4407af9b-8cf7-4552-9dfc-8324d9832655\") "
Oct 30 22:34:44.103518 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:44.103390    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4407af9b-8cf7-4552-9dfc-8324d9832655-local-certs\" (UniqueName: \"kubernetes.io/empty-dir/4407af9b-8cf7-4552-9dfc-8324d9832655-local-certs\") pod \"4407af9b-8cf7-4552-9dfc-8324d9832655\" (UID: \"4407af9b-8cf7-4552-9dfc-8324d9832655\") "
Oct 30 22:34:44.103518 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:44.103445    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4407af9b-8cf7-4552-9dfc-8324d9832655-istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/4407af9b-8cf7-4552-9dfc-8324d9832655-istio-kubeconfig\") pod \"4407af9b-8cf7-4552-9dfc-8324d9832655\" (UID: \"4407af9b-8cf7-4552-9dfc-8324d9832655\") "
Oct 30 22:34:44.103518 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:44.103493    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4407af9b-8cf7-4552-9dfc-8324d9832655-istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/4407af9b-8cf7-4552-9dfc-8324d9832655-istio-csr-ca-configmap\") pod \"4407af9b-8cf7-4552-9dfc-8324d9832655\" (UID: \"4407af9b-8cf7-4552-9dfc-8324d9832655\") "
Oct 30 22:34:44.103852 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:44.103531    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4407af9b-8cf7-4552-9dfc-8324d9832655-cacerts\" (UniqueName: \"kubernetes.io/secret/4407af9b-8cf7-4552-9dfc-8324d9832655-cacerts\") pod \"4407af9b-8cf7-4552-9dfc-8324d9832655\" (UID: \"4407af9b-8cf7-4552-9dfc-8324d9832655\") "
Oct 30 22:34:44.103852 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:44.103599    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4407af9b-8cf7-4552-9dfc-8324d9832655-istio-token\" (UniqueName: \"kubernetes.io/projected/4407af9b-8cf7-4552-9dfc-8324d9832655-istio-token\") pod \"4407af9b-8cf7-4552-9dfc-8324d9832655\" (UID: \"4407af9b-8cf7-4552-9dfc-8324d9832655\") "
Oct 30 22:34:44.103852 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:44.103636    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4407af9b-8cf7-4552-9dfc-8324d9832655-kube-api-access-455sr\" (UniqueName: \"kubernetes.io/projected/4407af9b-8cf7-4552-9dfc-8324d9832655-kube-api-access-455sr\") pod \"4407af9b-8cf7-4552-9dfc-8324d9832655\" (UID: \"4407af9b-8cf7-4552-9dfc-8324d9832655\") "
Oct 30 22:34:44.104014 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:44.103927    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4407af9b-8cf7-4552-9dfc-8324d9832655-istio-csr-ca-configmap" pod "4407af9b-8cf7-4552-9dfc-8324d9832655" (UID: "4407af9b-8cf7-4552-9dfc-8324d9832655"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:34:44.104176 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:44.104148    5510 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/4407af9b-8cf7-4552-9dfc-8324d9832655-istio-csr-ca-configmap\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:34:44.106500 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:44.106428    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4407af9b-8cf7-4552-9dfc-8324d9832655-local-certs" pod "4407af9b-8cf7-4552-9dfc-8324d9832655" (UID: "4407af9b-8cf7-4552-9dfc-8324d9832655"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:34:44.106500 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:44.106489    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4407af9b-8cf7-4552-9dfc-8324d9832655-istio-token" pod "4407af9b-8cf7-4552-9dfc-8324d9832655" (UID: "4407af9b-8cf7-4552-9dfc-8324d9832655"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:34:44.106755 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:44.106597    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4407af9b-8cf7-4552-9dfc-8324d9832655-istio-csr-dns-cert" pod "4407af9b-8cf7-4552-9dfc-8324d9832655" (UID: "4407af9b-8cf7-4552-9dfc-8324d9832655"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:34:44.106829 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:44.106768    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4407af9b-8cf7-4552-9dfc-8324d9832655-kube-api-access-455sr" pod "4407af9b-8cf7-4552-9dfc-8324d9832655" (UID: "4407af9b-8cf7-4552-9dfc-8324d9832655"). InnerVolumeSpecName "kube-api-access-455sr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:34:44.107511 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:44.107490    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4407af9b-8cf7-4552-9dfc-8324d9832655-istio-kubeconfig" pod "4407af9b-8cf7-4552-9dfc-8324d9832655" (UID: "4407af9b-8cf7-4552-9dfc-8324d9832655"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:34:44.107604 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:44.107556    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4407af9b-8cf7-4552-9dfc-8324d9832655-cacerts" pod "4407af9b-8cf7-4552-9dfc-8324d9832655" (UID: "4407af9b-8cf7-4552-9dfc-8324d9832655"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:34:44.205010 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:44.204976    5510 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/4407af9b-8cf7-4552-9dfc-8324d9832655-local-certs\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:34:44.205010 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:44.205007    5510 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/4407af9b-8cf7-4552-9dfc-8324d9832655-istio-kubeconfig\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:34:44.205010 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:44.205019    5510 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/4407af9b-8cf7-4552-9dfc-8324d9832655-cacerts\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:34:44.205010 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:44.205027    5510 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4407af9b-8cf7-4552-9dfc-8324d9832655-istio-token\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:34:44.205010 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:44.205037    5510 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-455sr\" (UniqueName: \"kubernetes.io/projected/4407af9b-8cf7-4552-9dfc-8324d9832655-kube-api-access-455sr\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:34:44.205010 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:44.205047    5510 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/4407af9b-8cf7-4552-9dfc-8324d9832655-istio-csr-dns-cert\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:34:44.380811 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:44.380719    5510 generic.go:356] "Generic (PLEG): container finished" podID="4407af9b-8cf7-4552-9dfc-8324d9832655" containerID="63bdd7c736f92dca64eb88441088f5d389f31638f5b377962b6df007730106be" exitCode=0
Oct 30 22:34:44.380811 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:44.380767    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-f68tv"
Oct 30 22:34:44.381017 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:44.380810    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-f68tv" event={"ID":"4407af9b-8cf7-4552-9dfc-8324d9832655","Type":"ContainerDied","Data":"63bdd7c736f92dca64eb88441088f5d389f31638f5b377962b6df007730106be"}
Oct 30 22:34:44.381017 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:44.380860    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-f68tv" event={"ID":"4407af9b-8cf7-4552-9dfc-8324d9832655","Type":"ContainerDied","Data":"d81f15e85249eadce963c305403b8ea501b1d852f49012e6aa4238d90bb37cb9"}
Oct 30 22:34:44.381017 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:44.380888    5510 scope.go:122] "RemoveContainer" containerID="63bdd7c736f92dca64eb88441088f5d389f31638f5b377962b6df007730106be"
Oct 30 22:34:44.395398 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:44.395384    5510 scope.go:122] "RemoveContainer" containerID="63bdd7c736f92dca64eb88441088f5d389f31638f5b377962b6df007730106be"
Oct 30 22:34:44.395775 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:34:44.395755    5510 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"63bdd7c736f92dca64eb88441088f5d389f31638f5b377962b6df007730106be\": container with ID starting with 63bdd7c736f92dca64eb88441088f5d389f31638f5b377962b6df007730106be not found: ID does not exist" containerID="63bdd7c736f92dca64eb88441088f5d389f31638f5b377962b6df007730106be"
Oct 30 22:34:44.395857 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:44.395784    5510 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"63bdd7c736f92dca64eb88441088f5d389f31638f5b377962b6df007730106be"} err="failed to get container status \"63bdd7c736f92dca64eb88441088f5d389f31638f5b377962b6df007730106be\": rpc error: code = NotFound desc = could not find container \"63bdd7c736f92dca64eb88441088f5d389f31638f5b377962b6df007730106be\": container with ID starting with 63bdd7c736f92dca64eb88441088f5d389f31638f5b377962b6df007730106be not found: ID does not exist"
Oct 30 22:34:44.537487 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:44.537445    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-f68tv"]
Oct 30 22:34:44.583874 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:44.583839    5510 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-f68tv"]
Oct 30 22:34:45.529078 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:45.529042    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:34:45.552985 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:45.552950    5510 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4407af9b-8cf7-4552-9dfc-8324d9832655" path="/var/lib/kubelet/pods/4407af9b-8cf7-4552-9dfc-8324d9832655/volumes"
Oct 30 22:34:46.719172 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:46.719136    5510 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-506964/test-model-506964-kserve-bc8f9f68c-bfkq5"]
Oct 30 22:34:46.719670 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:46.719642    5510 cpu_manager.go:414] "containerMap: removing container" podUID="ef500cdd-cc52-45b0-8dd7-1896897acc46" containerName="main"
Oct 30 22:34:46.719730 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:46.719670    5510 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ef500cdd-cc52-45b0-8dd7-1896897acc46" containerName="main"
Oct 30 22:34:46.719730 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:46.719693    5510 cpu_manager.go:414] "containerMap: removing container" podUID="7e6ab5c7-cadf-4bb4-9dda-68b17a855729" containerName="istio-proxy"
Oct 30 22:34:46.719730 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:46.719699    5510 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7e6ab5c7-cadf-4bb4-9dda-68b17a855729" containerName="istio-proxy"
Oct 30 22:34:46.719730 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:46.719713    5510 cpu_manager.go:414] "containerMap: removing container" podUID="4407af9b-8cf7-4552-9dfc-8324d9832655" containerName="discovery"
Oct 30 22:34:46.719730 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:46.719718    5510 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4407af9b-8cf7-4552-9dfc-8324d9832655" containerName="discovery"
Oct 30 22:34:46.719730 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:46.719726    5510 cpu_manager.go:414] "containerMap: removing container" podUID="e17cedd4-7d48-42b3-b127-b6192c1b3795" containerName="main"
Oct 30 22:34:46.719730 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:46.719731    5510 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e17cedd4-7d48-42b3-b127-b6192c1b3795" containerName="main"
Oct 30 22:34:46.719947 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:46.719821    5510 memory_manager.go:365] "RemoveStaleState removing state" podUID="ef500cdd-cc52-45b0-8dd7-1896897acc46" containerName="main"
Oct 30 22:34:46.719947 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:46.719831    5510 memory_manager.go:365] "RemoveStaleState removing state" podUID="7e6ab5c7-cadf-4bb4-9dda-68b17a855729" containerName="istio-proxy"
Oct 30 22:34:46.719947 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:46.719838    5510 memory_manager.go:365] "RemoveStaleState removing state" podUID="4407af9b-8cf7-4552-9dfc-8324d9832655" containerName="discovery"
Oct 30 22:34:46.719947 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:46.719844    5510 memory_manager.go:365] "RemoveStaleState removing state" podUID="e17cedd4-7d48-42b3-b127-b6192c1b3795" containerName="main"
Oct 30 22:34:46.719947 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:46.719852    5510 memory_manager.go:365] "RemoveStaleState removing state" podUID="21557bdf-26e4-4a0c-ad9f-50dcc391616b" containerName="istio-proxy"
Oct 30 22:34:46.723161 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:46.723142    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-506964/test-model-506964-kserve-bc8f9f68c-bfkq5"
Oct 30 22:34:46.726800 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:46.726775    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pqlwd\" (UniqueName: \"kubernetes.io/projected/b3eaa015-ceb2-41a4-b52f-67dd79e72b17-kube-api-access-pqlwd\") pod \"test-model-506964-kserve-bc8f9f68c-bfkq5\" (UID: \"b3eaa015-ceb2-41a4-b52f-67dd79e72b17\") " pod="ai-tenant-e2e-inf-b-506964/test-model-506964-kserve-bc8f9f68c-bfkq5"
Oct 30 22:34:46.726896 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:46.726824    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b3eaa015-ceb2-41a4-b52f-67dd79e72b17-model-cache\") pod \"test-model-506964-kserve-bc8f9f68c-bfkq5\" (UID: \"b3eaa015-ceb2-41a4-b52f-67dd79e72b17\") " pod="ai-tenant-e2e-inf-b-506964/test-model-506964-kserve-bc8f9f68c-bfkq5"
Oct 30 22:34:46.726955 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:46.726929    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b3eaa015-ceb2-41a4-b52f-67dd79e72b17-tls-certs\") pod \"test-model-506964-kserve-bc8f9f68c-bfkq5\" (UID: \"b3eaa015-ceb2-41a4-b52f-67dd79e72b17\") " pod="ai-tenant-e2e-inf-b-506964/test-model-506964-kserve-bc8f9f68c-bfkq5"
Oct 30 22:34:46.726999 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:46.726973    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b3eaa015-ceb2-41a4-b52f-67dd79e72b17-dshm\") pod \"test-model-506964-kserve-bc8f9f68c-bfkq5\" (UID: \"b3eaa015-ceb2-41a4-b52f-67dd79e72b17\") " pod="ai-tenant-e2e-inf-b-506964/test-model-506964-kserve-bc8f9f68c-bfkq5"
Oct 30 22:34:46.727158 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:46.727144    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b3eaa015-ceb2-41a4-b52f-67dd79e72b17-tmp-dir\") pod \"test-model-506964-kserve-bc8f9f68c-bfkq5\" (UID: \"b3eaa015-ceb2-41a4-b52f-67dd79e72b17\") " pod="ai-tenant-e2e-inf-b-506964/test-model-506964-kserve-bc8f9f68c-bfkq5"
Oct 30 22:34:46.727202 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:46.727187    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b3eaa015-ceb2-41a4-b52f-67dd79e72b17-home\") pod \"test-model-506964-kserve-bc8f9f68c-bfkq5\" (UID: \"b3eaa015-ceb2-41a4-b52f-67dd79e72b17\") " pod="ai-tenant-e2e-inf-b-506964/test-model-506964-kserve-bc8f9f68c-bfkq5"
Oct 30 22:34:46.810041 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:46.810002    5510 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-506964\"/\"openshift-service-ca.crt\""
Oct 30 22:34:46.810041 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:46.810013    5510 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-506964\"/\"kube-root-ca.crt\""
Oct 30 22:34:46.815191 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:46.815161    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-506964\"/\"test-model-506964-kserve-self-signed-certs\""
Oct 30 22:34:46.815350 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:46.815237    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-506964\"/\"default-dockercfg-x6wgv\""
Oct 30 22:34:46.817860 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:46.817832    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-506964/test-model-506964-kserve-bc8f9f68c-bfkq5"]
Oct 30 22:34:46.828077 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:46.828053    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b3eaa015-ceb2-41a4-b52f-67dd79e72b17-tmp-dir\") pod \"test-model-506964-kserve-bc8f9f68c-bfkq5\" (UID: \"b3eaa015-ceb2-41a4-b52f-67dd79e72b17\") " pod="ai-tenant-e2e-inf-b-506964/test-model-506964-kserve-bc8f9f68c-bfkq5"
Oct 30 22:34:46.828200 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:46.828096    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b3eaa015-ceb2-41a4-b52f-67dd79e72b17-home\") pod \"test-model-506964-kserve-bc8f9f68c-bfkq5\" (UID: \"b3eaa015-ceb2-41a4-b52f-67dd79e72b17\") " pod="ai-tenant-e2e-inf-b-506964/test-model-506964-kserve-bc8f9f68c-bfkq5"
Oct 30 22:34:46.828200 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:46.828157    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pqlwd\" (UniqueName: \"kubernetes.io/projected/b3eaa015-ceb2-41a4-b52f-67dd79e72b17-kube-api-access-pqlwd\") pod \"test-model-506964-kserve-bc8f9f68c-bfkq5\" (UID: \"b3eaa015-ceb2-41a4-b52f-67dd79e72b17\") " pod="ai-tenant-e2e-inf-b-506964/test-model-506964-kserve-bc8f9f68c-bfkq5"
Oct 30 22:34:46.828200 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:46.828191    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b3eaa015-ceb2-41a4-b52f-67dd79e72b17-model-cache\") pod \"test-model-506964-kserve-bc8f9f68c-bfkq5\" (UID: \"b3eaa015-ceb2-41a4-b52f-67dd79e72b17\") " pod="ai-tenant-e2e-inf-b-506964/test-model-506964-kserve-bc8f9f68c-bfkq5"
Oct 30 22:34:46.828535 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:46.828510    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b3eaa015-ceb2-41a4-b52f-67dd79e72b17-tls-certs\") pod \"test-model-506964-kserve-bc8f9f68c-bfkq5\" (UID: \"b3eaa015-ceb2-41a4-b52f-67dd79e72b17\") " pod="ai-tenant-e2e-inf-b-506964/test-model-506964-kserve-bc8f9f68c-bfkq5"
Oct 30 22:34:46.828604 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:46.828540    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b3eaa015-ceb2-41a4-b52f-67dd79e72b17-model-cache\") pod \"test-model-506964-kserve-bc8f9f68c-bfkq5\" (UID: \"b3eaa015-ceb2-41a4-b52f-67dd79e72b17\") " pod="ai-tenant-e2e-inf-b-506964/test-model-506964-kserve-bc8f9f68c-bfkq5"
Oct 30 22:34:46.828604 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:46.828554    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b3eaa015-ceb2-41a4-b52f-67dd79e72b17-dshm\") pod \"test-model-506964-kserve-bc8f9f68c-bfkq5\" (UID: \"b3eaa015-ceb2-41a4-b52f-67dd79e72b17\") " pod="ai-tenant-e2e-inf-b-506964/test-model-506964-kserve-bc8f9f68c-bfkq5"
Oct 30 22:34:46.828604 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:46.828575    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b3eaa015-ceb2-41a4-b52f-67dd79e72b17-tmp-dir\") pod \"test-model-506964-kserve-bc8f9f68c-bfkq5\" (UID: \"b3eaa015-ceb2-41a4-b52f-67dd79e72b17\") " pod="ai-tenant-e2e-inf-b-506964/test-model-506964-kserve-bc8f9f68c-bfkq5"
Oct 30 22:34:46.828604 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:46.828596    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b3eaa015-ceb2-41a4-b52f-67dd79e72b17-home\") pod \"test-model-506964-kserve-bc8f9f68c-bfkq5\" (UID: \"b3eaa015-ceb2-41a4-b52f-67dd79e72b17\") " pod="ai-tenant-e2e-inf-b-506964/test-model-506964-kserve-bc8f9f68c-bfkq5"
Oct 30 22:34:46.830869 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:46.830843    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b3eaa015-ceb2-41a4-b52f-67dd79e72b17-dshm\") pod \"test-model-506964-kserve-bc8f9f68c-bfkq5\" (UID: \"b3eaa015-ceb2-41a4-b52f-67dd79e72b17\") " pod="ai-tenant-e2e-inf-b-506964/test-model-506964-kserve-bc8f9f68c-bfkq5"
Oct 30 22:34:46.831157 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:46.831135    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b3eaa015-ceb2-41a4-b52f-67dd79e72b17-tls-certs\") pod \"test-model-506964-kserve-bc8f9f68c-bfkq5\" (UID: \"b3eaa015-ceb2-41a4-b52f-67dd79e72b17\") " pod="ai-tenant-e2e-inf-b-506964/test-model-506964-kserve-bc8f9f68c-bfkq5"
Oct 30 22:34:46.861683 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:46.861627    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:34:46.886123 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:46.886088    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pqlwd\" (UniqueName: \"kubernetes.io/projected/b3eaa015-ceb2-41a4-b52f-67dd79e72b17-kube-api-access-pqlwd\") pod \"test-model-506964-kserve-bc8f9f68c-bfkq5\" (UID: \"b3eaa015-ceb2-41a4-b52f-67dd79e72b17\") " pod="ai-tenant-e2e-inf-b-506964/test-model-506964-kserve-bc8f9f68c-bfkq5"
Oct 30 22:34:47.036365 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:47.036330    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-506964/test-model-506964-kserve-bc8f9f68c-bfkq5"
Oct 30 22:34:47.313150 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:47.313122    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-779f32-67bbb7bf95-sjnl9"
Oct 30 22:34:47.333868 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:47.333841    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4fd056ae-943a-43c1-8682-8edb3fe3b5ed-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4fd056ae-943a-43c1-8682-8edb3fe3b5ed-plugins-config-volume\") pod \"4fd056ae-943a-43c1-8682-8edb3fe3b5ed\" (UID: \"4fd056ae-943a-43c1-8682-8edb3fe3b5ed\") "
Oct 30 22:34:47.334024 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:47.333916    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4fd056ae-943a-43c1-8682-8edb3fe3b5ed-kube-api-access-4wmpg\" (UniqueName: \"kubernetes.io/projected/4fd056ae-943a-43c1-8682-8edb3fe3b5ed-kube-api-access-4wmpg\") pod \"4fd056ae-943a-43c1-8682-8edb3fe3b5ed\" (UID: \"4fd056ae-943a-43c1-8682-8edb3fe3b5ed\") "
Oct 30 22:34:47.334269 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:47.334236    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4fd056ae-943a-43c1-8682-8edb3fe3b5ed-plugins-config-volume" pod "4fd056ae-943a-43c1-8682-8edb3fe3b5ed" (UID: "4fd056ae-943a-43c1-8682-8edb3fe3b5ed"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:34:47.334440 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:47.334418    5510 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4fd056ae-943a-43c1-8682-8edb3fe3b5ed-plugins-config-volume\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:34:47.336259 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:47.336232    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4fd056ae-943a-43c1-8682-8edb3fe3b5ed-kube-api-access-4wmpg" pod "4fd056ae-943a-43c1-8682-8edb3fe3b5ed" (UID: "4fd056ae-943a-43c1-8682-8edb3fe3b5ed"). InnerVolumeSpecName "kube-api-access-4wmpg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:34:47.399533 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:47.399498    5510 generic.go:356] "Generic (PLEG): container finished" podID="4fd056ae-943a-43c1-8682-8edb3fe3b5ed" containerID="a46ab953c0354b4a66225f2364b7f14057e5153277721e749c2479e31b77ee2f" exitCode=0
Oct 30 22:34:47.399750 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:47.399558    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-779f32-67bbb7bf95-sjnl9"
Oct 30 22:34:47.399750 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:47.399582    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-779f32-67bbb7bf95-sjnl9" event={"ID":"4fd056ae-943a-43c1-8682-8edb3fe3b5ed","Type":"ContainerDied","Data":"a46ab953c0354b4a66225f2364b7f14057e5153277721e749c2479e31b77ee2f"}
Oct 30 22:34:47.399750 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:47.399636    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-779f32-67bbb7bf95-sjnl9" event={"ID":"4fd056ae-943a-43c1-8682-8edb3fe3b5ed","Type":"ContainerDied","Data":"8b2914f5436eca3bb7b62c78423554d4d4d8457ed9de72b5b2c45f774202c2b0"}
Oct 30 22:34:47.399750 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:47.399669    5510 scope.go:122] "RemoveContainer" containerID="a46ab953c0354b4a66225f2364b7f14057e5153277721e749c2479e31b77ee2f"
Oct 30 22:34:47.410459 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:47.410438    5510 scope.go:122] "RemoveContainer" containerID="a46ab953c0354b4a66225f2364b7f14057e5153277721e749c2479e31b77ee2f"
Oct 30 22:34:47.410827 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:34:47.410801    5510 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a46ab953c0354b4a66225f2364b7f14057e5153277721e749c2479e31b77ee2f\": container with ID starting with a46ab953c0354b4a66225f2364b7f14057e5153277721e749c2479e31b77ee2f not found: ID does not exist" containerID="a46ab953c0354b4a66225f2364b7f14057e5153277721e749c2479e31b77ee2f"
Oct 30 22:34:47.410949 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:47.410834    5510 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a46ab953c0354b4a66225f2364b7f14057e5153277721e749c2479e31b77ee2f"} err="failed to get container status \"a46ab953c0354b4a66225f2364b7f14057e5153277721e749c2479e31b77ee2f\": rpc error: code = NotFound desc = could not find container \"a46ab953c0354b4a66225f2364b7f14057e5153277721e749c2479e31b77ee2f\": container with ID starting with a46ab953c0354b4a66225f2364b7f14057e5153277721e749c2479e31b77ee2f not found: ID does not exist"
Oct 30 22:34:47.411908 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:47.411889    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-506964/test-model-506964-kserve-bc8f9f68c-bfkq5"]
Oct 30 22:34:47.414160 ip-10-0-136-210 kubenswrapper[5510]: W1030 22:34:47.414139    5510 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb3eaa015_ceb2_41a4_b52f_67dd79e72b17.slice/crio-056b342a85f56206e8ad988343886b25be812af1062a93da988cd32f31ccd53c WatchSource:0}: Error finding container 056b342a85f56206e8ad988343886b25be812af1062a93da988cd32f31ccd53c: Status 404 returned error can't find the container with id 056b342a85f56206e8ad988343886b25be812af1062a93da988cd32f31ccd53c
Oct 30 22:34:47.436285 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:47.436249    5510 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4wmpg\" (UniqueName: \"kubernetes.io/projected/4fd056ae-943a-43c1-8682-8edb3fe3b5ed-kube-api-access-4wmpg\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:34:47.530427 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:47.530396    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-779f32-67bbb7bf95-sjnl9"]
Oct 30 22:34:47.534636 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:47.534553    5510 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-779f32-67bbb7bf95-sjnl9"]
Oct 30 22:34:47.560015 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:47.552765    5510 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4fd056ae-943a-43c1-8682-8edb3fe3b5ed" path="/var/lib/kubelet/pods/4fd056ae-943a-43c1-8682-8edb3fe3b5ed/volumes"
Oct 30 22:34:48.407461 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:48.407422    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-506964/test-model-506964-kserve-bc8f9f68c-bfkq5" event={"ID":"b3eaa015-ceb2-41a4-b52f-67dd79e72b17","Type":"ContainerStarted","Data":"2667979b3f05d9ae75d6558234e3020cbfacdf3ebc0af93dec147a881b9c013b"}
Oct 30 22:34:48.407896 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:48.407469    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-506964/test-model-506964-kserve-bc8f9f68c-bfkq5" event={"ID":"b3eaa015-ceb2-41a4-b52f-67dd79e72b17","Type":"ContainerStarted","Data":"056b342a85f56206e8ad988343886b25be812af1062a93da988cd32f31ccd53c"}
Oct 30 22:34:48.533308 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:48.533250    5510 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-506964/test-model-506964-kserve-bc8f9f68c-bfkq5" podStartSLOduration=2.533236522 podStartE2EDuration="2.533236522s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:34:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:34:48.531950123 +0000 UTC m=+2669.092760128" watchObservedRunningTime="2026-10-30 22:34:48.533236522 +0000 UTC m=+2669.094046527"
Oct 30 22:34:48.824284 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:48.824259    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-lggs2"
Oct 30 22:34:48.851477 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:48.851440    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a3a8e626-b7fa-4fe1-8d74-0d8bef87fb63-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a3a8e626-b7fa-4fe1-8d74-0d8bef87fb63-istio-data\") pod \"a3a8e626-b7fa-4fe1-8d74-0d8bef87fb63\" (UID: \"a3a8e626-b7fa-4fe1-8d74-0d8bef87fb63\") "
Oct 30 22:34:48.851641 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:48.851497    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a3a8e626-b7fa-4fe1-8d74-0d8bef87fb63-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a3a8e626-b7fa-4fe1-8d74-0d8bef87fb63-istiod-ca-cert\") pod \"a3a8e626-b7fa-4fe1-8d74-0d8bef87fb63\" (UID: \"a3a8e626-b7fa-4fe1-8d74-0d8bef87fb63\") "
Oct 30 22:34:48.851641 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:48.851533    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a3a8e626-b7fa-4fe1-8d74-0d8bef87fb63-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a3a8e626-b7fa-4fe1-8d74-0d8bef87fb63-istio-envoy\") pod \"a3a8e626-b7fa-4fe1-8d74-0d8bef87fb63\" (UID: \"a3a8e626-b7fa-4fe1-8d74-0d8bef87fb63\") "
Oct 30 22:34:48.851641 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:48.851567    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a3a8e626-b7fa-4fe1-8d74-0d8bef87fb63-kube-api-access-lsgdn\" (UniqueName: \"kubernetes.io/projected/a3a8e626-b7fa-4fe1-8d74-0d8bef87fb63-kube-api-access-lsgdn\") pod \"a3a8e626-b7fa-4fe1-8d74-0d8bef87fb63\" (UID: \"a3a8e626-b7fa-4fe1-8d74-0d8bef87fb63\") "
Oct 30 22:34:48.851641 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:48.851625    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/a3a8e626-b7fa-4fe1-8d74-0d8bef87fb63-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a3a8e626-b7fa-4fe1-8d74-0d8bef87fb63-istio-podinfo\") pod \"a3a8e626-b7fa-4fe1-8d74-0d8bef87fb63\" (UID: \"a3a8e626-b7fa-4fe1-8d74-0d8bef87fb63\") "
Oct 30 22:34:48.851907 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:48.851721    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a3a8e626-b7fa-4fe1-8d74-0d8bef87fb63-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3a8e626-b7fa-4fe1-8d74-0d8bef87fb63-credential-socket\") pod \"a3a8e626-b7fa-4fe1-8d74-0d8bef87fb63\" (UID: \"a3a8e626-b7fa-4fe1-8d74-0d8bef87fb63\") "
Oct 30 22:34:48.851907 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:48.851786    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a3a8e626-b7fa-4fe1-8d74-0d8bef87fb63-istio-token\" (UniqueName: \"kubernetes.io/projected/a3a8e626-b7fa-4fe1-8d74-0d8bef87fb63-istio-token\") pod \"a3a8e626-b7fa-4fe1-8d74-0d8bef87fb63\" (UID: \"a3a8e626-b7fa-4fe1-8d74-0d8bef87fb63\") "
Oct 30 22:34:48.851907 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:48.851811    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3a8e626-b7fa-4fe1-8d74-0d8bef87fb63-istio-data" pod "a3a8e626-b7fa-4fe1-8d74-0d8bef87fb63" (UID: "a3a8e626-b7fa-4fe1-8d74-0d8bef87fb63"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:34:48.851907 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:48.851842    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a3a8e626-b7fa-4fe1-8d74-0d8bef87fb63-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a3a8e626-b7fa-4fe1-8d74-0d8bef87fb63-workload-certs\") pod \"a3a8e626-b7fa-4fe1-8d74-0d8bef87fb63\" (UID: \"a3a8e626-b7fa-4fe1-8d74-0d8bef87fb63\") "
Oct 30 22:34:48.852110 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:48.851914    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a3a8e626-b7fa-4fe1-8d74-0d8bef87fb63-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3a8e626-b7fa-4fe1-8d74-0d8bef87fb63-workload-socket\") pod \"a3a8e626-b7fa-4fe1-8d74-0d8bef87fb63\" (UID: \"a3a8e626-b7fa-4fe1-8d74-0d8bef87fb63\") "
Oct 30 22:34:48.852110 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:48.852086    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3a8e626-b7fa-4fe1-8d74-0d8bef87fb63-workload-certs" pod "a3a8e626-b7fa-4fe1-8d74-0d8bef87fb63" (UID: "a3a8e626-b7fa-4fe1-8d74-0d8bef87fb63"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:34:48.852110 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:48.852096    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a3a8e626-b7fa-4fe1-8d74-0d8bef87fb63-istiod-ca-cert" pod "a3a8e626-b7fa-4fe1-8d74-0d8bef87fb63" (UID: "a3a8e626-b7fa-4fe1-8d74-0d8bef87fb63"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:34:48.852267 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:48.852185    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3a8e626-b7fa-4fe1-8d74-0d8bef87fb63-credential-socket" pod "a3a8e626-b7fa-4fe1-8d74-0d8bef87fb63" (UID: "a3a8e626-b7fa-4fe1-8d74-0d8bef87fb63"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:34:48.852324 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:48.852274    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3a8e626-b7fa-4fe1-8d74-0d8bef87fb63-workload-socket" pod "a3a8e626-b7fa-4fe1-8d74-0d8bef87fb63" (UID: "a3a8e626-b7fa-4fe1-8d74-0d8bef87fb63"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:34:48.852537 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:48.852501    5510 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3a8e626-b7fa-4fe1-8d74-0d8bef87fb63-credential-socket\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:34:48.852537 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:48.852522    5510 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a3a8e626-b7fa-4fe1-8d74-0d8bef87fb63-workload-certs\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:34:48.852728 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:48.852538    5510 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3a8e626-b7fa-4fe1-8d74-0d8bef87fb63-workload-socket\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:34:48.852728 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:48.852553    5510 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a3a8e626-b7fa-4fe1-8d74-0d8bef87fb63-istio-data\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:34:48.852728 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:48.852568    5510 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a3a8e626-b7fa-4fe1-8d74-0d8bef87fb63-istiod-ca-cert\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:34:48.853878 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:48.853851    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a3a8e626-b7fa-4fe1-8d74-0d8bef87fb63-kube-api-access-lsgdn" pod "a3a8e626-b7fa-4fe1-8d74-0d8bef87fb63" (UID: "a3a8e626-b7fa-4fe1-8d74-0d8bef87fb63"). InnerVolumeSpecName "kube-api-access-lsgdn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:34:48.853976 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:48.853921    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a3a8e626-b7fa-4fe1-8d74-0d8bef87fb63-istio-podinfo" pod "a3a8e626-b7fa-4fe1-8d74-0d8bef87fb63" (UID: "a3a8e626-b7fa-4fe1-8d74-0d8bef87fb63"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:34:48.854022 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:48.853994    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3a8e626-b7fa-4fe1-8d74-0d8bef87fb63-istio-envoy" pod "a3a8e626-b7fa-4fe1-8d74-0d8bef87fb63" (UID: "a3a8e626-b7fa-4fe1-8d74-0d8bef87fb63"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:34:48.854022 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:48.854013    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a3a8e626-b7fa-4fe1-8d74-0d8bef87fb63-istio-token" pod "a3a8e626-b7fa-4fe1-8d74-0d8bef87fb63" (UID: "a3a8e626-b7fa-4fe1-8d74-0d8bef87fb63"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:34:48.954286 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:48.954207    5510 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a3a8e626-b7fa-4fe1-8d74-0d8bef87fb63-istio-token\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:34:48.954286 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:48.954237    5510 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a3a8e626-b7fa-4fe1-8d74-0d8bef87fb63-istio-envoy\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:34:48.954286 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:48.954248    5510 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lsgdn\" (UniqueName: \"kubernetes.io/projected/a3a8e626-b7fa-4fe1-8d74-0d8bef87fb63-kube-api-access-lsgdn\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:34:48.954286 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:48.954259    5510 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a3a8e626-b7fa-4fe1-8d74-0d8bef87fb63-istio-podinfo\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:34:49.037261 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:49.037224    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-506964/test-model-506964-kserve-bc8f9f68c-bfkq5"
Oct 30 22:34:49.414869 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:49.414832    5510 generic.go:356] "Generic (PLEG): container finished" podID="a3a8e626-b7fa-4fe1-8d74-0d8bef87fb63" containerID="5540c53ae933ff5b3eb69e9f0fb8a7ac5ed9a36279ae4079ea91a6f84f93b83c" exitCode=0
Oct 30 22:34:49.415319 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:49.414881    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-lggs2"
Oct 30 22:34:49.415319 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:49.414879    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-lggs2" event={"ID":"a3a8e626-b7fa-4fe1-8d74-0d8bef87fb63","Type":"ContainerDied","Data":"5540c53ae933ff5b3eb69e9f0fb8a7ac5ed9a36279ae4079ea91a6f84f93b83c"}
Oct 30 22:34:49.415319 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:49.414996    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-lggs2" event={"ID":"a3a8e626-b7fa-4fe1-8d74-0d8bef87fb63","Type":"ContainerDied","Data":"e1f005c2683ab35f2e1955016772aa195b783b6212395c735d1cda5f832bdb4b"}
Oct 30 22:34:49.415319 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:49.415018    5510 scope.go:122] "RemoveContainer" containerID="5540c53ae933ff5b3eb69e9f0fb8a7ac5ed9a36279ae4079ea91a6f84f93b83c"
Oct 30 22:34:49.426052 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:49.426035    5510 scope.go:122] "RemoveContainer" containerID="5540c53ae933ff5b3eb69e9f0fb8a7ac5ed9a36279ae4079ea91a6f84f93b83c"
Oct 30 22:34:49.426296 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:34:49.426268    5510 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5540c53ae933ff5b3eb69e9f0fb8a7ac5ed9a36279ae4079ea91a6f84f93b83c\": container with ID starting with 5540c53ae933ff5b3eb69e9f0fb8a7ac5ed9a36279ae4079ea91a6f84f93b83c not found: ID does not exist" containerID="5540c53ae933ff5b3eb69e9f0fb8a7ac5ed9a36279ae4079ea91a6f84f93b83c"
Oct 30 22:34:49.426406 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:49.426302    5510 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5540c53ae933ff5b3eb69e9f0fb8a7ac5ed9a36279ae4079ea91a6f84f93b83c"} err="failed to get container status \"5540c53ae933ff5b3eb69e9f0fb8a7ac5ed9a36279ae4079ea91a6f84f93b83c\": rpc error: code = NotFound desc = could not find container \"5540c53ae933ff5b3eb69e9f0fb8a7ac5ed9a36279ae4079ea91a6f84f93b83c\": container with ID starting with 5540c53ae933ff5b3eb69e9f0fb8a7ac5ed9a36279ae4079ea91a6f84f93b83c not found: ID does not exist"
Oct 30 22:34:49.573416 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:49.573387    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-lggs2"]
Oct 30 22:34:49.620687 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:49.620642    5510 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-a00d9b-openshift-default-847df8fc59-lggs2"]
Oct 30 22:34:50.240530 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:50.240489    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:34:51.553093 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:51.553058    5510 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a3a8e626-b7fa-4fe1-8d74-0d8bef87fb63" path="/var/lib/kubelet/pods/a3a8e626-b7fa-4fe1-8d74-0d8bef87fb63/volumes"
Oct 30 22:34:52.458901 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:52.458862    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:34:52.688617 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:52.688573    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:34:54.433366 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:54.433326    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:34:54.893516 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:54.893481    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:34:57.036724 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:57.036695    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-506964/test-model-506964-kserve-bc8f9f68c-bfkq5"
Oct 30 22:34:57.050808 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:57.050775    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-506964/test-model-506964-kserve-bc8f9f68c-bfkq5"
Oct 30 22:34:57.473214 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:57.473120    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-506964/test-model-506964-kserve-bc8f9f68c-bfkq5"
Oct 30 22:34:59.705491 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:34:59.705449    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:35:00.716369 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:35:00.716328    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:35:04.277207 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:35:04.277156    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:35:07.008483 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:35:07.008441    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:35:07.215998 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:35:07.215959    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:35:08.419431 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:35:08.419397    5510 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6gjbf" containerName="extractor"
Oct 30 22:35:08.918274 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:35:08.918239    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:35:13.711787 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:35:13.711746    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:35:18.297922 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:35:18.297882    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:35:22.898980 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:35:22.898927    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:35:27.488374 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:35:27.488327    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:35:32.114639 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:35:32.114553    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:35:33.837642 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:35:33.837608    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:35:36.678632 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:35:36.678573    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:35:38.419387 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:35:38.419355    5510 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6gjbf" containerName="extractor"
Oct 30 22:35:41.242919 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:35:41.242882    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:35:45.081615 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:35:45.081575    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-779f32-76b4d9d8c7-fxknc"]
Oct 30 22:35:45.082063 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:35:45.081818    5510 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-779f32-76b4d9d8c7-fxknc" podUID="4a00e5f4-7118-41ce-b4c7-eb0f74da8237" containerName="maas-api" containerID="cri-o://aac40e09210b083d154b781f485361fc2c7c33a4a4a0d1179f43cee494f0ed4c" gracePeriod=30
Oct 30 22:35:45.278288 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:35:45.278248    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:35:45.351114 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:35:45.351086    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-779f32-76b4d9d8c7-fxknc"
Oct 30 22:35:45.505028 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:35:45.504992    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4a00e5f4-7118-41ce-b4c7-eb0f74da8237-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4a00e5f4-7118-41ce-b4c7-eb0f74da8237-maas-api-tls\") pod \"4a00e5f4-7118-41ce-b4c7-eb0f74da8237\" (UID: \"4a00e5f4-7118-41ce-b4c7-eb0f74da8237\") "
Oct 30 22:35:45.505229 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:35:45.505145    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4a00e5f4-7118-41ce-b4c7-eb0f74da8237-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4a00e5f4-7118-41ce-b4c7-eb0f74da8237-openshift-service-ca\") pod \"4a00e5f4-7118-41ce-b4c7-eb0f74da8237\" (UID: \"4a00e5f4-7118-41ce-b4c7-eb0f74da8237\") "
Oct 30 22:35:45.505229 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:35:45.505212    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4a00e5f4-7118-41ce-b4c7-eb0f74da8237-kube-api-access-42qrm\" (UniqueName: \"kubernetes.io/projected/4a00e5f4-7118-41ce-b4c7-eb0f74da8237-kube-api-access-42qrm\") pod \"4a00e5f4-7118-41ce-b4c7-eb0f74da8237\" (UID: \"4a00e5f4-7118-41ce-b4c7-eb0f74da8237\") "
Oct 30 22:35:45.505366 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:35:45.505340    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4a00e5f4-7118-41ce-b4c7-eb0f74da8237-metrics-certs\" (UniqueName: \"kubernetes.io/secret/4a00e5f4-7118-41ce-b4c7-eb0f74da8237-metrics-certs\") pod \"4a00e5f4-7118-41ce-b4c7-eb0f74da8237\" (UID: \"4a00e5f4-7118-41ce-b4c7-eb0f74da8237\") "
Oct 30 22:35:45.505581 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:35:45.505555    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4a00e5f4-7118-41ce-b4c7-eb0f74da8237-openshift-service-ca" pod "4a00e5f4-7118-41ce-b4c7-eb0f74da8237" (UID: "4a00e5f4-7118-41ce-b4c7-eb0f74da8237"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:35:45.505959 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:35:45.505932    5510 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4a00e5f4-7118-41ce-b4c7-eb0f74da8237-openshift-service-ca\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:35:45.507584 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:35:45.507554    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4a00e5f4-7118-41ce-b4c7-eb0f74da8237-maas-api-tls" pod "4a00e5f4-7118-41ce-b4c7-eb0f74da8237" (UID: "4a00e5f4-7118-41ce-b4c7-eb0f74da8237"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:35:45.508039 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:35:45.508008    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4a00e5f4-7118-41ce-b4c7-eb0f74da8237-metrics-certs" pod "4a00e5f4-7118-41ce-b4c7-eb0f74da8237" (UID: "4a00e5f4-7118-41ce-b4c7-eb0f74da8237"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:35:45.508125 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:35:45.508048    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4a00e5f4-7118-41ce-b4c7-eb0f74da8237-kube-api-access-42qrm" pod "4a00e5f4-7118-41ce-b4c7-eb0f74da8237" (UID: "4a00e5f4-7118-41ce-b4c7-eb0f74da8237"). InnerVolumeSpecName "kube-api-access-42qrm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:35:45.607615 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:35:45.607477    5510 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-42qrm\" (UniqueName: \"kubernetes.io/projected/4a00e5f4-7118-41ce-b4c7-eb0f74da8237-kube-api-access-42qrm\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:35:45.607615 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:35:45.607517    5510 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4a00e5f4-7118-41ce-b4c7-eb0f74da8237-metrics-certs\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:35:45.607615 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:35:45.607532    5510 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4a00e5f4-7118-41ce-b4c7-eb0f74da8237-maas-api-tls\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:35:45.728818 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:35:45.728780    5510 generic.go:356] "Generic (PLEG): container finished" podID="4a00e5f4-7118-41ce-b4c7-eb0f74da8237" containerID="aac40e09210b083d154b781f485361fc2c7c33a4a4a0d1179f43cee494f0ed4c" exitCode=0
Oct 30 22:35:45.728993 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:35:45.728820    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-779f32-76b4d9d8c7-fxknc"
Oct 30 22:35:45.728993 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:35:45.728846    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-779f32-76b4d9d8c7-fxknc" event={"ID":"4a00e5f4-7118-41ce-b4c7-eb0f74da8237","Type":"ContainerDied","Data":"aac40e09210b083d154b781f485361fc2c7c33a4a4a0d1179f43cee494f0ed4c"}
Oct 30 22:35:45.728993 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:35:45.728906    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-779f32-76b4d9d8c7-fxknc" event={"ID":"4a00e5f4-7118-41ce-b4c7-eb0f74da8237","Type":"ContainerDied","Data":"1f023c1e4a91ad636f7f96833905aec1d86ce21e3e7384695862888de03ecd97"}
Oct 30 22:35:45.728993 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:35:45.728935    5510 scope.go:122] "RemoveContainer" containerID="aac40e09210b083d154b781f485361fc2c7c33a4a4a0d1179f43cee494f0ed4c"
Oct 30 22:35:45.738496 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:35:45.738476    5510 scope.go:122] "RemoveContainer" containerID="aac40e09210b083d154b781f485361fc2c7c33a4a4a0d1179f43cee494f0ed4c"
Oct 30 22:35:45.738797 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:35:45.738766    5510 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aac40e09210b083d154b781f485361fc2c7c33a4a4a0d1179f43cee494f0ed4c\": container with ID starting with aac40e09210b083d154b781f485361fc2c7c33a4a4a0d1179f43cee494f0ed4c not found: ID does not exist" containerID="aac40e09210b083d154b781f485361fc2c7c33a4a4a0d1179f43cee494f0ed4c"
Oct 30 22:35:45.738907 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:35:45.738803    5510 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aac40e09210b083d154b781f485361fc2c7c33a4a4a0d1179f43cee494f0ed4c"} err="failed to get container status \"aac40e09210b083d154b781f485361fc2c7c33a4a4a0d1179f43cee494f0ed4c\": rpc error: code = NotFound desc = could not find container \"aac40e09210b083d154b781f485361fc2c7c33a4a4a0d1179f43cee494f0ed4c\": container with ID starting with aac40e09210b083d154b781f485361fc2c7c33a4a4a0d1179f43cee494f0ed4c not found: ID does not exist"
Oct 30 22:35:45.889676 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:35:45.889574    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-779f32-76b4d9d8c7-fxknc"]
Oct 30 22:35:45.896272 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:35:45.896247    5510 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-779f32-76b4d9d8c7-fxknc"]
Oct 30 22:35:46.778807 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:35:46.778737    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:35:47.034566 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:35:47.034475    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:35:47.259433 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:35:47.259392    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:35:47.553420 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:35:47.553382    5510 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4a00e5f4-7118-41ce-b4c7-eb0f74da8237" path="/var/lib/kubelet/pods/4a00e5f4-7118-41ce-b4c7-eb0f74da8237/volumes"
Oct 30 22:35:47.849390 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:35:47.849291    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:35:51.925085 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:35:51.925047    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:35:54.550177 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:35:54.550101    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:35:54.759307 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:35:54.759272    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:35:54.859718 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:35:54.859618    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-qcxcm"]
Oct 30 22:35:54.860024 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:35:54.859996    5510 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-qcxcm" podUID="eb115651-ae12-43d8-8022-766c30575782" containerName="istio-proxy" containerID="cri-o://c27a66e8c84934801603a06cb32d4bd8174ddf789f08009f1f9fa9a48ce966ee" gracePeriod=30
Oct 30 22:35:54.982895 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:35:54.982861    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:35:56.496135 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:35:56.496097    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:35:59.780014 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:35:59.779977    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:36:00.120340 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:36:00.120313    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-qcxcm"
Oct 30 22:36:00.179182 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:36:00.179151    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/eb115651-ae12-43d8-8022-766c30575782-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eb115651-ae12-43d8-8022-766c30575782-istio-envoy\") pod \"eb115651-ae12-43d8-8022-766c30575782\" (UID: \"eb115651-ae12-43d8-8022-766c30575782\") "
Oct 30 22:36:00.179364 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:36:00.179187    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/eb115651-ae12-43d8-8022-766c30575782-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eb115651-ae12-43d8-8022-766c30575782-istiod-ca-cert\") pod \"eb115651-ae12-43d8-8022-766c30575782\" (UID: \"eb115651-ae12-43d8-8022-766c30575782\") "
Oct 30 22:36:00.179364 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:36:00.179226    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/eb115651-ae12-43d8-8022-766c30575782-istio-token\" (UniqueName: \"kubernetes.io/projected/eb115651-ae12-43d8-8022-766c30575782-istio-token\") pod \"eb115651-ae12-43d8-8022-766c30575782\" (UID: \"eb115651-ae12-43d8-8022-766c30575782\") "
Oct 30 22:36:00.179364 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:36:00.179335    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/eb115651-ae12-43d8-8022-766c30575782-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb115651-ae12-43d8-8022-766c30575782-credential-socket\") pod \"eb115651-ae12-43d8-8022-766c30575782\" (UID: \"eb115651-ae12-43d8-8022-766c30575782\") "
Oct 30 22:36:00.179548 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:36:00.179407    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/eb115651-ae12-43d8-8022-766c30575782-kube-api-access-zlf5v\" (UniqueName: \"kubernetes.io/projected/eb115651-ae12-43d8-8022-766c30575782-kube-api-access-zlf5v\") pod \"eb115651-ae12-43d8-8022-766c30575782\" (UID: \"eb115651-ae12-43d8-8022-766c30575782\") "
Oct 30 22:36:00.179548 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:36:00.179428    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/eb115651-ae12-43d8-8022-766c30575782-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb115651-ae12-43d8-8022-766c30575782-workload-socket\") pod \"eb115651-ae12-43d8-8022-766c30575782\" (UID: \"eb115651-ae12-43d8-8022-766c30575782\") "
Oct 30 22:36:00.179548 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:36:00.179450    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/eb115651-ae12-43d8-8022-766c30575782-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eb115651-ae12-43d8-8022-766c30575782-istio-data\") pod \"eb115651-ae12-43d8-8022-766c30575782\" (UID: \"eb115651-ae12-43d8-8022-766c30575782\") "
Oct 30 22:36:00.179548 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:36:00.179506    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/eb115651-ae12-43d8-8022-766c30575782-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eb115651-ae12-43d8-8022-766c30575782-workload-certs\") pod \"eb115651-ae12-43d8-8022-766c30575782\" (UID: \"eb115651-ae12-43d8-8022-766c30575782\") "
Oct 30 22:36:00.179792 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:36:00.179564    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/eb115651-ae12-43d8-8022-766c30575782-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eb115651-ae12-43d8-8022-766c30575782-istio-podinfo\") pod \"eb115651-ae12-43d8-8022-766c30575782\" (UID: \"eb115651-ae12-43d8-8022-766c30575782\") "
Oct 30 22:36:00.179792 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:36:00.179564    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb115651-ae12-43d8-8022-766c30575782-credential-socket" pod "eb115651-ae12-43d8-8022-766c30575782" (UID: "eb115651-ae12-43d8-8022-766c30575782"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:36:00.179792 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:36:00.179703    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb115651-ae12-43d8-8022-766c30575782-workload-socket" pod "eb115651-ae12-43d8-8022-766c30575782" (UID: "eb115651-ae12-43d8-8022-766c30575782"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:36:00.179792 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:36:00.179735    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eb115651-ae12-43d8-8022-766c30575782-istiod-ca-cert" pod "eb115651-ae12-43d8-8022-766c30575782" (UID: "eb115651-ae12-43d8-8022-766c30575782"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:36:00.179792 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:36:00.179747    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb115651-ae12-43d8-8022-766c30575782-istio-data" pod "eb115651-ae12-43d8-8022-766c30575782" (UID: "eb115651-ae12-43d8-8022-766c30575782"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:36:00.180009 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:36:00.179936    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb115651-ae12-43d8-8022-766c30575782-workload-certs" pod "eb115651-ae12-43d8-8022-766c30575782" (UID: "eb115651-ae12-43d8-8022-766c30575782"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:36:00.180207 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:36:00.180189    5510 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb115651-ae12-43d8-8022-766c30575782-credential-socket\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:36:00.180253 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:36:00.180213    5510 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb115651-ae12-43d8-8022-766c30575782-workload-socket\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:36:00.180253 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:36:00.180229    5510 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eb115651-ae12-43d8-8022-766c30575782-istio-data\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:36:00.180253 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:36:00.180243    5510 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eb115651-ae12-43d8-8022-766c30575782-workload-certs\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:36:00.180417 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:36:00.180258    5510 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eb115651-ae12-43d8-8022-766c30575782-istiod-ca-cert\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:36:00.181464 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:36:00.181437    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb115651-ae12-43d8-8022-766c30575782-istio-envoy" pod "eb115651-ae12-43d8-8022-766c30575782" (UID: "eb115651-ae12-43d8-8022-766c30575782"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:36:00.181605 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:36:00.181538    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/eb115651-ae12-43d8-8022-766c30575782-istio-podinfo" pod "eb115651-ae12-43d8-8022-766c30575782" (UID: "eb115651-ae12-43d8-8022-766c30575782"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:36:00.181605 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:36:00.181571    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eb115651-ae12-43d8-8022-766c30575782-istio-token" pod "eb115651-ae12-43d8-8022-766c30575782" (UID: "eb115651-ae12-43d8-8022-766c30575782"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:36:00.181880 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:36:00.181863    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eb115651-ae12-43d8-8022-766c30575782-kube-api-access-zlf5v" pod "eb115651-ae12-43d8-8022-766c30575782" (UID: "eb115651-ae12-43d8-8022-766c30575782"). InnerVolumeSpecName "kube-api-access-zlf5v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:36:00.281835 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:36:00.281802    5510 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eb115651-ae12-43d8-8022-766c30575782-istio-podinfo\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:36:00.281835 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:36:00.281828    5510 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eb115651-ae12-43d8-8022-766c30575782-istio-envoy\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:36:00.281835 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:36:00.281838    5510 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eb115651-ae12-43d8-8022-766c30575782-istio-token\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:36:00.282097 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:36:00.281847    5510 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zlf5v\" (UniqueName: \"kubernetes.io/projected/eb115651-ae12-43d8-8022-766c30575782-kube-api-access-zlf5v\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:36:00.817520 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:36:00.817485    5510 generic.go:356] "Generic (PLEG): container finished" podID="eb115651-ae12-43d8-8022-766c30575782" containerID="c27a66e8c84934801603a06cb32d4bd8174ddf789f08009f1f9fa9a48ce966ee" exitCode=0
Oct 30 22:36:00.817971 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:36:00.817536    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-qcxcm"
Oct 30 22:36:00.817971 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:36:00.817546    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-qcxcm" event={"ID":"eb115651-ae12-43d8-8022-766c30575782","Type":"ContainerDied","Data":"c27a66e8c84934801603a06cb32d4bd8174ddf789f08009f1f9fa9a48ce966ee"}
Oct 30 22:36:00.817971 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:36:00.817588    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-qcxcm" event={"ID":"eb115651-ae12-43d8-8022-766c30575782","Type":"ContainerDied","Data":"6e4bf47dc2eba883d5079cafe215df80e7bb2da6cd01062a6d09426f0a64d36f"}
Oct 30 22:36:00.817971 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:36:00.817609    5510 scope.go:122] "RemoveContainer" containerID="c27a66e8c84934801603a06cb32d4bd8174ddf789f08009f1f9fa9a48ce966ee"
Oct 30 22:36:00.827850 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:36:00.827800    5510 scope.go:122] "RemoveContainer" containerID="c27a66e8c84934801603a06cb32d4bd8174ddf789f08009f1f9fa9a48ce966ee"
Oct 30 22:36:00.828166 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:36:00.828139    5510 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c27a66e8c84934801603a06cb32d4bd8174ddf789f08009f1f9fa9a48ce966ee\": container with ID starting with c27a66e8c84934801603a06cb32d4bd8174ddf789f08009f1f9fa9a48ce966ee not found: ID does not exist" containerID="c27a66e8c84934801603a06cb32d4bd8174ddf789f08009f1f9fa9a48ce966ee"
Oct 30 22:36:00.828250 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:36:00.828181    5510 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c27a66e8c84934801603a06cb32d4bd8174ddf789f08009f1f9fa9a48ce966ee"} err="failed to get container status \"c27a66e8c84934801603a06cb32d4bd8174ddf789f08009f1f9fa9a48ce966ee\": rpc error: code = NotFound desc = could not find container \"c27a66e8c84934801603a06cb32d4bd8174ddf789f08009f1f9fa9a48ce966ee\": container with ID starting with c27a66e8c84934801603a06cb32d4bd8174ddf789f08009f1f9fa9a48ce966ee not found: ID does not exist"
Oct 30 22:36:00.973902 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:36:00.973864    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-qcxcm"]
Oct 30 22:36:01.020572 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:36:01.020532    5510 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-779f32-openshift-default-75ddfd96d9-qcxcm"]
Oct 30 22:36:01.095155 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:36:01.095009    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:36:01.554078 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:36:01.554045    5510 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="eb115651-ae12-43d8-8022-766c30575782" path="/var/lib/kubelet/pods/eb115651-ae12-43d8-8022-766c30575782/volumes"
Oct 30 22:36:05.692973 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:36:05.692930    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:36:08.420584 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:36:08.420548    5510 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6gjbf" containerName="extractor"
Oct 30 22:36:10.338707 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:36:10.338646    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:36:14.892402 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:36:14.892367    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:36:17.091018 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:36:17.090978    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:36:17.373243 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:36:17.373141    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:36:19.491373 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:36:19.491331    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:36:24.178551 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:36:24.178508    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:36:24.374649 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:36:24.374609    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:36:28.898565 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:36:28.898523    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:36:33.484171 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:36:33.484134    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:36:34.173923 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:36:34.173887    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:36:38.085750 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:36:38.085705    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:36:38.421139 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:36:38.421051    5510 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6gjbf" containerName="extractor"
Oct 30 22:36:42.863309 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:36:42.863263    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:36:47.440429 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:36:47.440392    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:36:50.726701 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:36:50.726640    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:36:50.981356 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:36:50.981268    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:36:51.997090 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:36:51.997043    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:36:56.605516 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:36:56.605478    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:37:01.202025 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:37:01.201942    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:37:05.794958 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:37:05.794919    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:37:08.420033 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:37:08.419999    5510 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6gjbf" containerName="extractor"
Oct 30 22:37:08.644804 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:37:08.644774    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-89h6s"]
Oct 30 22:37:09.188875 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:37:09.188816    5510 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-89h6s" podUID="85f900f2-2722-4143-8974-d1c7ab81770b" containerName="istio-proxy" containerID="cri-o://734bd0c319d99e4c352fe98062c5d98140fcaf5ac298096be7c343827d2ece60" gracePeriod=30
Oct 30 22:37:10.405041 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:37:10.404999    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:37:14.446857 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:37:14.446829    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-89h6s"
Oct 30 22:37:14.609617 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:37:14.609388    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/85f900f2-2722-4143-8974-d1c7ab81770b-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/85f900f2-2722-4143-8974-d1c7ab81770b-workload-certs\") pod \"85f900f2-2722-4143-8974-d1c7ab81770b\" (UID: \"85f900f2-2722-4143-8974-d1c7ab81770b\") "
Oct 30 22:37:14.609617 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:37:14.609460    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/85f900f2-2722-4143-8974-d1c7ab81770b-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/85f900f2-2722-4143-8974-d1c7ab81770b-workload-socket\") pod \"85f900f2-2722-4143-8974-d1c7ab81770b\" (UID: \"85f900f2-2722-4143-8974-d1c7ab81770b\") "
Oct 30 22:37:14.609617 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:37:14.609509    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/85f900f2-2722-4143-8974-d1c7ab81770b-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/85f900f2-2722-4143-8974-d1c7ab81770b-istio-data\") pod \"85f900f2-2722-4143-8974-d1c7ab81770b\" (UID: \"85f900f2-2722-4143-8974-d1c7ab81770b\") "
Oct 30 22:37:14.609617 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:37:14.609535    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/85f900f2-2722-4143-8974-d1c7ab81770b-istio-token\" (UniqueName: \"kubernetes.io/projected/85f900f2-2722-4143-8974-d1c7ab81770b-istio-token\") pod \"85f900f2-2722-4143-8974-d1c7ab81770b\" (UID: \"85f900f2-2722-4143-8974-d1c7ab81770b\") "
Oct 30 22:37:14.609617 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:37:14.609566    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/85f900f2-2722-4143-8974-d1c7ab81770b-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/85f900f2-2722-4143-8974-d1c7ab81770b-istiod-ca-cert\") pod \"85f900f2-2722-4143-8974-d1c7ab81770b\" (UID: \"85f900f2-2722-4143-8974-d1c7ab81770b\") "
Oct 30 22:37:14.609617 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:37:14.609584    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/85f900f2-2722-4143-8974-d1c7ab81770b-kube-api-access-wwmj4\" (UniqueName: \"kubernetes.io/projected/85f900f2-2722-4143-8974-d1c7ab81770b-kube-api-access-wwmj4\") pod \"85f900f2-2722-4143-8974-d1c7ab81770b\" (UID: \"85f900f2-2722-4143-8974-d1c7ab81770b\") "
Oct 30 22:37:14.610108 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:37:14.609651    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/85f900f2-2722-4143-8974-d1c7ab81770b-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/85f900f2-2722-4143-8974-d1c7ab81770b-istio-envoy\") pod \"85f900f2-2722-4143-8974-d1c7ab81770b\" (UID: \"85f900f2-2722-4143-8974-d1c7ab81770b\") "
Oct 30 22:37:14.610108 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:37:14.609701    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/85f900f2-2722-4143-8974-d1c7ab81770b-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/85f900f2-2722-4143-8974-d1c7ab81770b-credential-socket\") pod \"85f900f2-2722-4143-8974-d1c7ab81770b\" (UID: \"85f900f2-2722-4143-8974-d1c7ab81770b\") "
Oct 30 22:37:14.610108 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:37:14.609759    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/85f900f2-2722-4143-8974-d1c7ab81770b-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/85f900f2-2722-4143-8974-d1c7ab81770b-istio-podinfo\") pod \"85f900f2-2722-4143-8974-d1c7ab81770b\" (UID: \"85f900f2-2722-4143-8974-d1c7ab81770b\") "
Oct 30 22:37:14.610928 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:37:14.610315    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/85f900f2-2722-4143-8974-d1c7ab81770b-workload-socket" pod "85f900f2-2722-4143-8974-d1c7ab81770b" (UID: "85f900f2-2722-4143-8974-d1c7ab81770b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:37:14.610928 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:37:14.610484    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/85f900f2-2722-4143-8974-d1c7ab81770b-istio-data" pod "85f900f2-2722-4143-8974-d1c7ab81770b" (UID: "85f900f2-2722-4143-8974-d1c7ab81770b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:37:14.610928 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:37:14.610558    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/85f900f2-2722-4143-8974-d1c7ab81770b-credential-socket" pod "85f900f2-2722-4143-8974-d1c7ab81770b" (UID: "85f900f2-2722-4143-8974-d1c7ab81770b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:37:14.610928 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:37:14.610575    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/85f900f2-2722-4143-8974-d1c7ab81770b-workload-certs" pod "85f900f2-2722-4143-8974-d1c7ab81770b" (UID: "85f900f2-2722-4143-8974-d1c7ab81770b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:37:14.610928 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:37:14.610874    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/85f900f2-2722-4143-8974-d1c7ab81770b-istiod-ca-cert" pod "85f900f2-2722-4143-8974-d1c7ab81770b" (UID: "85f900f2-2722-4143-8974-d1c7ab81770b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:37:14.613037 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:37:14.613005    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/85f900f2-2722-4143-8974-d1c7ab81770b-istio-token" pod "85f900f2-2722-4143-8974-d1c7ab81770b" (UID: "85f900f2-2722-4143-8974-d1c7ab81770b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:37:14.613560 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:37:14.613108    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/85f900f2-2722-4143-8974-d1c7ab81770b-istio-podinfo" pod "85f900f2-2722-4143-8974-d1c7ab81770b" (UID: "85f900f2-2722-4143-8974-d1c7ab81770b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:37:14.613560 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:37:14.613121    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/85f900f2-2722-4143-8974-d1c7ab81770b-istio-envoy" pod "85f900f2-2722-4143-8974-d1c7ab81770b" (UID: "85f900f2-2722-4143-8974-d1c7ab81770b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:37:14.613560 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:37:14.613147    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/85f900f2-2722-4143-8974-d1c7ab81770b-kube-api-access-wwmj4" pod "85f900f2-2722-4143-8974-d1c7ab81770b" (UID: "85f900f2-2722-4143-8974-d1c7ab81770b"). InnerVolumeSpecName "kube-api-access-wwmj4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:37:14.711606 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:37:14.711566    5510 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/85f900f2-2722-4143-8974-d1c7ab81770b-istiod-ca-cert\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:37:14.711606 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:37:14.711597    5510 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wwmj4\" (UniqueName: \"kubernetes.io/projected/85f900f2-2722-4143-8974-d1c7ab81770b-kube-api-access-wwmj4\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:37:14.711606 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:37:14.711611    5510 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/85f900f2-2722-4143-8974-d1c7ab81770b-istio-envoy\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:37:14.711898 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:37:14.711624    5510 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/85f900f2-2722-4143-8974-d1c7ab81770b-credential-socket\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:37:14.711898 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:37:14.711638    5510 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/85f900f2-2722-4143-8974-d1c7ab81770b-istio-podinfo\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:37:14.711898 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:37:14.711649    5510 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/85f900f2-2722-4143-8974-d1c7ab81770b-workload-certs\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:37:14.711898 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:37:14.711690    5510 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/85f900f2-2722-4143-8974-d1c7ab81770b-workload-socket\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:37:14.711898 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:37:14.711703    5510 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/85f900f2-2722-4143-8974-d1c7ab81770b-istio-data\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:37:14.711898 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:37:14.711715    5510 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/85f900f2-2722-4143-8974-d1c7ab81770b-istio-token\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:37:15.011915 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:37:15.011806    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:37:15.223635 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:37:15.223599    5510 generic.go:356] "Generic (PLEG): container finished" podID="85f900f2-2722-4143-8974-d1c7ab81770b" containerID="734bd0c319d99e4c352fe98062c5d98140fcaf5ac298096be7c343827d2ece60" exitCode=0
Oct 30 22:37:15.223859 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:37:15.223675    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-89h6s"
Oct 30 22:37:15.223859 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:37:15.223684    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-89h6s" event={"ID":"85f900f2-2722-4143-8974-d1c7ab81770b","Type":"ContainerDied","Data":"734bd0c319d99e4c352fe98062c5d98140fcaf5ac298096be7c343827d2ece60"}
Oct 30 22:37:15.223859 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:37:15.223740    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-89h6s" event={"ID":"85f900f2-2722-4143-8974-d1c7ab81770b","Type":"ContainerDied","Data":"62dd34e3fde3204018c7f692174b98fcfdc439a4d658339544915025ae14de8f"}
Oct 30 22:37:15.223859 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:37:15.223761    5510 scope.go:122] "RemoveContainer" containerID="734bd0c319d99e4c352fe98062c5d98140fcaf5ac298096be7c343827d2ece60"
Oct 30 22:37:15.233392 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:37:15.233375    5510 scope.go:122] "RemoveContainer" containerID="734bd0c319d99e4c352fe98062c5d98140fcaf5ac298096be7c343827d2ece60"
Oct 30 22:37:15.233741 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:37:15.233718    5510 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"734bd0c319d99e4c352fe98062c5d98140fcaf5ac298096be7c343827d2ece60\": container with ID starting with 734bd0c319d99e4c352fe98062c5d98140fcaf5ac298096be7c343827d2ece60 not found: ID does not exist" containerID="734bd0c319d99e4c352fe98062c5d98140fcaf5ac298096be7c343827d2ece60"
Oct 30 22:37:15.233832 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:37:15.233750    5510 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"734bd0c319d99e4c352fe98062c5d98140fcaf5ac298096be7c343827d2ece60"} err="failed to get container status \"734bd0c319d99e4c352fe98062c5d98140fcaf5ac298096be7c343827d2ece60\": rpc error: code = NotFound desc = could not find container \"734bd0c319d99e4c352fe98062c5d98140fcaf5ac298096be7c343827d2ece60\": container with ID starting with 734bd0c319d99e4c352fe98062c5d98140fcaf5ac298096be7c343827d2ece60 not found: ID does not exist"
Oct 30 22:37:15.427668 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:37:15.427633    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-89h6s"]
Oct 30 22:37:15.474416 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:37:15.474387    5510 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-89h6s"]
Oct 30 22:37:15.552696 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:37:15.552643    5510 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="85f900f2-2722-4143-8974-d1c7ab81770b" path="/var/lib/kubelet/pods/85f900f2-2722-4143-8974-d1c7ab81770b/volumes"
Oct 30 22:37:19.613318 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:37:19.613267    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:37:24.315297 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:37:24.315248    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:37:28.895471 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:37:28.895429    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:37:33.533338 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:37:33.533292    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:37:34.504547 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:37:34.504499    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:37:34.708033 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:37:34.707990    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:37:38.183179 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:37:38.183119    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:37:38.419470 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:37:38.419435    5510 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6gjbf" containerName="extractor"
Oct 30 22:37:42.791279 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:37:42.791220    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:37:47.422777 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:37:47.422738    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:37:52.093403 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:37:52.093366    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:37:56.684338 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:37:56.684282    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:38:01.327477 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:38:01.327437    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:38:03.289001 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:38:03.288964    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-506964/test-model-506964-kserve-bc8f9f68c-bfkq5"]
Oct 30 22:38:03.361855 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:38:03.361824    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:38:06.135405 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:38:06.135363    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:38:08.419198 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:38:08.419162    5510 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6gjbf" containerName="extractor"
Oct 30 22:38:11.004129 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:38:11.004093    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:38:13.597000 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:38:13.596965    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:38:14.606023 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:38:14.605991    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:38:14.828945 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:38:14.828905    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:38:15.788215 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:38:15.788180    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:38:18.309191 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:38:18.309143    5510 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-506964/test-model-506964-kserve-bc8f9f68c-bfkq5" podUID="b3eaa015-ceb2-41a4-b52f-67dd79e72b17" containerName="main" containerID="cri-o://2667979b3f05d9ae75d6558234e3020cbfacdf3ebc0af93dec147a881b9c013b" gracePeriod=45
Oct 30 22:38:18.463012 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:38:18.462966    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-inf-b-506964/test-model-506964-kserve-bc8f9f68c-bfkq5" podUID="b3eaa015-ceb2-41a4-b52f-67dd79e72b17" containerName="main" probeResult="failure" output="Get \"https://10.133.0.147:8000/ready\": dial tcp 10.133.0.147:8000: connect: connection refused"
Oct 30 22:38:18.574397 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:38:18.574323    5510 generic.go:356] "Generic (PLEG): container finished" podID="b3eaa015-ceb2-41a4-b52f-67dd79e72b17" containerID="2667979b3f05d9ae75d6558234e3020cbfacdf3ebc0af93dec147a881b9c013b" exitCode=0
Oct 30 22:38:18.574539 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:38:18.574397    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-506964/test-model-506964-kserve-bc8f9f68c-bfkq5" event={"ID":"b3eaa015-ceb2-41a4-b52f-67dd79e72b17","Type":"ContainerDied","Data":"2667979b3f05d9ae75d6558234e3020cbfacdf3ebc0af93dec147a881b9c013b"}
Oct 30 22:38:18.574539 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:38:18.574447    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-506964/test-model-506964-kserve-bc8f9f68c-bfkq5" event={"ID":"b3eaa015-ceb2-41a4-b52f-67dd79e72b17","Type":"ContainerDied","Data":"056b342a85f56206e8ad988343886b25be812af1062a93da988cd32f31ccd53c"}
Oct 30 22:38:18.574539 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:38:18.574464    5510 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="056b342a85f56206e8ad988343886b25be812af1062a93da988cd32f31ccd53c"
Oct 30 22:38:18.576251 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:38:18.576235    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-506964/test-model-506964-kserve-bc8f9f68c-bfkq5"
Oct 30 22:38:18.726815 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:38:18.726783    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b3eaa015-ceb2-41a4-b52f-67dd79e72b17-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b3eaa015-ceb2-41a4-b52f-67dd79e72b17-model-cache\") pod \"b3eaa015-ceb2-41a4-b52f-67dd79e72b17\" (UID: \"b3eaa015-ceb2-41a4-b52f-67dd79e72b17\") "
Oct 30 22:38:18.727027 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:38:18.726948    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b3eaa015-ceb2-41a4-b52f-67dd79e72b17-kube-api-access-pqlwd\" (UniqueName: \"kubernetes.io/projected/b3eaa015-ceb2-41a4-b52f-67dd79e72b17-kube-api-access-pqlwd\") pod \"b3eaa015-ceb2-41a4-b52f-67dd79e72b17\" (UID: \"b3eaa015-ceb2-41a4-b52f-67dd79e72b17\") "
Oct 30 22:38:18.727027 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:38:18.727021    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b3eaa015-ceb2-41a4-b52f-67dd79e72b17-tls-certs\" (UniqueName: \"kubernetes.io/secret/b3eaa015-ceb2-41a4-b52f-67dd79e72b17-tls-certs\") pod \"b3eaa015-ceb2-41a4-b52f-67dd79e72b17\" (UID: \"b3eaa015-ceb2-41a4-b52f-67dd79e72b17\") "
Oct 30 22:38:18.727151 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:38:18.727051    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b3eaa015-ceb2-41a4-b52f-67dd79e72b17-dshm\" (UniqueName: \"kubernetes.io/empty-dir/b3eaa015-ceb2-41a4-b52f-67dd79e72b17-dshm\") pod \"b3eaa015-ceb2-41a4-b52f-67dd79e72b17\" (UID: \"b3eaa015-ceb2-41a4-b52f-67dd79e72b17\") "
Oct 30 22:38:18.727151 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:38:18.727077    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b3eaa015-ceb2-41a4-b52f-67dd79e72b17-model-cache" pod "b3eaa015-ceb2-41a4-b52f-67dd79e72b17" (UID: "b3eaa015-ceb2-41a4-b52f-67dd79e72b17"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:38:18.727151 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:38:18.727121    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b3eaa015-ceb2-41a4-b52f-67dd79e72b17-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b3eaa015-ceb2-41a4-b52f-67dd79e72b17-tmp-dir\") pod \"b3eaa015-ceb2-41a4-b52f-67dd79e72b17\" (UID: \"b3eaa015-ceb2-41a4-b52f-67dd79e72b17\") "
Oct 30 22:38:18.727314 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:38:18.727150    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b3eaa015-ceb2-41a4-b52f-67dd79e72b17-home\" (UniqueName: \"kubernetes.io/empty-dir/b3eaa015-ceb2-41a4-b52f-67dd79e72b17-home\") pod \"b3eaa015-ceb2-41a4-b52f-67dd79e72b17\" (UID: \"b3eaa015-ceb2-41a4-b52f-67dd79e72b17\") "
Oct 30 22:38:18.727367 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:38:18.727323    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b3eaa015-ceb2-41a4-b52f-67dd79e72b17-tmp-dir" pod "b3eaa015-ceb2-41a4-b52f-67dd79e72b17" (UID: "b3eaa015-ceb2-41a4-b52f-67dd79e72b17"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:38:18.727483 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:38:18.727442    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b3eaa015-ceb2-41a4-b52f-67dd79e72b17-home" pod "b3eaa015-ceb2-41a4-b52f-67dd79e72b17" (UID: "b3eaa015-ceb2-41a4-b52f-67dd79e72b17"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:38:18.727605 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:38:18.727587    5510 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b3eaa015-ceb2-41a4-b52f-67dd79e72b17-home\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:38:18.727685 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:38:18.727610    5510 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b3eaa015-ceb2-41a4-b52f-67dd79e72b17-model-cache\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:38:18.727685 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:38:18.727624    5510 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b3eaa015-ceb2-41a4-b52f-67dd79e72b17-tmp-dir\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:38:18.729325 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:38:18.729295    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b3eaa015-ceb2-41a4-b52f-67dd79e72b17-dshm" pod "b3eaa015-ceb2-41a4-b52f-67dd79e72b17" (UID: "b3eaa015-ceb2-41a4-b52f-67dd79e72b17"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:38:18.729452 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:38:18.729334    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b3eaa015-ceb2-41a4-b52f-67dd79e72b17-kube-api-access-pqlwd" pod "b3eaa015-ceb2-41a4-b52f-67dd79e72b17" (UID: "b3eaa015-ceb2-41a4-b52f-67dd79e72b17"). InnerVolumeSpecName "kube-api-access-pqlwd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:38:18.729452 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:38:18.729373    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b3eaa015-ceb2-41a4-b52f-67dd79e72b17-tls-certs" pod "b3eaa015-ceb2-41a4-b52f-67dd79e72b17" (UID: "b3eaa015-ceb2-41a4-b52f-67dd79e72b17"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:38:18.828705 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:38:18.828603    5510 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pqlwd\" (UniqueName: \"kubernetes.io/projected/b3eaa015-ceb2-41a4-b52f-67dd79e72b17-kube-api-access-pqlwd\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:38:18.828705 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:38:18.828635    5510 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b3eaa015-ceb2-41a4-b52f-67dd79e72b17-tls-certs\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:38:18.828705 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:38:18.828645    5510 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b3eaa015-ceb2-41a4-b52f-67dd79e72b17-dshm\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:38:19.581075 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:38:19.581042    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-506964/test-model-506964-kserve-bc8f9f68c-bfkq5"
Oct 30 22:38:19.736794 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:38:19.736750    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-506964/test-model-506964-kserve-bc8f9f68c-bfkq5"]
Oct 30 22:38:19.783204 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:38:19.783166    5510 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-506964/test-model-506964-kserve-bc8f9f68c-bfkq5"]
Oct 30 22:38:20.293518 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:38:20.293483    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:38:21.552986 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:38:21.552952    5510 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b3eaa015-ceb2-41a4-b52f-67dd79e72b17" path="/var/lib/kubelet/pods/b3eaa015-ceb2-41a4-b52f-67dd79e72b17/volumes"
Oct 30 22:38:25.033171 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:38:25.033134    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:38:29.700204 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:38:29.700161    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:38:33.697981 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:38:33.697940    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:38:34.837280 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:38:34.837240    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:38:35.183519 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:38:35.183435    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:38:35.394369 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:38:35.394326    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:38:35.618605 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:38:35.618560    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:38:36.251249 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:38:36.251213    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:38:38.420322 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:38:38.420287    5510 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6gjbf" containerName="extractor"
Oct 30 22:38:39.485506 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:38:39.485470    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:38:39.851991 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:38:39.851947    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:38:40.500689 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:38:40.500642    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:38:42.892136 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:38:42.892095    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:38:43.187509 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:38:43.187430    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:38:43.404629 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:38:43.404591    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:38:45.314634 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:38:45.314596    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:38:45.369645 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:38:45.369609    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-bxlgp"]
Oct 30 22:38:45.369903 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:38:45.369874    5510 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-bxlgp" podUID="f789c7e9-26e6-4b2d-b1d7-536dc5d526d8" containerName="istio-proxy" containerID="cri-o://8dee132045c366b87c195a3373e8435381cc8da438522107be31803f8cafbc07" gracePeriod=30
Oct 30 22:38:50.195002 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:38:50.194963    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:38:50.629235 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:38:50.629209    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-bxlgp"
Oct 30 22:38:50.669926 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:38:50.669894    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f789c7e9-26e6-4b2d-b1d7-536dc5d526d8-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f789c7e9-26e6-4b2d-b1d7-536dc5d526d8-istiod-ca-cert\") pod \"f789c7e9-26e6-4b2d-b1d7-536dc5d526d8\" (UID: \"f789c7e9-26e6-4b2d-b1d7-536dc5d526d8\") "
Oct 30 22:38:50.670094 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:38:50.669937    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f789c7e9-26e6-4b2d-b1d7-536dc5d526d8-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f789c7e9-26e6-4b2d-b1d7-536dc5d526d8-credential-socket\") pod \"f789c7e9-26e6-4b2d-b1d7-536dc5d526d8\" (UID: \"f789c7e9-26e6-4b2d-b1d7-536dc5d526d8\") "
Oct 30 22:38:50.670094 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:38:50.669980    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/f789c7e9-26e6-4b2d-b1d7-536dc5d526d8-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f789c7e9-26e6-4b2d-b1d7-536dc5d526d8-istio-podinfo\") pod \"f789c7e9-26e6-4b2d-b1d7-536dc5d526d8\" (UID: \"f789c7e9-26e6-4b2d-b1d7-536dc5d526d8\") "
Oct 30 22:38:50.670228 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:38:50.670143    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f789c7e9-26e6-4b2d-b1d7-536dc5d526d8-kube-api-access-8ckpx\" (UniqueName: \"kubernetes.io/projected/f789c7e9-26e6-4b2d-b1d7-536dc5d526d8-kube-api-access-8ckpx\") pod \"f789c7e9-26e6-4b2d-b1d7-536dc5d526d8\" (UID: \"f789c7e9-26e6-4b2d-b1d7-536dc5d526d8\") "
Oct 30 22:38:50.670228 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:38:50.670182    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f789c7e9-26e6-4b2d-b1d7-536dc5d526d8-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f789c7e9-26e6-4b2d-b1d7-536dc5d526d8-workload-certs\") pod \"f789c7e9-26e6-4b2d-b1d7-536dc5d526d8\" (UID: \"f789c7e9-26e6-4b2d-b1d7-536dc5d526d8\") "
Oct 30 22:38:50.670342 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:38:50.670265    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f789c7e9-26e6-4b2d-b1d7-536dc5d526d8-istio-token\" (UniqueName: \"kubernetes.io/projected/f789c7e9-26e6-4b2d-b1d7-536dc5d526d8-istio-token\") pod \"f789c7e9-26e6-4b2d-b1d7-536dc5d526d8\" (UID: \"f789c7e9-26e6-4b2d-b1d7-536dc5d526d8\") "
Oct 30 22:38:50.670342 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:38:50.670300    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f789c7e9-26e6-4b2d-b1d7-536dc5d526d8-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f789c7e9-26e6-4b2d-b1d7-536dc5d526d8-istio-envoy\") pod \"f789c7e9-26e6-4b2d-b1d7-536dc5d526d8\" (UID: \"f789c7e9-26e6-4b2d-b1d7-536dc5d526d8\") "
Oct 30 22:38:50.670342 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:38:50.670327    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f789c7e9-26e6-4b2d-b1d7-536dc5d526d8-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f789c7e9-26e6-4b2d-b1d7-536dc5d526d8-workload-socket\") pod \"f789c7e9-26e6-4b2d-b1d7-536dc5d526d8\" (UID: \"f789c7e9-26e6-4b2d-b1d7-536dc5d526d8\") "
Oct 30 22:38:50.670491 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:38:50.670363    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f789c7e9-26e6-4b2d-b1d7-536dc5d526d8-credential-socket" pod "f789c7e9-26e6-4b2d-b1d7-536dc5d526d8" (UID: "f789c7e9-26e6-4b2d-b1d7-536dc5d526d8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:38:50.670491 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:38:50.670379    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f789c7e9-26e6-4b2d-b1d7-536dc5d526d8-istiod-ca-cert" pod "f789c7e9-26e6-4b2d-b1d7-536dc5d526d8" (UID: "f789c7e9-26e6-4b2d-b1d7-536dc5d526d8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:38:50.670491 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:38:50.670376    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f789c7e9-26e6-4b2d-b1d7-536dc5d526d8-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f789c7e9-26e6-4b2d-b1d7-536dc5d526d8-istio-data\") pod \"f789c7e9-26e6-4b2d-b1d7-536dc5d526d8\" (UID: \"f789c7e9-26e6-4b2d-b1d7-536dc5d526d8\") "
Oct 30 22:38:50.670637 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:38:50.670511    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f789c7e9-26e6-4b2d-b1d7-536dc5d526d8-workload-certs" pod "f789c7e9-26e6-4b2d-b1d7-536dc5d526d8" (UID: "f789c7e9-26e6-4b2d-b1d7-536dc5d526d8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:38:50.670719 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:38:50.670619    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f789c7e9-26e6-4b2d-b1d7-536dc5d526d8-istio-data" pod "f789c7e9-26e6-4b2d-b1d7-536dc5d526d8" (UID: "f789c7e9-26e6-4b2d-b1d7-536dc5d526d8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:38:50.670767 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:38:50.670714    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f789c7e9-26e6-4b2d-b1d7-536dc5d526d8-workload-socket" pod "f789c7e9-26e6-4b2d-b1d7-536dc5d526d8" (UID: "f789c7e9-26e6-4b2d-b1d7-536dc5d526d8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:38:50.670982 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:38:50.670957    5510 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f789c7e9-26e6-4b2d-b1d7-536dc5d526d8-workload-socket\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:38:50.671078 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:38:50.670985    5510 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f789c7e9-26e6-4b2d-b1d7-536dc5d526d8-istio-data\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:38:50.671078 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:38:50.671000    5510 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f789c7e9-26e6-4b2d-b1d7-536dc5d526d8-istiod-ca-cert\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:38:50.671078 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:38:50.671013    5510 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f789c7e9-26e6-4b2d-b1d7-536dc5d526d8-credential-socket\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:38:50.671078 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:38:50.671026    5510 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f789c7e9-26e6-4b2d-b1d7-536dc5d526d8-workload-certs\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:38:50.672320 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:38:50.672292    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f789c7e9-26e6-4b2d-b1d7-536dc5d526d8-istio-podinfo" pod "f789c7e9-26e6-4b2d-b1d7-536dc5d526d8" (UID: "f789c7e9-26e6-4b2d-b1d7-536dc5d526d8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:38:50.672435 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:38:50.672369    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f789c7e9-26e6-4b2d-b1d7-536dc5d526d8-istio-token" pod "f789c7e9-26e6-4b2d-b1d7-536dc5d526d8" (UID: "f789c7e9-26e6-4b2d-b1d7-536dc5d526d8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:38:50.672435 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:38:50.672388    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f789c7e9-26e6-4b2d-b1d7-536dc5d526d8-kube-api-access-8ckpx" pod "f789c7e9-26e6-4b2d-b1d7-536dc5d526d8" (UID: "f789c7e9-26e6-4b2d-b1d7-536dc5d526d8"). InnerVolumeSpecName "kube-api-access-8ckpx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:38:50.672553 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:38:50.672533    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f789c7e9-26e6-4b2d-b1d7-536dc5d526d8-istio-envoy" pod "f789c7e9-26e6-4b2d-b1d7-536dc5d526d8" (UID: "f789c7e9-26e6-4b2d-b1d7-536dc5d526d8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:38:50.751738 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:38:50.751623    5510 generic.go:356] "Generic (PLEG): container finished" podID="f789c7e9-26e6-4b2d-b1d7-536dc5d526d8" containerID="8dee132045c366b87c195a3373e8435381cc8da438522107be31803f8cafbc07" exitCode=0
Oct 30 22:38:50.751738 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:38:50.751691    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-bxlgp"
Oct 30 22:38:50.751738 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:38:50.751689    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-bxlgp" event={"ID":"f789c7e9-26e6-4b2d-b1d7-536dc5d526d8","Type":"ContainerDied","Data":"8dee132045c366b87c195a3373e8435381cc8da438522107be31803f8cafbc07"}
Oct 30 22:38:50.751987 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:38:50.751748    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-bxlgp" event={"ID":"f789c7e9-26e6-4b2d-b1d7-536dc5d526d8","Type":"ContainerDied","Data":"0c127be2182b40f2dd4e92eaa96fc43698b20e9e54f975988d2b68929bd8ede2"}
Oct 30 22:38:50.751987 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:38:50.751770    5510 scope.go:122] "RemoveContainer" containerID="8dee132045c366b87c195a3373e8435381cc8da438522107be31803f8cafbc07"
Oct 30 22:38:50.762448 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:38:50.762428    5510 scope.go:122] "RemoveContainer" containerID="8dee132045c366b87c195a3373e8435381cc8da438522107be31803f8cafbc07"
Oct 30 22:38:50.762734 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:38:50.762716    5510 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8dee132045c366b87c195a3373e8435381cc8da438522107be31803f8cafbc07\": container with ID starting with 8dee132045c366b87c195a3373e8435381cc8da438522107be31803f8cafbc07 not found: ID does not exist" containerID="8dee132045c366b87c195a3373e8435381cc8da438522107be31803f8cafbc07"
Oct 30 22:38:50.762791 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:38:50.762743    5510 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8dee132045c366b87c195a3373e8435381cc8da438522107be31803f8cafbc07"} err="failed to get container status \"8dee132045c366b87c195a3373e8435381cc8da438522107be31803f8cafbc07\": rpc error: code = NotFound desc = could not find container \"8dee132045c366b87c195a3373e8435381cc8da438522107be31803f8cafbc07\": container with ID starting with 8dee132045c366b87c195a3373e8435381cc8da438522107be31803f8cafbc07 not found: ID does not exist"
Oct 30 22:38:50.772386 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:38:50.772364    5510 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8ckpx\" (UniqueName: \"kubernetes.io/projected/f789c7e9-26e6-4b2d-b1d7-536dc5d526d8-kube-api-access-8ckpx\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:38:50.772386 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:38:50.772387    5510 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f789c7e9-26e6-4b2d-b1d7-536dc5d526d8-istio-token\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:38:50.772521 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:38:50.772397    5510 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f789c7e9-26e6-4b2d-b1d7-536dc5d526d8-istio-envoy\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:38:50.772521 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:38:50.772406    5510 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f789c7e9-26e6-4b2d-b1d7-536dc5d526d8-istio-podinfo\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:38:50.900609 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:38:50.900573    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-bxlgp"]
Oct 30 22:38:50.906040 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:38:50.906008    5510 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-fb0b93-openshift-default-75ccb77bb5-bxlgp"]
Oct 30 22:38:51.552719 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:38:51.552684    5510 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f789c7e9-26e6-4b2d-b1d7-536dc5d526d8" path="/var/lib/kubelet/pods/f789c7e9-26e6-4b2d-b1d7-536dc5d526d8/volumes"
Oct 30 22:38:54.471176 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:38:54.471139    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-506964-56d6b85d7f-2lbbz"]
Oct 30 22:38:54.902786 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:38:54.902745    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:38:55.110762 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:38:55.110719    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:38:59.472151 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:38:59.472100    5510 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-b-506964-56d6b85d7f-2lbbz" podUID="d29d64dd-be94-4d3b-9888-3196c0c8b5f0" containerName="payload-processing" containerID="cri-o://7f9881c346cbeb848a99b083ddd523952bf71f18ef64f1eeeee7af9061b6a6d0" gracePeriod=25
Oct 30 22:38:59.684136 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:38:59.683975    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:38:59.730777 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:38:59.730715    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-506964-56d6b85d7f-2lbbz"
Oct 30 22:38:59.767729 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:38:59.767687    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d29d64dd-be94-4d3b-9888-3196c0c8b5f0-kube-api-access-ddbjk\" (UniqueName: \"kubernetes.io/projected/d29d64dd-be94-4d3b-9888-3196c0c8b5f0-kube-api-access-ddbjk\") pod \"d29d64dd-be94-4d3b-9888-3196c0c8b5f0\" (UID: \"d29d64dd-be94-4d3b-9888-3196c0c8b5f0\") "
Oct 30 22:38:59.767973 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:38:59.767784    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d29d64dd-be94-4d3b-9888-3196c0c8b5f0-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d29d64dd-be94-4d3b-9888-3196c0c8b5f0-plugins-config-volume\") pod \"d29d64dd-be94-4d3b-9888-3196c0c8b5f0\" (UID: \"d29d64dd-be94-4d3b-9888-3196c0c8b5f0\") "
Oct 30 22:38:59.768164 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:38:59.768142    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d29d64dd-be94-4d3b-9888-3196c0c8b5f0-plugins-config-volume" pod "d29d64dd-be94-4d3b-9888-3196c0c8b5f0" (UID: "d29d64dd-be94-4d3b-9888-3196c0c8b5f0"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:38:59.769741 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:38:59.769721    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d29d64dd-be94-4d3b-9888-3196c0c8b5f0-kube-api-access-ddbjk" pod "d29d64dd-be94-4d3b-9888-3196c0c8b5f0" (UID: "d29d64dd-be94-4d3b-9888-3196c0c8b5f0"). InnerVolumeSpecName "kube-api-access-ddbjk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:38:59.800884 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:38:59.800858    5510 generic.go:356] "Generic (PLEG): container finished" podID="d29d64dd-be94-4d3b-9888-3196c0c8b5f0" containerID="7f9881c346cbeb848a99b083ddd523952bf71f18ef64f1eeeee7af9061b6a6d0" exitCode=0
Oct 30 22:38:59.801010 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:38:59.800904    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-506964-56d6b85d7f-2lbbz"
Oct 30 22:38:59.801010 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:38:59.800945    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-506964-56d6b85d7f-2lbbz" event={"ID":"d29d64dd-be94-4d3b-9888-3196c0c8b5f0","Type":"ContainerDied","Data":"7f9881c346cbeb848a99b083ddd523952bf71f18ef64f1eeeee7af9061b6a6d0"}
Oct 30 22:38:59.801010 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:38:59.800999    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-506964-56d6b85d7f-2lbbz" event={"ID":"d29d64dd-be94-4d3b-9888-3196c0c8b5f0","Type":"ContainerDied","Data":"97477b55143a11454f026c4dfe3210c2cffea005c349ce6982b15f8cdad34546"}
Oct 30 22:38:59.801129 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:38:59.801022    5510 scope.go:122] "RemoveContainer" containerID="7f9881c346cbeb848a99b083ddd523952bf71f18ef64f1eeeee7af9061b6a6d0"
Oct 30 22:38:59.810112 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:38:59.810096    5510 scope.go:122] "RemoveContainer" containerID="7f9881c346cbeb848a99b083ddd523952bf71f18ef64f1eeeee7af9061b6a6d0"
Oct 30 22:38:59.810378 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:38:59.810360    5510 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7f9881c346cbeb848a99b083ddd523952bf71f18ef64f1eeeee7af9061b6a6d0\": container with ID starting with 7f9881c346cbeb848a99b083ddd523952bf71f18ef64f1eeeee7af9061b6a6d0 not found: ID does not exist" containerID="7f9881c346cbeb848a99b083ddd523952bf71f18ef64f1eeeee7af9061b6a6d0"
Oct 30 22:38:59.810482 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:38:59.810387    5510 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7f9881c346cbeb848a99b083ddd523952bf71f18ef64f1eeeee7af9061b6a6d0"} err="failed to get container status \"7f9881c346cbeb848a99b083ddd523952bf71f18ef64f1eeeee7af9061b6a6d0\": rpc error: code = NotFound desc = could not find container \"7f9881c346cbeb848a99b083ddd523952bf71f18ef64f1eeeee7af9061b6a6d0\": container with ID starting with 7f9881c346cbeb848a99b083ddd523952bf71f18ef64f1eeeee7af9061b6a6d0 not found: ID does not exist"
Oct 30 22:38:59.869624 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:38:59.869590    5510 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ddbjk\" (UniqueName: \"kubernetes.io/projected/d29d64dd-be94-4d3b-9888-3196c0c8b5f0-kube-api-access-ddbjk\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:38:59.869624 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:38:59.869623    5510 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d29d64dd-be94-4d3b-9888-3196c0c8b5f0-plugins-config-volume\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:38:59.946538 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:38:59.946502    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-506964-56d6b85d7f-2lbbz"]
Oct 30 22:38:59.952631 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:38:59.952606    5510 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-506964-56d6b85d7f-2lbbz"]
Oct 30 22:39:01.552447 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:39:01.552403    5510 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d29d64dd-be94-4d3b-9888-3196c0c8b5f0" path="/var/lib/kubelet/pods/d29d64dd-be94-4d3b-9888-3196c0c8b5f0/volumes"
Oct 30 22:39:04.290939 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:39:04.290902    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:39:06.430993 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:39:06.430956    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-07e9c8/test-model-07e9c8-kserve-9c9cbf4bc-b88r4"]
Oct 30 22:39:06.504120 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:39:06.504064    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:39:08.420588 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:39:08.420553    5510 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6gjbf" containerName="extractor"
Oct 30 22:39:08.980316 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:39:08.980280    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:39:13.907483 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:39:13.907444    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:39:17.724939 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:39:17.724895    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:39:17.966978 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:39:17.966941    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:39:18.483885 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:39:18.483849    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:39:21.451191 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:39:21.451149    5510 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-07e9c8/test-model-07e9c8-kserve-9c9cbf4bc-b88r4" podUID="6824a63b-432d-4967-9b83-742599c9cd94" containerName="main" containerID="cri-o://ff16cfb1db3109775a5076d569bd19166470c44b7ef63f2d8bc5828f6cada460" gracePeriod=45
Oct 30 22:39:21.702038 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:39:21.701976    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-07e9c8/test-model-07e9c8-kserve-9c9cbf4bc-b88r4"
Oct 30 22:39:21.804112 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:39:21.804079    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6824a63b-432d-4967-9b83-742599c9cd94-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6824a63b-432d-4967-9b83-742599c9cd94-model-cache\") pod \"6824a63b-432d-4967-9b83-742599c9cd94\" (UID: \"6824a63b-432d-4967-9b83-742599c9cd94\") "
Oct 30 22:39:21.804313 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:39:21.804130    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/6824a63b-432d-4967-9b83-742599c9cd94-tls-certs\" (UniqueName: \"kubernetes.io/secret/6824a63b-432d-4967-9b83-742599c9cd94-tls-certs\") pod \"6824a63b-432d-4967-9b83-742599c9cd94\" (UID: \"6824a63b-432d-4967-9b83-742599c9cd94\") "
Oct 30 22:39:21.804313 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:39:21.804168    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6824a63b-432d-4967-9b83-742599c9cd94-kube-api-access-p4qt8\" (UniqueName: \"kubernetes.io/projected/6824a63b-432d-4967-9b83-742599c9cd94-kube-api-access-p4qt8\") pod \"6824a63b-432d-4967-9b83-742599c9cd94\" (UID: \"6824a63b-432d-4967-9b83-742599c9cd94\") "
Oct 30 22:39:21.804313 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:39:21.804224    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6824a63b-432d-4967-9b83-742599c9cd94-dshm\" (UniqueName: \"kubernetes.io/empty-dir/6824a63b-432d-4967-9b83-742599c9cd94-dshm\") pod \"6824a63b-432d-4967-9b83-742599c9cd94\" (UID: \"6824a63b-432d-4967-9b83-742599c9cd94\") "
Oct 30 22:39:21.804313 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:39:21.804266    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6824a63b-432d-4967-9b83-742599c9cd94-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6824a63b-432d-4967-9b83-742599c9cd94-tmp-dir\") pod \"6824a63b-432d-4967-9b83-742599c9cd94\" (UID: \"6824a63b-432d-4967-9b83-742599c9cd94\") "
Oct 30 22:39:21.804313 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:39:21.804298    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6824a63b-432d-4967-9b83-742599c9cd94-home\" (UniqueName: \"kubernetes.io/empty-dir/6824a63b-432d-4967-9b83-742599c9cd94-home\") pod \"6824a63b-432d-4967-9b83-742599c9cd94\" (UID: \"6824a63b-432d-4967-9b83-742599c9cd94\") "
Oct 30 22:39:21.804622 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:39:21.804360    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6824a63b-432d-4967-9b83-742599c9cd94-model-cache" pod "6824a63b-432d-4967-9b83-742599c9cd94" (UID: "6824a63b-432d-4967-9b83-742599c9cd94"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:39:21.804622 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:39:21.804551    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6824a63b-432d-4967-9b83-742599c9cd94-tmp-dir" pod "6824a63b-432d-4967-9b83-742599c9cd94" (UID: "6824a63b-432d-4967-9b83-742599c9cd94"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:39:21.804622 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:39:21.804617    5510 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6824a63b-432d-4967-9b83-742599c9cd94-model-cache\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:39:21.804799 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:39:21.804633    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6824a63b-432d-4967-9b83-742599c9cd94-home" pod "6824a63b-432d-4967-9b83-742599c9cd94" (UID: "6824a63b-432d-4967-9b83-742599c9cd94"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:39:21.806560 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:39:21.806537    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6824a63b-432d-4967-9b83-742599c9cd94-kube-api-access-p4qt8" pod "6824a63b-432d-4967-9b83-742599c9cd94" (UID: "6824a63b-432d-4967-9b83-742599c9cd94"). InnerVolumeSpecName "kube-api-access-p4qt8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:39:21.806680 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:39:21.806582    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6824a63b-432d-4967-9b83-742599c9cd94-tls-certs" pod "6824a63b-432d-4967-9b83-742599c9cd94" (UID: "6824a63b-432d-4967-9b83-742599c9cd94"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:39:21.806680 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:39:21.806592    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6824a63b-432d-4967-9b83-742599c9cd94-dshm" pod "6824a63b-432d-4967-9b83-742599c9cd94" (UID: "6824a63b-432d-4967-9b83-742599c9cd94"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:39:21.905485 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:39:21.905448    5510 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6824a63b-432d-4967-9b83-742599c9cd94-tmp-dir\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:39:21.905485 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:39:21.905483    5510 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6824a63b-432d-4967-9b83-742599c9cd94-home\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:39:21.905485 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:39:21.905493    5510 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6824a63b-432d-4967-9b83-742599c9cd94-tls-certs\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:39:21.905848 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:39:21.905504    5510 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p4qt8\" (UniqueName: \"kubernetes.io/projected/6824a63b-432d-4967-9b83-742599c9cd94-kube-api-access-p4qt8\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:39:21.905848 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:39:21.905513    5510 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6824a63b-432d-4967-9b83-742599c9cd94-dshm\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:39:21.919038 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:39:21.919005    5510 generic.go:356] "Generic (PLEG): container finished" podID="6824a63b-432d-4967-9b83-742599c9cd94" containerID="ff16cfb1db3109775a5076d569bd19166470c44b7ef63f2d8bc5828f6cada460" exitCode=0
Oct 30 22:39:21.919185 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:39:21.919048    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-07e9c8/test-model-07e9c8-kserve-9c9cbf4bc-b88r4"
Oct 30 22:39:21.919185 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:39:21.919083    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-07e9c8/test-model-07e9c8-kserve-9c9cbf4bc-b88r4" event={"ID":"6824a63b-432d-4967-9b83-742599c9cd94","Type":"ContainerDied","Data":"ff16cfb1db3109775a5076d569bd19166470c44b7ef63f2d8bc5828f6cada460"}
Oct 30 22:39:21.919185 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:39:21.919126    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-07e9c8/test-model-07e9c8-kserve-9c9cbf4bc-b88r4" event={"ID":"6824a63b-432d-4967-9b83-742599c9cd94","Type":"ContainerDied","Data":"49398718e4308e147ff5582677d084ded61a71967339a97760725ba46917c3e3"}
Oct 30 22:39:21.919185 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:39:21.919144    5510 scope.go:122] "RemoveContainer" containerID="ff16cfb1db3109775a5076d569bd19166470c44b7ef63f2d8bc5828f6cada460"
Oct 30 22:39:21.931475 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:39:21.931449    5510 scope.go:122] "RemoveContainer" containerID="ff16cfb1db3109775a5076d569bd19166470c44b7ef63f2d8bc5828f6cada460"
Oct 30 22:39:21.931939 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:39:21.931900    5510 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ff16cfb1db3109775a5076d569bd19166470c44b7ef63f2d8bc5828f6cada460\": container with ID starting with ff16cfb1db3109775a5076d569bd19166470c44b7ef63f2d8bc5828f6cada460 not found: ID does not exist" containerID="ff16cfb1db3109775a5076d569bd19166470c44b7ef63f2d8bc5828f6cada460"
Oct 30 22:39:21.932018 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:39:21.931946    5510 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ff16cfb1db3109775a5076d569bd19166470c44b7ef63f2d8bc5828f6cada460"} err="failed to get container status \"ff16cfb1db3109775a5076d569bd19166470c44b7ef63f2d8bc5828f6cada460\": rpc error: code = NotFound desc = could not find container \"ff16cfb1db3109775a5076d569bd19166470c44b7ef63f2d8bc5828f6cada460\": container with ID starting with ff16cfb1db3109775a5076d569bd19166470c44b7ef63f2d8bc5828f6cada460 not found: ID does not exist"
Oct 30 22:39:22.078229 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:39:22.078193    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-07e9c8/test-model-07e9c8-kserve-9c9cbf4bc-b88r4"]
Oct 30 22:39:22.085207 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:39:22.085178    5510 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-07e9c8/test-model-07e9c8-kserve-9c9cbf4bc-b88r4"]
Oct 30 22:39:22.568552 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:39:22.568518    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:39:23.551893 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:39:23.551861    5510 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6824a63b-432d-4967-9b83-742599c9cd94" path="/var/lib/kubelet/pods/6824a63b-432d-4967-9b83-742599c9cd94/volumes"
Oct 30 22:39:24.075077 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:39:24.075041    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:39:24.275081 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:39:24.275047    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:39:24.462457 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:39:24.462373    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:39:25.025762 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:39:25.025721    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:39:29.199979 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:39:29.199935    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:39:31.258153 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:39:31.258122    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:39:31.458683 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:39:31.458613    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:39:31.675233 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:39:31.675188    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:39:33.327123 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:39:33.327085    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:39:34.937918 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:39:34.937886    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:39:36.310048 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:39:36.310009    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:39:38.419890 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:39:38.419846    5510 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6gjbf" containerName="extractor"
Oct 30 22:39:39.888250 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:39:39.888213    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-6dft4"]
Oct 30 22:39:39.888686 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:39:39.888423    5510 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-6dft4" podUID="cc069342-244c-467a-9d89-2b7fb41338e9" containerName="istio-proxy" containerID="cri-o://eda1bf976520f473dae4c2d09c2e060f7ddc72c5b16ba293ca1ffbe62b420107" gracePeriod=30
Oct 30 22:39:40.387871 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:39:40.387832    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:39:42.265799 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:39:42.265761    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:39:42.474429 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:39:42.474395    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:39:42.704297 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:39:42.704258    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:39:42.866016 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:39:42.865980    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w1-a6b6a5-7976bff669-4lf96"]
Oct 30 22:39:42.994073 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:39:42.994000    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w1-a6b6a5-85cc5f8687-hdbfk"]
Oct 30 22:39:43.289954 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:39:43.289893    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:39:43.521980 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:39:43.521946    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:39:45.047275 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:39:45.047243    5510 generic.go:356] "Generic (PLEG): container finished" podID="cc069342-244c-467a-9d89-2b7fb41338e9" containerID="eda1bf976520f473dae4c2d09c2e060f7ddc72c5b16ba293ca1ffbe62b420107" exitCode=0
Oct 30 22:39:45.047709 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:39:45.047313    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-6dft4" event={"ID":"cc069342-244c-467a-9d89-2b7fb41338e9","Type":"ContainerDied","Data":"eda1bf976520f473dae4c2d09c2e060f7ddc72c5b16ba293ca1ffbe62b420107"}
Oct 30 22:39:45.143306 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:39:45.143275    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-6dft4"
Oct 30 22:39:45.193493 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:39:45.193397    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:39:45.248055 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:39:45.248022    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cc069342-244c-467a-9d89-2b7fb41338e9-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cc069342-244c-467a-9d89-2b7fb41338e9-istio-envoy\") pod \"cc069342-244c-467a-9d89-2b7fb41338e9\" (UID: \"cc069342-244c-467a-9d89-2b7fb41338e9\") "
Oct 30 22:39:45.248276 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:39:45.248099    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/cc069342-244c-467a-9d89-2b7fb41338e9-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cc069342-244c-467a-9d89-2b7fb41338e9-istio-podinfo\") pod \"cc069342-244c-467a-9d89-2b7fb41338e9\" (UID: \"cc069342-244c-467a-9d89-2b7fb41338e9\") "
Oct 30 22:39:45.248276 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:39:45.248121    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cc069342-244c-467a-9d89-2b7fb41338e9-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cc069342-244c-467a-9d89-2b7fb41338e9-istio-data\") pod \"cc069342-244c-467a-9d89-2b7fb41338e9\" (UID: \"cc069342-244c-467a-9d89-2b7fb41338e9\") "
Oct 30 22:39:45.248276 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:39:45.248168    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cc069342-244c-467a-9d89-2b7fb41338e9-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cc069342-244c-467a-9d89-2b7fb41338e9-workload-certs\") pod \"cc069342-244c-467a-9d89-2b7fb41338e9\" (UID: \"cc069342-244c-467a-9d89-2b7fb41338e9\") "
Oct 30 22:39:45.248276 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:39:45.248195    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cc069342-244c-467a-9d89-2b7fb41338e9-istio-token\" (UniqueName: \"kubernetes.io/projected/cc069342-244c-467a-9d89-2b7fb41338e9-istio-token\") pod \"cc069342-244c-467a-9d89-2b7fb41338e9\" (UID: \"cc069342-244c-467a-9d89-2b7fb41338e9\") "
Oct 30 22:39:45.248276 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:39:45.248225    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cc069342-244c-467a-9d89-2b7fb41338e9-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cc069342-244c-467a-9d89-2b7fb41338e9-workload-socket\") pod \"cc069342-244c-467a-9d89-2b7fb41338e9\" (UID: \"cc069342-244c-467a-9d89-2b7fb41338e9\") "
Oct 30 22:39:45.248579 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:39:45.248277    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cc069342-244c-467a-9d89-2b7fb41338e9-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cc069342-244c-467a-9d89-2b7fb41338e9-credential-socket\") pod \"cc069342-244c-467a-9d89-2b7fb41338e9\" (UID: \"cc069342-244c-467a-9d89-2b7fb41338e9\") "
Oct 30 22:39:45.248579 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:39:45.248335    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/cc069342-244c-467a-9d89-2b7fb41338e9-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cc069342-244c-467a-9d89-2b7fb41338e9-istiod-ca-cert\") pod \"cc069342-244c-467a-9d89-2b7fb41338e9\" (UID: \"cc069342-244c-467a-9d89-2b7fb41338e9\") "
Oct 30 22:39:45.248579 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:39:45.248382    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cc069342-244c-467a-9d89-2b7fb41338e9-kube-api-access-twgnb\" (UniqueName: \"kubernetes.io/projected/cc069342-244c-467a-9d89-2b7fb41338e9-kube-api-access-twgnb\") pod \"cc069342-244c-467a-9d89-2b7fb41338e9\" (UID: \"cc069342-244c-467a-9d89-2b7fb41338e9\") "
Oct 30 22:39:45.248579 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:39:45.248404    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cc069342-244c-467a-9d89-2b7fb41338e9-istio-data" pod "cc069342-244c-467a-9d89-2b7fb41338e9" (UID: "cc069342-244c-467a-9d89-2b7fb41338e9"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:39:45.248817 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:39:45.248570    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cc069342-244c-467a-9d89-2b7fb41338e9-workload-certs" pod "cc069342-244c-467a-9d89-2b7fb41338e9" (UID: "cc069342-244c-467a-9d89-2b7fb41338e9"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:39:45.248817 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:39:45.248626    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cc069342-244c-467a-9d89-2b7fb41338e9-workload-socket" pod "cc069342-244c-467a-9d89-2b7fb41338e9" (UID: "cc069342-244c-467a-9d89-2b7fb41338e9"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:39:45.248817 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:39:45.248688    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cc069342-244c-467a-9d89-2b7fb41338e9-credential-socket" pod "cc069342-244c-467a-9d89-2b7fb41338e9" (UID: "cc069342-244c-467a-9d89-2b7fb41338e9"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:39:45.248943 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:39:45.248870    5510 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cc069342-244c-467a-9d89-2b7fb41338e9-istio-data\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:39:45.248943 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:39:45.248893    5510 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cc069342-244c-467a-9d89-2b7fb41338e9-workload-certs\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:39:45.248943 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:39:45.248910    5510 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cc069342-244c-467a-9d89-2b7fb41338e9-workload-socket\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:39:45.248943 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:39:45.248913    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cc069342-244c-467a-9d89-2b7fb41338e9-istiod-ca-cert" pod "cc069342-244c-467a-9d89-2b7fb41338e9" (UID: "cc069342-244c-467a-9d89-2b7fb41338e9"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:39:45.248943 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:39:45.248925    5510 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cc069342-244c-467a-9d89-2b7fb41338e9-credential-socket\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:39:45.250616 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:39:45.250587    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cc069342-244c-467a-9d89-2b7fb41338e9-istio-token" pod "cc069342-244c-467a-9d89-2b7fb41338e9" (UID: "cc069342-244c-467a-9d89-2b7fb41338e9"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:39:45.250784 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:39:45.250700    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cc069342-244c-467a-9d89-2b7fb41338e9-kube-api-access-twgnb" pod "cc069342-244c-467a-9d89-2b7fb41338e9" (UID: "cc069342-244c-467a-9d89-2b7fb41338e9"). InnerVolumeSpecName "kube-api-access-twgnb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:39:45.250860 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:39:45.250793    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/cc069342-244c-467a-9d89-2b7fb41338e9-istio-podinfo" pod "cc069342-244c-467a-9d89-2b7fb41338e9" (UID: "cc069342-244c-467a-9d89-2b7fb41338e9"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:39:45.250954 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:39:45.250933    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cc069342-244c-467a-9d89-2b7fb41338e9-istio-envoy" pod "cc069342-244c-467a-9d89-2b7fb41338e9" (UID: "cc069342-244c-467a-9d89-2b7fb41338e9"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:39:45.350277 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:39:45.350240    5510 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cc069342-244c-467a-9d89-2b7fb41338e9-istio-envoy\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:39:45.350277 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:39:45.350271    5510 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cc069342-244c-467a-9d89-2b7fb41338e9-istio-podinfo\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:39:45.350489 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:39:45.350285    5510 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cc069342-244c-467a-9d89-2b7fb41338e9-istio-token\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:39:45.350489 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:39:45.350298    5510 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cc069342-244c-467a-9d89-2b7fb41338e9-istiod-ca-cert\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:39:45.350489 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:39:45.350311    5510 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-twgnb\" (UniqueName: \"kubernetes.io/projected/cc069342-244c-467a-9d89-2b7fb41338e9-kube-api-access-twgnb\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:39:46.054965 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:39:46.054931    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-6dft4"
Oct 30 22:39:46.054965 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:39:46.054960    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-6dft4" event={"ID":"cc069342-244c-467a-9d89-2b7fb41338e9","Type":"ContainerDied","Data":"f7aab8caef859c88ad85688eca7c4b008b7040cc4705e2d80067e64973be5e31"}
Oct 30 22:39:46.055523 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:39:46.055022    5510 scope.go:122] "RemoveContainer" containerID="eda1bf976520f473dae4c2d09c2e060f7ddc72c5b16ba293ca1ffbe62b420107"
Oct 30 22:39:46.258191 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:39:46.258149    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-6dft4"]
Oct 30 22:39:46.305686 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:39:46.305571    5510 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-6dft4"]
Oct 30 22:39:47.552460 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:39:47.552358    5510 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="cc069342-244c-467a-9d89-2b7fb41338e9" path="/var/lib/kubelet/pods/cc069342-244c-467a-9d89-2b7fb41338e9/volumes"
Oct 30 22:39:47.866792 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:39:47.866694    5510 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-a6b6a5-7976bff669-4lf96" podUID="19d32fc2-60a2-4040-813c-8b2127047510" containerName="payload-processing" containerID="cri-o://3408840d0822d9773658d8401b20199ad8c7ac7017a38b2af2dc94ef2ec0b9c9" gracePeriod=25
Oct 30 22:39:47.994507 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:39:47.994460    5510 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-a6b6a5-85cc5f8687-hdbfk" podUID="7554b329-1324-4e03-96ac-f8545413fbe3" containerName="payload-pre-processing" containerID="cri-o://e88750cbf071a7b3e3d980b3ecb989642c87dfc4460813b30c3eb1f3cfb7aa7f" gracePeriod=25
Oct 30 22:39:48.037649 ip-10-0-136-210 kubenswrapper[5510]: W1030 22:39:48.037458    5510 logging.go:55] [core] [Channel #6300 SubChannel #6302]grpc: addrConn.createTransport failed to connect to {Addr: "10.133.0.113:50052", ServerName: "10.133.0.113:50052", BalancerAttributes: {"<%!p(pickfirstleaf.managedByPickfirstKeyType={})>": "<%!p(bool=true)>" }}. Err: connection error: desc = "transport: Error while dialing: dial tcp 10.133.0.113:50052: connect: connection refused"
Oct 30 22:39:48.037754 ip-10-0-136-210 kubenswrapper[5510]: W1030 22:39:48.037666    5510 logging.go:55] [core] [Channel #6299 SubChannel #6301]grpc: addrConn.createTransport failed to connect to {Addr: "10.133.0.114:50052", ServerName: "10.133.0.114:50052", BalancerAttributes: {"<%!p(pickfirstleaf.managedByPickfirstKeyType={})>": "<%!p(bool=true)>" }}. Err: connection error: desc = "transport: Error while dialing: dial tcp 10.133.0.114:50052: connect: connection refused"
Oct 30 22:39:48.073081 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:39:48.073049    5510 generic.go:356] "Generic (PLEG): container finished" podID="19d32fc2-60a2-4040-813c-8b2127047510" containerID="3408840d0822d9773658d8401b20199ad8c7ac7017a38b2af2dc94ef2ec0b9c9" exitCode=0
Oct 30 22:39:48.073234 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:39:48.073087    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-a6b6a5-7976bff669-4lf96" event={"ID":"19d32fc2-60a2-4040-813c-8b2127047510","Type":"ContainerDied","Data":"3408840d0822d9773658d8401b20199ad8c7ac7017a38b2af2dc94ef2ec0b9c9"}
Oct 30 22:39:48.123924 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:39:48.123868    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-a6b6a5-7976bff669-4lf96"
Oct 30 22:39:48.240129 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:39:48.240104    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-a6b6a5-85cc5f8687-hdbfk"
Oct 30 22:39:48.281102 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:39:48.281058    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/19d32fc2-60a2-4040-813c-8b2127047510-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/19d32fc2-60a2-4040-813c-8b2127047510-plugins-config-volume\") pod \"19d32fc2-60a2-4040-813c-8b2127047510\" (UID: \"19d32fc2-60a2-4040-813c-8b2127047510\") "
Oct 30 22:39:48.281233 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:39:48.281144    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/19d32fc2-60a2-4040-813c-8b2127047510-kube-api-access-4567t\" (UniqueName: \"kubernetes.io/projected/19d32fc2-60a2-4040-813c-8b2127047510-kube-api-access-4567t\") pod \"19d32fc2-60a2-4040-813c-8b2127047510\" (UID: \"19d32fc2-60a2-4040-813c-8b2127047510\") "
Oct 30 22:39:48.281437 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:39:48.281416    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/19d32fc2-60a2-4040-813c-8b2127047510-plugins-config-volume" pod "19d32fc2-60a2-4040-813c-8b2127047510" (UID: "19d32fc2-60a2-4040-813c-8b2127047510"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:39:48.283212 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:39:48.283191    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/19d32fc2-60a2-4040-813c-8b2127047510-kube-api-access-4567t" pod "19d32fc2-60a2-4040-813c-8b2127047510" (UID: "19d32fc2-60a2-4040-813c-8b2127047510"). InnerVolumeSpecName "kube-api-access-4567t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:39:48.382823 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:39:48.382731    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7554b329-1324-4e03-96ac-f8545413fbe3-kube-api-access-rpwpb\" (UniqueName: \"kubernetes.io/projected/7554b329-1324-4e03-96ac-f8545413fbe3-kube-api-access-rpwpb\") pod \"7554b329-1324-4e03-96ac-f8545413fbe3\" (UID: \"7554b329-1324-4e03-96ac-f8545413fbe3\") "
Oct 30 22:39:48.382823 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:39:48.382811    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7554b329-1324-4e03-96ac-f8545413fbe3-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7554b329-1324-4e03-96ac-f8545413fbe3-plugins-config-volume\") pod \"7554b329-1324-4e03-96ac-f8545413fbe3\" (UID: \"7554b329-1324-4e03-96ac-f8545413fbe3\") "
Oct 30 22:39:48.383594 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:39:48.383547    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7554b329-1324-4e03-96ac-f8545413fbe3-plugins-config-volume" pod "7554b329-1324-4e03-96ac-f8545413fbe3" (UID: "7554b329-1324-4e03-96ac-f8545413fbe3"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:39:48.384818 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:39:48.384070    5510 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/19d32fc2-60a2-4040-813c-8b2127047510-plugins-config-volume\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:39:48.384818 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:39:48.384108    5510 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4567t\" (UniqueName: \"kubernetes.io/projected/19d32fc2-60a2-4040-813c-8b2127047510-kube-api-access-4567t\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:39:48.384818 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:39:48.384124    5510 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7554b329-1324-4e03-96ac-f8545413fbe3-plugins-config-volume\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:39:48.389717 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:39:48.389683    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7554b329-1324-4e03-96ac-f8545413fbe3-kube-api-access-rpwpb" pod "7554b329-1324-4e03-96ac-f8545413fbe3" (UID: "7554b329-1324-4e03-96ac-f8545413fbe3"). InnerVolumeSpecName "kube-api-access-rpwpb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:39:48.484839 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:39:48.484801    5510 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rpwpb\" (UniqueName: \"kubernetes.io/projected/7554b329-1324-4e03-96ac-f8545413fbe3-kube-api-access-rpwpb\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:39:49.037208 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:39:49.037154    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-a6b6a5-7976bff669-4lf96" podUID="19d32fc2-60a2-4040-813c-8b2127047510" containerName="payload-processing" probeResult="failure" output="timeout: failed to connect service \"10.133.0.114:50052\" within 1s: context deadline exceeded"
Oct 30 22:39:49.037602 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:39:49.037165    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-a6b6a5-85cc5f8687-hdbfk" podUID="7554b329-1324-4e03-96ac-f8545413fbe3" containerName="payload-pre-processing" probeResult="failure" output="timeout: failed to connect service \"10.133.0.113:50052\" within 1s: context deadline exceeded"
Oct 30 22:39:49.079942 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:39:49.079903    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-a6b6a5-7976bff669-4lf96" event={"ID":"19d32fc2-60a2-4040-813c-8b2127047510","Type":"ContainerDied","Data":"901dfa620e2980c52fee51e7391af162a90b597a29c66ccd28ad5a7b1a896404"}
Oct 30 22:39:49.080132 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:39:49.079972    5510 scope.go:122] "RemoveContainer" containerID="3408840d0822d9773658d8401b20199ad8c7ac7017a38b2af2dc94ef2ec0b9c9"
Oct 30 22:39:49.080132 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:39:49.079912    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-a6b6a5-7976bff669-4lf96"
Oct 30 22:39:49.081362 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:39:49.081246    5510 generic.go:356] "Generic (PLEG): container finished" podID="7554b329-1324-4e03-96ac-f8545413fbe3" containerID="e88750cbf071a7b3e3d980b3ecb989642c87dfc4460813b30c3eb1f3cfb7aa7f" exitCode=0
Oct 30 22:39:49.081362 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:39:49.081283    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-a6b6a5-85cc5f8687-hdbfk" event={"ID":"7554b329-1324-4e03-96ac-f8545413fbe3","Type":"ContainerDied","Data":"e88750cbf071a7b3e3d980b3ecb989642c87dfc4460813b30c3eb1f3cfb7aa7f"}
Oct 30 22:39:49.081362 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:39:49.081316    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-a6b6a5-85cc5f8687-hdbfk"
Oct 30 22:39:49.081554 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:39:49.081322    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-a6b6a5-85cc5f8687-hdbfk" event={"ID":"7554b329-1324-4e03-96ac-f8545413fbe3","Type":"ContainerDied","Data":"c2c534e4e5e78552824c093cb7971e959f06c8057b99d1da6c56efee427e235f"}
Oct 30 22:39:49.089763 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:39:49.089743    5510 scope.go:122] "RemoveContainer" containerID="e88750cbf071a7b3e3d980b3ecb989642c87dfc4460813b30c3eb1f3cfb7aa7f"
Oct 30 22:39:49.098094 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:39:49.098074    5510 scope.go:122] "RemoveContainer" containerID="e88750cbf071a7b3e3d980b3ecb989642c87dfc4460813b30c3eb1f3cfb7aa7f"
Oct 30 22:39:49.098359 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:39:49.098338    5510 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e88750cbf071a7b3e3d980b3ecb989642c87dfc4460813b30c3eb1f3cfb7aa7f\": container with ID starting with e88750cbf071a7b3e3d980b3ecb989642c87dfc4460813b30c3eb1f3cfb7aa7f not found: ID does not exist" containerID="e88750cbf071a7b3e3d980b3ecb989642c87dfc4460813b30c3eb1f3cfb7aa7f"
Oct 30 22:39:49.098406 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:39:49.098368    5510 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e88750cbf071a7b3e3d980b3ecb989642c87dfc4460813b30c3eb1f3cfb7aa7f"} err="failed to get container status \"e88750cbf071a7b3e3d980b3ecb989642c87dfc4460813b30c3eb1f3cfb7aa7f\": rpc error: code = NotFound desc = could not find container \"e88750cbf071a7b3e3d980b3ecb989642c87dfc4460813b30c3eb1f3cfb7aa7f\": container with ID starting with e88750cbf071a7b3e3d980b3ecb989642c87dfc4460813b30c3eb1f3cfb7aa7f not found: ID does not exist"
Oct 30 22:39:49.238452 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:39:49.238361    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w1-a6b6a5-85cc5f8687-hdbfk"]
Oct 30 22:39:49.242617 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:39:49.242591    5510 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w1-a6b6a5-85cc5f8687-hdbfk"]
Oct 30 22:39:49.346382 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:39:49.346342    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w1-a6b6a5-7976bff669-4lf96"]
Oct 30 22:39:49.351614 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:39:49.351584    5510 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w1-a6b6a5-7976bff669-4lf96"]
Oct 30 22:39:49.553050 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:39:49.553017    5510 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="19d32fc2-60a2-4040-813c-8b2127047510" path="/var/lib/kubelet/pods/19d32fc2-60a2-4040-813c-8b2127047510/volumes"
Oct 30 22:39:49.553401 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:39:49.553389    5510 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7554b329-1324-4e03-96ac-f8545413fbe3" path="/var/lib/kubelet/pods/7554b329-1324-4e03-96ac-f8545413fbe3/volumes"
Oct 30 22:39:50.000295 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:39:50.000216    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:39:54.034457 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:39:54.034413    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-a6b6a5-cdb55bb68-ljgn7"]
Oct 30 22:39:54.034969 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:39:54.034688    5510 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-a6b6a5-cdb55bb68-ljgn7" podUID="195dcf2f-1af0-4404-ac3c-2e3424b9b266" containerName="maas-api" containerID="cri-o://3c7c302174804a1909818ec8b4442d44876684c989a0be28a2419784b2a32b62" gracePeriod=30
Oct 30 22:39:54.192491 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:39:54.192453    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:39:54.283405 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:39:54.283379    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-a6b6a5-cdb55bb68-ljgn7"
Oct 30 22:39:54.383968 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:39:54.383870    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:39:54.443486 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:39:54.443452    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/195dcf2f-1af0-4404-ac3c-2e3424b9b266-metrics-certs\" (UniqueName: \"kubernetes.io/secret/195dcf2f-1af0-4404-ac3c-2e3424b9b266-metrics-certs\") pod \"195dcf2f-1af0-4404-ac3c-2e3424b9b266\" (UID: \"195dcf2f-1af0-4404-ac3c-2e3424b9b266\") "
Oct 30 22:39:54.443697 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:39:54.443537    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/195dcf2f-1af0-4404-ac3c-2e3424b9b266-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/195dcf2f-1af0-4404-ac3c-2e3424b9b266-maas-api-tls\") pod \"195dcf2f-1af0-4404-ac3c-2e3424b9b266\" (UID: \"195dcf2f-1af0-4404-ac3c-2e3424b9b266\") "
Oct 30 22:39:54.443766 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:39:54.443727    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/195dcf2f-1af0-4404-ac3c-2e3424b9b266-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/195dcf2f-1af0-4404-ac3c-2e3424b9b266-openshift-service-ca\") pod \"195dcf2f-1af0-4404-ac3c-2e3424b9b266\" (UID: \"195dcf2f-1af0-4404-ac3c-2e3424b9b266\") "
Oct 30 22:39:54.443821 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:39:54.443770    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/195dcf2f-1af0-4404-ac3c-2e3424b9b266-kube-api-access-zc7z8\" (UniqueName: \"kubernetes.io/projected/195dcf2f-1af0-4404-ac3c-2e3424b9b266-kube-api-access-zc7z8\") pod \"195dcf2f-1af0-4404-ac3c-2e3424b9b266\" (UID: \"195dcf2f-1af0-4404-ac3c-2e3424b9b266\") "
Oct 30 22:39:54.444212 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:39:54.444143    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/195dcf2f-1af0-4404-ac3c-2e3424b9b266-openshift-service-ca" pod "195dcf2f-1af0-4404-ac3c-2e3424b9b266" (UID: "195dcf2f-1af0-4404-ac3c-2e3424b9b266"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:39:54.445850 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:39:54.445815    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/195dcf2f-1af0-4404-ac3c-2e3424b9b266-kube-api-access-zc7z8" pod "195dcf2f-1af0-4404-ac3c-2e3424b9b266" (UID: "195dcf2f-1af0-4404-ac3c-2e3424b9b266"). InnerVolumeSpecName "kube-api-access-zc7z8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:39:54.446126 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:39:54.446103    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/195dcf2f-1af0-4404-ac3c-2e3424b9b266-metrics-certs" pod "195dcf2f-1af0-4404-ac3c-2e3424b9b266" (UID: "195dcf2f-1af0-4404-ac3c-2e3424b9b266"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:39:54.446310 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:39:54.446293    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/195dcf2f-1af0-4404-ac3c-2e3424b9b266-maas-api-tls" pod "195dcf2f-1af0-4404-ac3c-2e3424b9b266" (UID: "195dcf2f-1af0-4404-ac3c-2e3424b9b266"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:39:54.545413 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:39:54.545373    5510 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/195dcf2f-1af0-4404-ac3c-2e3424b9b266-openshift-service-ca\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:39:54.545413 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:39:54.545406    5510 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zc7z8\" (UniqueName: \"kubernetes.io/projected/195dcf2f-1af0-4404-ac3c-2e3424b9b266-kube-api-access-zc7z8\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:39:54.545413 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:39:54.545417    5510 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/195dcf2f-1af0-4404-ac3c-2e3424b9b266-metrics-certs\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:39:54.545696 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:39:54.545427    5510 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/195dcf2f-1af0-4404-ac3c-2e3424b9b266-maas-api-tls\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:39:55.119968 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:39:55.119930    5510 generic.go:356] "Generic (PLEG): container finished" podID="195dcf2f-1af0-4404-ac3c-2e3424b9b266" containerID="3c7c302174804a1909818ec8b4442d44876684c989a0be28a2419784b2a32b62" exitCode=0
Oct 30 22:39:55.120414 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:39:55.119982    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-a6b6a5-cdb55bb68-ljgn7" event={"ID":"195dcf2f-1af0-4404-ac3c-2e3424b9b266","Type":"ContainerDied","Data":"3c7c302174804a1909818ec8b4442d44876684c989a0be28a2419784b2a32b62"}
Oct 30 22:39:55.120414 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:39:55.120023    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-a6b6a5-cdb55bb68-ljgn7" event={"ID":"195dcf2f-1af0-4404-ac3c-2e3424b9b266","Type":"ContainerDied","Data":"5d7606da0cf2835e5c53c6481dac273aac45d156b0deb23e911b3ce3600dbc5b"}
Oct 30 22:39:55.120414 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:39:55.120046    5510 scope.go:122] "RemoveContainer" containerID="3c7c302174804a1909818ec8b4442d44876684c989a0be28a2419784b2a32b62"
Oct 30 22:39:55.120414 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:39:55.119987    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-a6b6a5-cdb55bb68-ljgn7"
Oct 30 22:39:55.129761 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:39:55.129737    5510 scope.go:122] "RemoveContainer" containerID="3c7c302174804a1909818ec8b4442d44876684c989a0be28a2419784b2a32b62"
Oct 30 22:39:55.130009 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:39:55.129989    5510 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3c7c302174804a1909818ec8b4442d44876684c989a0be28a2419784b2a32b62\": container with ID starting with 3c7c302174804a1909818ec8b4442d44876684c989a0be28a2419784b2a32b62 not found: ID does not exist" containerID="3c7c302174804a1909818ec8b4442d44876684c989a0be28a2419784b2a32b62"
Oct 30 22:39:55.130078 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:39:55.130022    5510 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3c7c302174804a1909818ec8b4442d44876684c989a0be28a2419784b2a32b62"} err="failed to get container status \"3c7c302174804a1909818ec8b4442d44876684c989a0be28a2419784b2a32b62\": rpc error: code = NotFound desc = could not find container \"3c7c302174804a1909818ec8b4442d44876684c989a0be28a2419784b2a32b62\": container with ID starting with 3c7c302174804a1909818ec8b4442d44876684c989a0be28a2419784b2a32b62 not found: ID does not exist"
Oct 30 22:39:55.275493 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:39:55.275460    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-a6b6a5-cdb55bb68-ljgn7"]
Oct 30 22:39:55.321957 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:39:55.321927    5510 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-a6b6a5-cdb55bb68-ljgn7"]
Oct 30 22:39:55.552201 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:39:55.552170    5510 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="195dcf2f-1af0-4404-ac3c-2e3424b9b266" path="/var/lib/kubelet/pods/195dcf2f-1af0-4404-ac3c-2e3424b9b266/volumes"
Oct 30 22:39:55.623670 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:39:55.623635    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:39:55.892612 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:39:55.892515    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:39:56.088384 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:39:56.088351    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:39:56.532095 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:39:56.532061    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:40:00.274251 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:00.274173    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:40:01.718102 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:01.718057    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:40:02.018948 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:02.018899    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:40:02.222167 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:02.222127    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:40:02.628563 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:02.628526    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:40:03.668959 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:03.668922    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:40:03.909959 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:03.909922    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:40:03.980546 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:03.980450    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w1-a6b6a5-openshift-default-f7cd49795-zlkqg"]
Oct 30 22:40:03.980751 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:03.980728    5510 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w1-a6b6a5-openshift-default-f7cd49795-zlkqg" podUID="61fdf6bd-8922-4db6-a4cf-4fa9ea967b20" containerName="istio-proxy" containerID="cri-o://33a8fb2c026ec7712d6106b877ef39350263d933eb1f22bb8130b0091637edc1" gracePeriod=30
Oct 30 22:40:04.107337 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:04.107297    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:40:06.970325 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:06.970272    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:40:08.419417 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:40:08.419378    5510 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6gjbf" containerName="extractor"
Oct 30 22:40:09.195410 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:09.195374    5510 generic.go:356] "Generic (PLEG): container finished" podID="61fdf6bd-8922-4db6-a4cf-4fa9ea967b20" containerID="33a8fb2c026ec7712d6106b877ef39350263d933eb1f22bb8130b0091637edc1" exitCode=0
Oct 30 22:40:09.195556 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:09.195451    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w1-a6b6a5-openshift-default-f7cd49795-zlkqg" event={"ID":"61fdf6bd-8922-4db6-a4cf-4fa9ea967b20","Type":"ContainerDied","Data":"33a8fb2c026ec7712d6106b877ef39350263d933eb1f22bb8130b0091637edc1"}
Oct 30 22:40:09.195556 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:09.195510    5510 scope.go:122] "RemoveContainer" containerID="244da90203c70fbbaf03359fbaf0ae00b6517b196db93176f7390a74b61d45e6"
Oct 30 22:40:09.236287 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:09.236259    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w1-a6b6a5-openshift-default-f7cd49795-zlkqg"
Oct 30 22:40:09.295969 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:09.295937    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/61fdf6bd-8922-4db6-a4cf-4fa9ea967b20-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/61fdf6bd-8922-4db6-a4cf-4fa9ea967b20-credential-socket\") pod \"61fdf6bd-8922-4db6-a4cf-4fa9ea967b20\" (UID: \"61fdf6bd-8922-4db6-a4cf-4fa9ea967b20\") "
Oct 30 22:40:09.296150 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:09.295987    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/61fdf6bd-8922-4db6-a4cf-4fa9ea967b20-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/61fdf6bd-8922-4db6-a4cf-4fa9ea967b20-istio-data\") pod \"61fdf6bd-8922-4db6-a4cf-4fa9ea967b20\" (UID: \"61fdf6bd-8922-4db6-a4cf-4fa9ea967b20\") "
Oct 30 22:40:09.296150 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:09.296061    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/61fdf6bd-8922-4db6-a4cf-4fa9ea967b20-kube-api-access-6d2xc\" (UniqueName: \"kubernetes.io/projected/61fdf6bd-8922-4db6-a4cf-4fa9ea967b20-kube-api-access-6d2xc\") pod \"61fdf6bd-8922-4db6-a4cf-4fa9ea967b20\" (UID: \"61fdf6bd-8922-4db6-a4cf-4fa9ea967b20\") "
Oct 30 22:40:09.296150 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:09.296124    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/61fdf6bd-8922-4db6-a4cf-4fa9ea967b20-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/61fdf6bd-8922-4db6-a4cf-4fa9ea967b20-workload-certs\") pod \"61fdf6bd-8922-4db6-a4cf-4fa9ea967b20\" (UID: \"61fdf6bd-8922-4db6-a4cf-4fa9ea967b20\") "
Oct 30 22:40:09.296331 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:09.296152    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/61fdf6bd-8922-4db6-a4cf-4fa9ea967b20-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/61fdf6bd-8922-4db6-a4cf-4fa9ea967b20-workload-socket\") pod \"61fdf6bd-8922-4db6-a4cf-4fa9ea967b20\" (UID: \"61fdf6bd-8922-4db6-a4cf-4fa9ea967b20\") "
Oct 30 22:40:09.296331 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:09.296183    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/61fdf6bd-8922-4db6-a4cf-4fa9ea967b20-istio-token\" (UniqueName: \"kubernetes.io/projected/61fdf6bd-8922-4db6-a4cf-4fa9ea967b20-istio-token\") pod \"61fdf6bd-8922-4db6-a4cf-4fa9ea967b20\" (UID: \"61fdf6bd-8922-4db6-a4cf-4fa9ea967b20\") "
Oct 30 22:40:09.296331 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:09.296207    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/61fdf6bd-8922-4db6-a4cf-4fa9ea967b20-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/61fdf6bd-8922-4db6-a4cf-4fa9ea967b20-istiod-ca-cert\") pod \"61fdf6bd-8922-4db6-a4cf-4fa9ea967b20\" (UID: \"61fdf6bd-8922-4db6-a4cf-4fa9ea967b20\") "
Oct 30 22:40:09.296331 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:09.296235    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/61fdf6bd-8922-4db6-a4cf-4fa9ea967b20-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/61fdf6bd-8922-4db6-a4cf-4fa9ea967b20-istio-envoy\") pod \"61fdf6bd-8922-4db6-a4cf-4fa9ea967b20\" (UID: \"61fdf6bd-8922-4db6-a4cf-4fa9ea967b20\") "
Oct 30 22:40:09.296331 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:09.296255    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/61fdf6bd-8922-4db6-a4cf-4fa9ea967b20-istio-data" pod "61fdf6bd-8922-4db6-a4cf-4fa9ea967b20" (UID: "61fdf6bd-8922-4db6-a4cf-4fa9ea967b20"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:40:09.296331 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:09.296266    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/61fdf6bd-8922-4db6-a4cf-4fa9ea967b20-credential-socket" pod "61fdf6bd-8922-4db6-a4cf-4fa9ea967b20" (UID: "61fdf6bd-8922-4db6-a4cf-4fa9ea967b20"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:40:09.296331 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:09.296285    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/61fdf6bd-8922-4db6-a4cf-4fa9ea967b20-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/61fdf6bd-8922-4db6-a4cf-4fa9ea967b20-istio-podinfo\") pod \"61fdf6bd-8922-4db6-a4cf-4fa9ea967b20\" (UID: \"61fdf6bd-8922-4db6-a4cf-4fa9ea967b20\") "
Oct 30 22:40:09.296700 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:09.296436    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/61fdf6bd-8922-4db6-a4cf-4fa9ea967b20-workload-socket" pod "61fdf6bd-8922-4db6-a4cf-4fa9ea967b20" (UID: "61fdf6bd-8922-4db6-a4cf-4fa9ea967b20"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:40:09.296700 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:09.296456    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/61fdf6bd-8922-4db6-a4cf-4fa9ea967b20-workload-certs" pod "61fdf6bd-8922-4db6-a4cf-4fa9ea967b20" (UID: "61fdf6bd-8922-4db6-a4cf-4fa9ea967b20"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:40:09.296700 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:09.296691    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/61fdf6bd-8922-4db6-a4cf-4fa9ea967b20-istiod-ca-cert" pod "61fdf6bd-8922-4db6-a4cf-4fa9ea967b20" (UID: "61fdf6bd-8922-4db6-a4cf-4fa9ea967b20"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:40:09.296876 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:09.296800    5510 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/61fdf6bd-8922-4db6-a4cf-4fa9ea967b20-workload-certs\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:40:09.296876 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:09.296820    5510 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/61fdf6bd-8922-4db6-a4cf-4fa9ea967b20-workload-socket\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:40:09.296876 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:09.296835    5510 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/61fdf6bd-8922-4db6-a4cf-4fa9ea967b20-istiod-ca-cert\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:40:09.296876 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:09.296849    5510 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/61fdf6bd-8922-4db6-a4cf-4fa9ea967b20-credential-socket\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:40:09.296876 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:09.296864    5510 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/61fdf6bd-8922-4db6-a4cf-4fa9ea967b20-istio-data\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:40:09.298476 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:09.298403    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/61fdf6bd-8922-4db6-a4cf-4fa9ea967b20-istio-podinfo" pod "61fdf6bd-8922-4db6-a4cf-4fa9ea967b20" (UID: "61fdf6bd-8922-4db6-a4cf-4fa9ea967b20"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:40:09.298614 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:09.298518    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/61fdf6bd-8922-4db6-a4cf-4fa9ea967b20-istio-envoy" pod "61fdf6bd-8922-4db6-a4cf-4fa9ea967b20" (UID: "61fdf6bd-8922-4db6-a4cf-4fa9ea967b20"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:40:09.298614 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:09.298476    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/61fdf6bd-8922-4db6-a4cf-4fa9ea967b20-istio-token" pod "61fdf6bd-8922-4db6-a4cf-4fa9ea967b20" (UID: "61fdf6bd-8922-4db6-a4cf-4fa9ea967b20"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:40:09.298614 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:09.298520    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/61fdf6bd-8922-4db6-a4cf-4fa9ea967b20-kube-api-access-6d2xc" pod "61fdf6bd-8922-4db6-a4cf-4fa9ea967b20" (UID: "61fdf6bd-8922-4db6-a4cf-4fa9ea967b20"). InnerVolumeSpecName "kube-api-access-6d2xc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:40:09.351010 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:09.350973    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:40:09.398163 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:09.398130    5510 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/61fdf6bd-8922-4db6-a4cf-4fa9ea967b20-istio-token\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:40:09.398163 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:09.398156    5510 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/61fdf6bd-8922-4db6-a4cf-4fa9ea967b20-istio-envoy\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:40:09.398163 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:09.398168    5510 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/61fdf6bd-8922-4db6-a4cf-4fa9ea967b20-istio-podinfo\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:40:09.398400 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:09.398179    5510 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6d2xc\" (UniqueName: \"kubernetes.io/projected/61fdf6bd-8922-4db6-a4cf-4fa9ea967b20-kube-api-access-6d2xc\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:40:09.604154 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:09.604122    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:40:09.848353 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:09.848257    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:40:10.038785 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:10.038740    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:40:10.201929 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:10.201835    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w1-a6b6a5-openshift-default-f7cd49795-zlkqg" event={"ID":"61fdf6bd-8922-4db6-a4cf-4fa9ea967b20","Type":"ContainerDied","Data":"618e747c0d0ca490e2801e15d62e5547e180323deedafed4276ca20cfba1046f"}
Oct 30 22:40:10.201929 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:10.201883    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w1-a6b6a5-openshift-default-f7cd49795-zlkqg"
Oct 30 22:40:10.202137 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:10.201940    5510 scope.go:122] "RemoveContainer" containerID="33a8fb2c026ec7712d6106b877ef39350263d933eb1f22bb8130b0091637edc1"
Oct 30 22:40:10.239006 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:10.238970    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:40:10.411283 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:10.411251    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w1-a6b6a5-openshift-default-f7cd49795-zlkqg"]
Oct 30 22:40:10.457995 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:10.457912    5510 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w1-a6b6a5-openshift-default-f7cd49795-zlkqg"]
Oct 30 22:40:11.552180 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:11.552142    5510 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="61fdf6bd-8922-4db6-a4cf-4fa9ea967b20" path="/var/lib/kubelet/pods/61fdf6bd-8922-4db6-a4cf-4fa9ea967b20/volumes"
Oct 30 22:40:12.044430 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:12.044399    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:40:15.101862 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:15.101820    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w6-fb0b93/distinct-w6-fb0b93-kserve-58db6b6757-6mww8"]
Oct 30 22:40:15.148410 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:15.148379    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w6-fb0b93/premium-w6-fb0b93-kserve-5cb9948c54-w7pnk"]
Oct 30 22:40:15.149768 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:15.149743    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w1-943d8f-7556597485-mznq2"]
Oct 30 22:40:15.247893 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:15.247604    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w1-943d8f-7b5555d7f9-v5c6w"]
Oct 30 22:40:15.252087 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:15.252047    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:40:15.613502 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:15.613442    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:40:15.810033 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:15.809984    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:40:16.425450 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:16.425415    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:40:18.288699 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:18.288641    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:40:19.741915 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:19.741877    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:40:20.114034 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:20.114001    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:40:20.150835 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:20.150780    5510 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-943d8f-7556597485-mznq2" podUID="1cd4d179-c02d-439e-afea-7dbf8b89d3ea" containerName="payload-processing" containerID="cri-o://38ab9efb9db90ac7407b447c51b01274bbe9aed79c4195cec5c448ad4024e61f" gracePeriod=25
Oct 30 22:40:20.248871 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:20.248826    5510 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-943d8f-7b5555d7f9-v5c6w" podUID="d303451d-0d81-4a9b-8bc6-91f019cfc4a0" containerName="payload-pre-processing" containerID="cri-o://d26b719290619fc74c172ef711ab53a39c5792ed79d4a6920aca02c769635075" gracePeriod=25
Oct 30 22:40:20.413699 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:20.413670    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-943d8f-7556597485-mznq2"
Oct 30 22:40:20.512468 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:20.512441    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-943d8f-7b5555d7f9-v5c6w"
Oct 30 22:40:20.514520 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:20.514492    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1cd4d179-c02d-439e-afea-7dbf8b89d3ea-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1cd4d179-c02d-439e-afea-7dbf8b89d3ea-plugins-config-volume\") pod \"1cd4d179-c02d-439e-afea-7dbf8b89d3ea\" (UID: \"1cd4d179-c02d-439e-afea-7dbf8b89d3ea\") "
Oct 30 22:40:20.514692 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:20.514673    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1cd4d179-c02d-439e-afea-7dbf8b89d3ea-kube-api-access-jz6p4\" (UniqueName: \"kubernetes.io/projected/1cd4d179-c02d-439e-afea-7dbf8b89d3ea-kube-api-access-jz6p4\") pod \"1cd4d179-c02d-439e-afea-7dbf8b89d3ea\" (UID: \"1cd4d179-c02d-439e-afea-7dbf8b89d3ea\") "
Oct 30 22:40:20.514963 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:20.514942    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1cd4d179-c02d-439e-afea-7dbf8b89d3ea-plugins-config-volume" pod "1cd4d179-c02d-439e-afea-7dbf8b89d3ea" (UID: "1cd4d179-c02d-439e-afea-7dbf8b89d3ea"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:40:20.516683 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:20.516639    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1cd4d179-c02d-439e-afea-7dbf8b89d3ea-kube-api-access-jz6p4" pod "1cd4d179-c02d-439e-afea-7dbf8b89d3ea" (UID: "1cd4d179-c02d-439e-afea-7dbf8b89d3ea"). InnerVolumeSpecName "kube-api-access-jz6p4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:40:20.615718 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:20.615681    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d303451d-0d81-4a9b-8bc6-91f019cfc4a0-kube-api-access-njt4t\" (UniqueName: \"kubernetes.io/projected/d303451d-0d81-4a9b-8bc6-91f019cfc4a0-kube-api-access-njt4t\") pod \"d303451d-0d81-4a9b-8bc6-91f019cfc4a0\" (UID: \"d303451d-0d81-4a9b-8bc6-91f019cfc4a0\") "
Oct 30 22:40:20.615902 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:20.615843    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d303451d-0d81-4a9b-8bc6-91f019cfc4a0-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d303451d-0d81-4a9b-8bc6-91f019cfc4a0-plugins-config-volume\") pod \"d303451d-0d81-4a9b-8bc6-91f019cfc4a0\" (UID: \"d303451d-0d81-4a9b-8bc6-91f019cfc4a0\") "
Oct 30 22:40:20.616198 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:20.616176    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d303451d-0d81-4a9b-8bc6-91f019cfc4a0-plugins-config-volume" pod "d303451d-0d81-4a9b-8bc6-91f019cfc4a0" (UID: "d303451d-0d81-4a9b-8bc6-91f019cfc4a0"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:40:20.616246 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:20.616231    5510 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jz6p4\" (UniqueName: \"kubernetes.io/projected/1cd4d179-c02d-439e-afea-7dbf8b89d3ea-kube-api-access-jz6p4\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:40:20.616289 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:20.616247    5510 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1cd4d179-c02d-439e-afea-7dbf8b89d3ea-plugins-config-volume\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:40:20.617880 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:20.617863    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d303451d-0d81-4a9b-8bc6-91f019cfc4a0-kube-api-access-njt4t" pod "d303451d-0d81-4a9b-8bc6-91f019cfc4a0" (UID: "d303451d-0d81-4a9b-8bc6-91f019cfc4a0"). InnerVolumeSpecName "kube-api-access-njt4t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:40:20.664706 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:20.664583    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:40:20.717500 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:20.717455    5510 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d303451d-0d81-4a9b-8bc6-91f019cfc4a0-plugins-config-volume\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:40:20.717500 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:20.717489    5510 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-njt4t\" (UniqueName: \"kubernetes.io/projected/d303451d-0d81-4a9b-8bc6-91f019cfc4a0-kube-api-access-njt4t\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:40:21.263300 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:21.263266    5510 generic.go:356] "Generic (PLEG): container finished" podID="d303451d-0d81-4a9b-8bc6-91f019cfc4a0" containerID="d26b719290619fc74c172ef711ab53a39c5792ed79d4a6920aca02c769635075" exitCode=0
Oct 30 22:40:21.263748 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:21.263311    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-943d8f-7b5555d7f9-v5c6w"
Oct 30 22:40:21.263748 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:21.263354    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-943d8f-7b5555d7f9-v5c6w" event={"ID":"d303451d-0d81-4a9b-8bc6-91f019cfc4a0","Type":"ContainerDied","Data":"d26b719290619fc74c172ef711ab53a39c5792ed79d4a6920aca02c769635075"}
Oct 30 22:40:21.263748 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:21.263410    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-943d8f-7b5555d7f9-v5c6w" event={"ID":"d303451d-0d81-4a9b-8bc6-91f019cfc4a0","Type":"ContainerDied","Data":"b4cb23f118ad5b95c16cc5659aac0b73847635679d9aa578c6369ac341572115"}
Oct 30 22:40:21.263748 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:21.263436    5510 scope.go:122] "RemoveContainer" containerID="d26b719290619fc74c172ef711ab53a39c5792ed79d4a6920aca02c769635075"
Oct 30 22:40:21.264999 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:21.264976    5510 generic.go:356] "Generic (PLEG): container finished" podID="1cd4d179-c02d-439e-afea-7dbf8b89d3ea" containerID="38ab9efb9db90ac7407b447c51b01274bbe9aed79c4195cec5c448ad4024e61f" exitCode=0
Oct 30 22:40:21.265094 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:21.265013    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-943d8f-7556597485-mznq2" event={"ID":"1cd4d179-c02d-439e-afea-7dbf8b89d3ea","Type":"ContainerDied","Data":"38ab9efb9db90ac7407b447c51b01274bbe9aed79c4195cec5c448ad4024e61f"}
Oct 30 22:40:21.265094 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:21.265025    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-943d8f-7556597485-mznq2"
Oct 30 22:40:21.265094 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:21.265043    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-943d8f-7556597485-mznq2" event={"ID":"1cd4d179-c02d-439e-afea-7dbf8b89d3ea","Type":"ContainerDied","Data":"cfdcd4a6eaab7985da997e65bfbd31a03ca0d8d8198e46f4bcddc9c9b213d00e"}
Oct 30 22:40:21.275063 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:21.275043    5510 scope.go:122] "RemoveContainer" containerID="d26b719290619fc74c172ef711ab53a39c5792ed79d4a6920aca02c769635075"
Oct 30 22:40:21.275379 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:40:21.275356    5510 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d26b719290619fc74c172ef711ab53a39c5792ed79d4a6920aca02c769635075\": container with ID starting with d26b719290619fc74c172ef711ab53a39c5792ed79d4a6920aca02c769635075 not found: ID does not exist" containerID="d26b719290619fc74c172ef711ab53a39c5792ed79d4a6920aca02c769635075"
Oct 30 22:40:21.275476 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:21.275390    5510 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d26b719290619fc74c172ef711ab53a39c5792ed79d4a6920aca02c769635075"} err="failed to get container status \"d26b719290619fc74c172ef711ab53a39c5792ed79d4a6920aca02c769635075\": rpc error: code = NotFound desc = could not find container \"d26b719290619fc74c172ef711ab53a39c5792ed79d4a6920aca02c769635075\": container with ID starting with d26b719290619fc74c172ef711ab53a39c5792ed79d4a6920aca02c769635075 not found: ID does not exist"
Oct 30 22:40:21.275476 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:21.275415    5510 scope.go:122] "RemoveContainer" containerID="38ab9efb9db90ac7407b447c51b01274bbe9aed79c4195cec5c448ad4024e61f"
Oct 30 22:40:21.283785 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:21.283765    5510 scope.go:122] "RemoveContainer" containerID="38ab9efb9db90ac7407b447c51b01274bbe9aed79c4195cec5c448ad4024e61f"
Oct 30 22:40:21.284067 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:40:21.284045    5510 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"38ab9efb9db90ac7407b447c51b01274bbe9aed79c4195cec5c448ad4024e61f\": container with ID starting with 38ab9efb9db90ac7407b447c51b01274bbe9aed79c4195cec5c448ad4024e61f not found: ID does not exist" containerID="38ab9efb9db90ac7407b447c51b01274bbe9aed79c4195cec5c448ad4024e61f"
Oct 30 22:40:21.284135 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:21.284068    5510 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"38ab9efb9db90ac7407b447c51b01274bbe9aed79c4195cec5c448ad4024e61f"} err="failed to get container status \"38ab9efb9db90ac7407b447c51b01274bbe9aed79c4195cec5c448ad4024e61f\": rpc error: code = NotFound desc = could not find container \"38ab9efb9db90ac7407b447c51b01274bbe9aed79c4195cec5c448ad4024e61f\": container with ID starting with 38ab9efb9db90ac7407b447c51b01274bbe9aed79c4195cec5c448ad4024e61f not found: ID does not exist"
Oct 30 22:40:21.419458 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:21.419371    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w1-943d8f-7556597485-mznq2"]
Oct 30 22:40:21.450296 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:21.450266    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:40:21.465844 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:21.465814    5510 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w1-943d8f-7556597485-mznq2"]
Oct 30 22:40:21.530078 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:21.530041    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w1-943d8f-7b5555d7f9-v5c6w"]
Oct 30 22:40:21.535644 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:21.535614    5510 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w1-943d8f-7b5555d7f9-v5c6w"]
Oct 30 22:40:21.552188 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:21.552145    5510 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1cd4d179-c02d-439e-afea-7dbf8b89d3ea" path="/var/lib/kubelet/pods/1cd4d179-c02d-439e-afea-7dbf8b89d3ea/volumes"
Oct 30 22:40:21.552642 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:21.552621    5510 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d303451d-0d81-4a9b-8bc6-91f019cfc4a0" path="/var/lib/kubelet/pods/d303451d-0d81-4a9b-8bc6-91f019cfc4a0/volumes"
Oct 30 22:40:21.698333 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:21.698237    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:40:23.124125 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:23.124081    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:40:26.935115 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:26.935073    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:40:27.150969 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:27.150929    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:40:27.226629 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:27.226552    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w1-943d8f-openshift-default-6c57fb7747-z6x89"]
Oct 30 22:40:27.226833 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:27.226810    5510 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w1-943d8f-openshift-default-6c57fb7747-z6x89" podUID="dcbe788e-cc5c-4807-af27-b98a507ea810" containerName="istio-proxy" containerID="cri-o://eb7566c34b76578a2c3664993b4ec0c90a0b953ce9f17a85614c79f8ee92657b" gracePeriod=30
Oct 30 22:40:27.352003 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:27.351966    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:40:27.684189 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:27.684150    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:40:27.878324 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:27.878291    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:40:30.122156 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:30.122120    5510 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w6-fb0b93/distinct-w6-fb0b93-kserve-58db6b6757-6mww8" podUID="8ee6579b-acea-4492-862b-9883f70d00c9" containerName="main" containerID="cri-o://097741694ce77df0e84010eb664b49a7ca93e0eac3e0a699133ff9d5af9e104d" gracePeriod=45
Oct 30 22:40:30.168140 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:30.168079    5510 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w6-fb0b93/premium-w6-fb0b93-kserve-5cb9948c54-w7pnk" podUID="290ae7a2-de37-4af3-82ae-a8e855b83bb9" containerName="main" containerID="cri-o://9f802ceb1f913781c756a6ea49cc3aef5e94b3a6b6e72248b31cc0a33d3e9691" gracePeriod=45
Oct 30 22:40:30.319130 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:30.319092    5510 generic.go:356] "Generic (PLEG): container finished" podID="290ae7a2-de37-4af3-82ae-a8e855b83bb9" containerID="9f802ceb1f913781c756a6ea49cc3aef5e94b3a6b6e72248b31cc0a33d3e9691" exitCode=0
Oct 30 22:40:30.319321 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:30.319162    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-fb0b93/premium-w6-fb0b93-kserve-5cb9948c54-w7pnk" event={"ID":"290ae7a2-de37-4af3-82ae-a8e855b83bb9","Type":"ContainerDied","Data":"9f802ceb1f913781c756a6ea49cc3aef5e94b3a6b6e72248b31cc0a33d3e9691"}
Oct 30 22:40:30.320798 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:30.320774    5510 generic.go:356] "Generic (PLEG): container finished" podID="8ee6579b-acea-4492-862b-9883f70d00c9" containerID="097741694ce77df0e84010eb664b49a7ca93e0eac3e0a699133ff9d5af9e104d" exitCode=0
Oct 30 22:40:30.320913 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:30.320851    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-fb0b93/distinct-w6-fb0b93-kserve-58db6b6757-6mww8" event={"ID":"8ee6579b-acea-4492-862b-9883f70d00c9","Type":"ContainerDied","Data":"097741694ce77df0e84010eb664b49a7ca93e0eac3e0a699133ff9d5af9e104d"}
Oct 30 22:40:30.413944 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:30.413920    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-fb0b93/distinct-w6-fb0b93-kserve-58db6b6757-6mww8"
Oct 30 22:40:30.440607 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:30.440582    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-fb0b93/premium-w6-fb0b93-kserve-5cb9948c54-w7pnk"
Oct 30 22:40:30.518825 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:30.518786    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/290ae7a2-de37-4af3-82ae-a8e855b83bb9-home\" (UniqueName: \"kubernetes.io/empty-dir/290ae7a2-de37-4af3-82ae-a8e855b83bb9-home\") pod \"290ae7a2-de37-4af3-82ae-a8e855b83bb9\" (UID: \"290ae7a2-de37-4af3-82ae-a8e855b83bb9\") "
Oct 30 22:40:30.519013 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:30.518832    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/290ae7a2-de37-4af3-82ae-a8e855b83bb9-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/290ae7a2-de37-4af3-82ae-a8e855b83bb9-tmp-dir\") pod \"290ae7a2-de37-4af3-82ae-a8e855b83bb9\" (UID: \"290ae7a2-de37-4af3-82ae-a8e855b83bb9\") "
Oct 30 22:40:30.519013 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:30.518862    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8ee6579b-acea-4492-862b-9883f70d00c9-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8ee6579b-acea-4492-862b-9883f70d00c9-tmp-dir\") pod \"8ee6579b-acea-4492-862b-9883f70d00c9\" (UID: \"8ee6579b-acea-4492-862b-9883f70d00c9\") "
Oct 30 22:40:30.519013 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:30.518898    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/290ae7a2-de37-4af3-82ae-a8e855b83bb9-tls-certs\" (UniqueName: \"kubernetes.io/secret/290ae7a2-de37-4af3-82ae-a8e855b83bb9-tls-certs\") pod \"290ae7a2-de37-4af3-82ae-a8e855b83bb9\" (UID: \"290ae7a2-de37-4af3-82ae-a8e855b83bb9\") "
Oct 30 22:40:30.519013 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:30.518925    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/290ae7a2-de37-4af3-82ae-a8e855b83bb9-kube-api-access-jlzzj\" (UniqueName: \"kubernetes.io/projected/290ae7a2-de37-4af3-82ae-a8e855b83bb9-kube-api-access-jlzzj\") pod \"290ae7a2-de37-4af3-82ae-a8e855b83bb9\" (UID: \"290ae7a2-de37-4af3-82ae-a8e855b83bb9\") "
Oct 30 22:40:30.519013 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:30.518970    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8ee6579b-acea-4492-862b-9883f70d00c9-dshm\" (UniqueName: \"kubernetes.io/empty-dir/8ee6579b-acea-4492-862b-9883f70d00c9-dshm\") pod \"8ee6579b-acea-4492-862b-9883f70d00c9\" (UID: \"8ee6579b-acea-4492-862b-9883f70d00c9\") "
Oct 30 22:40:30.519292 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:30.519015    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/290ae7a2-de37-4af3-82ae-a8e855b83bb9-dshm\" (UniqueName: \"kubernetes.io/empty-dir/290ae7a2-de37-4af3-82ae-a8e855b83bb9-dshm\") pod \"290ae7a2-de37-4af3-82ae-a8e855b83bb9\" (UID: \"290ae7a2-de37-4af3-82ae-a8e855b83bb9\") "
Oct 30 22:40:30.519292 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:30.519121    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/290ae7a2-de37-4af3-82ae-a8e855b83bb9-home" pod "290ae7a2-de37-4af3-82ae-a8e855b83bb9" (UID: "290ae7a2-de37-4af3-82ae-a8e855b83bb9"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:40:30.519292 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:30.519130    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8ee6579b-acea-4492-862b-9883f70d00c9-home\" (UniqueName: \"kubernetes.io/empty-dir/8ee6579b-acea-4492-862b-9883f70d00c9-home\") pod \"8ee6579b-acea-4492-862b-9883f70d00c9\" (UID: \"8ee6579b-acea-4492-862b-9883f70d00c9\") "
Oct 30 22:40:30.519292 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:30.519154    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8ee6579b-acea-4492-862b-9883f70d00c9-tmp-dir" pod "8ee6579b-acea-4492-862b-9883f70d00c9" (UID: "8ee6579b-acea-4492-862b-9883f70d00c9"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:40:30.519292 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:30.519152    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/290ae7a2-de37-4af3-82ae-a8e855b83bb9-tmp-dir" pod "290ae7a2-de37-4af3-82ae-a8e855b83bb9" (UID: "290ae7a2-de37-4af3-82ae-a8e855b83bb9"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:40:30.519292 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:30.519201    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8ee6579b-acea-4492-862b-9883f70d00c9-tls-certs\" (UniqueName: \"kubernetes.io/secret/8ee6579b-acea-4492-862b-9883f70d00c9-tls-certs\") pod \"8ee6579b-acea-4492-862b-9883f70d00c9\" (UID: \"8ee6579b-acea-4492-862b-9883f70d00c9\") "
Oct 30 22:40:30.519292 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:30.519234    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8ee6579b-acea-4492-862b-9883f70d00c9-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8ee6579b-acea-4492-862b-9883f70d00c9-model-cache\") pod \"8ee6579b-acea-4492-862b-9883f70d00c9\" (UID: \"8ee6579b-acea-4492-862b-9883f70d00c9\") "
Oct 30 22:40:30.519292 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:30.519265    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8ee6579b-acea-4492-862b-9883f70d00c9-kube-api-access-tn89t\" (UniqueName: \"kubernetes.io/projected/8ee6579b-acea-4492-862b-9883f70d00c9-kube-api-access-tn89t\") pod \"8ee6579b-acea-4492-862b-9883f70d00c9\" (UID: \"8ee6579b-acea-4492-862b-9883f70d00c9\") "
Oct 30 22:40:30.519755 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:30.519305    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/290ae7a2-de37-4af3-82ae-a8e855b83bb9-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/290ae7a2-de37-4af3-82ae-a8e855b83bb9-model-cache\") pod \"290ae7a2-de37-4af3-82ae-a8e855b83bb9\" (UID: \"290ae7a2-de37-4af3-82ae-a8e855b83bb9\") "
Oct 30 22:40:30.519755 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:30.519341    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8ee6579b-acea-4492-862b-9883f70d00c9-home" pod "8ee6579b-acea-4492-862b-9883f70d00c9" (UID: "8ee6579b-acea-4492-862b-9883f70d00c9"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:40:30.519755 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:30.519682    5510 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8ee6579b-acea-4492-862b-9883f70d00c9-home\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:40:30.519755 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:30.519703    5510 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/290ae7a2-de37-4af3-82ae-a8e855b83bb9-home\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:40:30.519755 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:30.519717    5510 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/290ae7a2-de37-4af3-82ae-a8e855b83bb9-tmp-dir\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:40:30.519755 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:30.519729    5510 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8ee6579b-acea-4492-862b-9883f70d00c9-tmp-dir\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:40:30.520120 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:30.519754    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8ee6579b-acea-4492-862b-9883f70d00c9-model-cache" pod "8ee6579b-acea-4492-862b-9883f70d00c9" (UID: "8ee6579b-acea-4492-862b-9883f70d00c9"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:40:30.520120 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:30.519943    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/290ae7a2-de37-4af3-82ae-a8e855b83bb9-model-cache" pod "290ae7a2-de37-4af3-82ae-a8e855b83bb9" (UID: "290ae7a2-de37-4af3-82ae-a8e855b83bb9"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:40:30.522079 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:30.522038    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8ee6579b-acea-4492-862b-9883f70d00c9-kube-api-access-tn89t" pod "8ee6579b-acea-4492-862b-9883f70d00c9" (UID: "8ee6579b-acea-4492-862b-9883f70d00c9"). InnerVolumeSpecName "kube-api-access-tn89t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:40:30.522203 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:30.522158    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8ee6579b-acea-4492-862b-9883f70d00c9-tls-certs" pod "8ee6579b-acea-4492-862b-9883f70d00c9" (UID: "8ee6579b-acea-4492-862b-9883f70d00c9"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:40:30.522203 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:30.522168    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8ee6579b-acea-4492-862b-9883f70d00c9-dshm" pod "8ee6579b-acea-4492-862b-9883f70d00c9" (UID: "8ee6579b-acea-4492-862b-9883f70d00c9"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:40:30.522405 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:30.522380    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/290ae7a2-de37-4af3-82ae-a8e855b83bb9-kube-api-access-jlzzj" pod "290ae7a2-de37-4af3-82ae-a8e855b83bb9" (UID: "290ae7a2-de37-4af3-82ae-a8e855b83bb9"). InnerVolumeSpecName "kube-api-access-jlzzj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:40:30.522613 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:30.522594    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/290ae7a2-de37-4af3-82ae-a8e855b83bb9-tls-certs" pod "290ae7a2-de37-4af3-82ae-a8e855b83bb9" (UID: "290ae7a2-de37-4af3-82ae-a8e855b83bb9"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:40:30.522787 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:30.522763    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/290ae7a2-de37-4af3-82ae-a8e855b83bb9-dshm" pod "290ae7a2-de37-4af3-82ae-a8e855b83bb9" (UID: "290ae7a2-de37-4af3-82ae-a8e855b83bb9"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:40:30.620835 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:30.620695    5510 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/290ae7a2-de37-4af3-82ae-a8e855b83bb9-tls-certs\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:40:30.620835 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:30.620731    5510 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jlzzj\" (UniqueName: \"kubernetes.io/projected/290ae7a2-de37-4af3-82ae-a8e855b83bb9-kube-api-access-jlzzj\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:40:30.620835 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:30.620748    5510 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8ee6579b-acea-4492-862b-9883f70d00c9-dshm\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:40:30.620835 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:30.620763    5510 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/290ae7a2-de37-4af3-82ae-a8e855b83bb9-dshm\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:40:30.620835 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:30.620776    5510 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8ee6579b-acea-4492-862b-9883f70d00c9-tls-certs\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:40:30.620835 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:30.620788    5510 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8ee6579b-acea-4492-862b-9883f70d00c9-model-cache\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:40:30.620835 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:30.620801    5510 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tn89t\" (UniqueName: \"kubernetes.io/projected/8ee6579b-acea-4492-862b-9883f70d00c9-kube-api-access-tn89t\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:40:30.620835 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:30.620814    5510 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/290ae7a2-de37-4af3-82ae-a8e855b83bb9-model-cache\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:40:31.327174 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:31.327138    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-fb0b93/premium-w6-fb0b93-kserve-5cb9948c54-w7pnk"
Oct 30 22:40:31.327174 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:31.327166    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-fb0b93/premium-w6-fb0b93-kserve-5cb9948c54-w7pnk" event={"ID":"290ae7a2-de37-4af3-82ae-a8e855b83bb9","Type":"ContainerDied","Data":"3e22d6082f796674e2c31844b9107580ee5df38cf95021ec51f88cefde8ebe72"}
Oct 30 22:40:31.327765 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:31.327216    5510 scope.go:122] "RemoveContainer" containerID="9f802ceb1f913781c756a6ea49cc3aef5e94b3a6b6e72248b31cc0a33d3e9691"
Oct 30 22:40:31.328551 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:31.328532    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-fb0b93/distinct-w6-fb0b93-kserve-58db6b6757-6mww8"
Oct 30 22:40:31.328613 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:31.328560    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-fb0b93/distinct-w6-fb0b93-kserve-58db6b6757-6mww8" event={"ID":"8ee6579b-acea-4492-862b-9883f70d00c9","Type":"ContainerDied","Data":"a4a36d48d33b4b00ce9271673d274ba760abb031ce06b77b78a7d926c5d2a656"}
Oct 30 22:40:31.335887 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:31.335865    5510 scope.go:122] "RemoveContainer" containerID="097741694ce77df0e84010eb664b49a7ca93e0eac3e0a699133ff9d5af9e104d"
Oct 30 22:40:31.368346 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:31.368315    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-fb0b93/distinct-w6-fb0b93-kserve-58db6b6757-6mww8" podUID="8ee6579b-acea-4492-862b-9883f70d00c9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:40:31.481074 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:31.480990    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w6-fb0b93/distinct-w6-fb0b93-kserve-58db6b6757-6mww8"]
Oct 30 22:40:31.483865 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:31.483840    5510 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w6-fb0b93/distinct-w6-fb0b93-kserve-58db6b6757-6mww8"]
Oct 30 22:40:31.551756 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:31.551727    5510 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8ee6579b-acea-4492-862b-9883f70d00c9" path="/var/lib/kubelet/pods/8ee6579b-acea-4492-862b-9883f70d00c9/volumes"
Oct 30 22:40:31.587203 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:31.587169    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w6-fb0b93/premium-w6-fb0b93-kserve-5cb9948c54-w7pnk"]
Oct 30 22:40:31.591351 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:31.591329    5510 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w6-fb0b93/premium-w6-fb0b93-kserve-5cb9948c54-w7pnk"]
Oct 30 22:40:32.479214 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:32.479189    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w1-943d8f-openshift-default-6c57fb7747-z6x89"
Oct 30 22:40:32.538399 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:32.538365    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dcbe788e-cc5c-4807-af27-b98a507ea810-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dcbe788e-cc5c-4807-af27-b98a507ea810-istio-data\") pod \"dcbe788e-cc5c-4807-af27-b98a507ea810\" (UID: \"dcbe788e-cc5c-4807-af27-b98a507ea810\") "
Oct 30 22:40:32.538556 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:32.538407    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dcbe788e-cc5c-4807-af27-b98a507ea810-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dcbe788e-cc5c-4807-af27-b98a507ea810-workload-socket\") pod \"dcbe788e-cc5c-4807-af27-b98a507ea810\" (UID: \"dcbe788e-cc5c-4807-af27-b98a507ea810\") "
Oct 30 22:40:32.538556 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:32.538474    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dcbe788e-cc5c-4807-af27-b98a507ea810-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dcbe788e-cc5c-4807-af27-b98a507ea810-credential-socket\") pod \"dcbe788e-cc5c-4807-af27-b98a507ea810\" (UID: \"dcbe788e-cc5c-4807-af27-b98a507ea810\") "
Oct 30 22:40:32.538556 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:32.538509    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/dcbe788e-cc5c-4807-af27-b98a507ea810-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dcbe788e-cc5c-4807-af27-b98a507ea810-istiod-ca-cert\") pod \"dcbe788e-cc5c-4807-af27-b98a507ea810\" (UID: \"dcbe788e-cc5c-4807-af27-b98a507ea810\") "
Oct 30 22:40:32.538556 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:32.538534    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dcbe788e-cc5c-4807-af27-b98a507ea810-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dcbe788e-cc5c-4807-af27-b98a507ea810-istio-envoy\") pod \"dcbe788e-cc5c-4807-af27-b98a507ea810\" (UID: \"dcbe788e-cc5c-4807-af27-b98a507ea810\") "
Oct 30 22:40:32.538800 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:32.538565    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/dcbe788e-cc5c-4807-af27-b98a507ea810-kube-api-access-jt726\" (UniqueName: \"kubernetes.io/projected/dcbe788e-cc5c-4807-af27-b98a507ea810-kube-api-access-jt726\") pod \"dcbe788e-cc5c-4807-af27-b98a507ea810\" (UID: \"dcbe788e-cc5c-4807-af27-b98a507ea810\") "
Oct 30 22:40:32.538800 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:32.538687    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/dcbe788e-cc5c-4807-af27-b98a507ea810-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dcbe788e-cc5c-4807-af27-b98a507ea810-istio-podinfo\") pod \"dcbe788e-cc5c-4807-af27-b98a507ea810\" (UID: \"dcbe788e-cc5c-4807-af27-b98a507ea810\") "
Oct 30 22:40:32.538800 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:32.538714    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dcbe788e-cc5c-4807-af27-b98a507ea810-istio-data" pod "dcbe788e-cc5c-4807-af27-b98a507ea810" (UID: "dcbe788e-cc5c-4807-af27-b98a507ea810"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:40:32.538800 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:32.538725    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/dcbe788e-cc5c-4807-af27-b98a507ea810-istio-token\" (UniqueName: \"kubernetes.io/projected/dcbe788e-cc5c-4807-af27-b98a507ea810-istio-token\") pod \"dcbe788e-cc5c-4807-af27-b98a507ea810\" (UID: \"dcbe788e-cc5c-4807-af27-b98a507ea810\") "
Oct 30 22:40:32.538800 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:32.538741    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dcbe788e-cc5c-4807-af27-b98a507ea810-workload-socket" pod "dcbe788e-cc5c-4807-af27-b98a507ea810" (UID: "dcbe788e-cc5c-4807-af27-b98a507ea810"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:40:32.538800 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:32.538767    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dcbe788e-cc5c-4807-af27-b98a507ea810-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dcbe788e-cc5c-4807-af27-b98a507ea810-workload-certs\") pod \"dcbe788e-cc5c-4807-af27-b98a507ea810\" (UID: \"dcbe788e-cc5c-4807-af27-b98a507ea810\") "
Oct 30 22:40:32.539100 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:32.538881    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dcbe788e-cc5c-4807-af27-b98a507ea810-credential-socket" pod "dcbe788e-cc5c-4807-af27-b98a507ea810" (UID: "dcbe788e-cc5c-4807-af27-b98a507ea810"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:40:32.539205 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:32.539156    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dcbe788e-cc5c-4807-af27-b98a507ea810-workload-certs" pod "dcbe788e-cc5c-4807-af27-b98a507ea810" (UID: "dcbe788e-cc5c-4807-af27-b98a507ea810"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:40:32.539205 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:32.539164    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dcbe788e-cc5c-4807-af27-b98a507ea810-istiod-ca-cert" pod "dcbe788e-cc5c-4807-af27-b98a507ea810" (UID: "dcbe788e-cc5c-4807-af27-b98a507ea810"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:40:32.539416 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:32.539259    5510 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dcbe788e-cc5c-4807-af27-b98a507ea810-credential-socket\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:40:32.539416 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:32.539277    5510 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dcbe788e-cc5c-4807-af27-b98a507ea810-istiod-ca-cert\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:40:32.539416 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:32.539292    5510 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dcbe788e-cc5c-4807-af27-b98a507ea810-workload-certs\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:40:32.539416 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:32.539305    5510 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dcbe788e-cc5c-4807-af27-b98a507ea810-istio-data\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:40:32.539416 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:32.539320    5510 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dcbe788e-cc5c-4807-af27-b98a507ea810-workload-socket\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:40:32.540878 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:32.540851    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dcbe788e-cc5c-4807-af27-b98a507ea810-istio-envoy" pod "dcbe788e-cc5c-4807-af27-b98a507ea810" (UID: "dcbe788e-cc5c-4807-af27-b98a507ea810"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:40:32.541002 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:32.540953    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dcbe788e-cc5c-4807-af27-b98a507ea810-kube-api-access-jt726" pod "dcbe788e-cc5c-4807-af27-b98a507ea810" (UID: "dcbe788e-cc5c-4807-af27-b98a507ea810"). InnerVolumeSpecName "kube-api-access-jt726". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:40:32.541002 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:32.540978    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/dcbe788e-cc5c-4807-af27-b98a507ea810-istio-podinfo" pod "dcbe788e-cc5c-4807-af27-b98a507ea810" (UID: "dcbe788e-cc5c-4807-af27-b98a507ea810"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:40:32.541002 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:32.540979    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dcbe788e-cc5c-4807-af27-b98a507ea810-istio-token" pod "dcbe788e-cc5c-4807-af27-b98a507ea810" (UID: "dcbe788e-cc5c-4807-af27-b98a507ea810"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:40:32.640651 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:32.640612    5510 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dcbe788e-cc5c-4807-af27-b98a507ea810-istio-envoy\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:40:32.640651 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:32.640648    5510 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jt726\" (UniqueName: \"kubernetes.io/projected/dcbe788e-cc5c-4807-af27-b98a507ea810-kube-api-access-jt726\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:40:32.640889 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:32.640690    5510 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dcbe788e-cc5c-4807-af27-b98a507ea810-istio-podinfo\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:40:32.640889 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:32.640702    5510 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dcbe788e-cc5c-4807-af27-b98a507ea810-istio-token\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:40:32.817694 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:32.817576    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:40:33.042071 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:33.042039    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:40:33.342558 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:33.342523    5510 generic.go:356] "Generic (PLEG): container finished" podID="dcbe788e-cc5c-4807-af27-b98a507ea810" containerID="eb7566c34b76578a2c3664993b4ec0c90a0b953ce9f17a85614c79f8ee92657b" exitCode=0
Oct 30 22:40:33.342742 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:33.342593    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w1-943d8f-openshift-default-6c57fb7747-z6x89"
Oct 30 22:40:33.342742 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:33.342610    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w1-943d8f-openshift-default-6c57fb7747-z6x89" event={"ID":"dcbe788e-cc5c-4807-af27-b98a507ea810","Type":"ContainerDied","Data":"eb7566c34b76578a2c3664993b4ec0c90a0b953ce9f17a85614c79f8ee92657b"}
Oct 30 22:40:33.342742 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:33.342726    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w1-943d8f-openshift-default-6c57fb7747-z6x89" event={"ID":"dcbe788e-cc5c-4807-af27-b98a507ea810","Type":"ContainerDied","Data":"a097d38a60706d1740c4dea99d27bab8bce52b1031a17d69723ab7fc6fb7caf4"}
Oct 30 22:40:33.342890 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:33.342753    5510 scope.go:122] "RemoveContainer" containerID="eb7566c34b76578a2c3664993b4ec0c90a0b953ce9f17a85614c79f8ee92657b"
Oct 30 22:40:33.352457 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:33.352441    5510 scope.go:122] "RemoveContainer" containerID="d7f8bddcd52f0750ee71642815306a33677a593752edf6eb4fdc0e6bdbe20a33"
Oct 30 22:40:33.361369 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:33.361331    5510 scope.go:122] "RemoveContainer" containerID="eb7566c34b76578a2c3664993b4ec0c90a0b953ce9f17a85614c79f8ee92657b"
Oct 30 22:40:33.361667 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:40:33.361634    5510 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eb7566c34b76578a2c3664993b4ec0c90a0b953ce9f17a85614c79f8ee92657b\": container with ID starting with eb7566c34b76578a2c3664993b4ec0c90a0b953ce9f17a85614c79f8ee92657b not found: ID does not exist" containerID="eb7566c34b76578a2c3664993b4ec0c90a0b953ce9f17a85614c79f8ee92657b"
Oct 30 22:40:33.361733 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:33.361678    5510 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eb7566c34b76578a2c3664993b4ec0c90a0b953ce9f17a85614c79f8ee92657b"} err="failed to get container status \"eb7566c34b76578a2c3664993b4ec0c90a0b953ce9f17a85614c79f8ee92657b\": rpc error: code = NotFound desc = could not find container \"eb7566c34b76578a2c3664993b4ec0c90a0b953ce9f17a85614c79f8ee92657b\": container with ID starting with eb7566c34b76578a2c3664993b4ec0c90a0b953ce9f17a85614c79f8ee92657b not found: ID does not exist"
Oct 30 22:40:33.361733 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:33.361696    5510 scope.go:122] "RemoveContainer" containerID="d7f8bddcd52f0750ee71642815306a33677a593752edf6eb4fdc0e6bdbe20a33"
Oct 30 22:40:33.361963 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:40:33.361948    5510 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d7f8bddcd52f0750ee71642815306a33677a593752edf6eb4fdc0e6bdbe20a33\": container with ID starting with d7f8bddcd52f0750ee71642815306a33677a593752edf6eb4fdc0e6bdbe20a33 not found: ID does not exist" containerID="d7f8bddcd52f0750ee71642815306a33677a593752edf6eb4fdc0e6bdbe20a33"
Oct 30 22:40:33.362020 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:33.361967    5510 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d7f8bddcd52f0750ee71642815306a33677a593752edf6eb4fdc0e6bdbe20a33"} err="failed to get container status \"d7f8bddcd52f0750ee71642815306a33677a593752edf6eb4fdc0e6bdbe20a33\": rpc error: code = NotFound desc = could not find container \"d7f8bddcd52f0750ee71642815306a33677a593752edf6eb4fdc0e6bdbe20a33\": container with ID starting with d7f8bddcd52f0750ee71642815306a33677a593752edf6eb4fdc0e6bdbe20a33 not found: ID does not exist"
Oct 30 22:40:33.544894 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:33.544857    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w1-943d8f-openshift-default-6c57fb7747-z6x89"]
Oct 30 22:40:33.553613 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:33.553582    5510 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="290ae7a2-de37-4af3-82ae-a8e855b83bb9" path="/var/lib/kubelet/pods/290ae7a2-de37-4af3-82ae-a8e855b83bb9/volumes"
Oct 30 22:40:33.591493 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:33.591453    5510 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w1-943d8f-openshift-default-6c57fb7747-z6x89"]
Oct 30 22:40:35.505613 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:35.505576    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:40:35.551359 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:35.551317    5510 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="dcbe788e-cc5c-4807-af27-b98a507ea810" path="/var/lib/kubelet/pods/dcbe788e-cc5c-4807-af27-b98a507ea810/volumes"
Oct 30 22:40:37.596606 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:37.596572    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:40:38.419311 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:40:38.419274    5510 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6gjbf" containerName="extractor"
Oct 30 22:40:39.030269 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:39.030219    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:40:39.281609 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:39.281518    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:40:46.585462 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:46.585426    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-fb0b93-65f446cc49-89bdv"]
Oct 30 22:40:46.585988 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:46.585633    5510 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-fb0b93-65f446cc49-89bdv" podUID="50ad0ebb-47d3-476a-8b19-33d875cd0b1f" containerName="maas-api" containerID="cri-o://35e95cc60a10e1a0e206f56a33188e2abf680825405c912866a191799285fe67" gracePeriod=30
Oct 30 22:40:46.736579 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:46.736537    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:40:46.825006 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:46.824982    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-fb0b93-65f446cc49-89bdv"
Oct 30 22:40:46.873829 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:46.873737    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/50ad0ebb-47d3-476a-8b19-33d875cd0b1f-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/50ad0ebb-47d3-476a-8b19-33d875cd0b1f-maas-api-tls\") pod \"50ad0ebb-47d3-476a-8b19-33d875cd0b1f\" (UID: \"50ad0ebb-47d3-476a-8b19-33d875cd0b1f\") "
Oct 30 22:40:46.873829 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:46.873792    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/50ad0ebb-47d3-476a-8b19-33d875cd0b1f-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/50ad0ebb-47d3-476a-8b19-33d875cd0b1f-openshift-service-ca\") pod \"50ad0ebb-47d3-476a-8b19-33d875cd0b1f\" (UID: \"50ad0ebb-47d3-476a-8b19-33d875cd0b1f\") "
Oct 30 22:40:46.873829 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:46.873820    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/50ad0ebb-47d3-476a-8b19-33d875cd0b1f-metrics-certs\" (UniqueName: \"kubernetes.io/secret/50ad0ebb-47d3-476a-8b19-33d875cd0b1f-metrics-certs\") pod \"50ad0ebb-47d3-476a-8b19-33d875cd0b1f\" (UID: \"50ad0ebb-47d3-476a-8b19-33d875cd0b1f\") "
Oct 30 22:40:46.874123 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:46.873867    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/50ad0ebb-47d3-476a-8b19-33d875cd0b1f-kube-api-access-jnmzx\" (UniqueName: \"kubernetes.io/projected/50ad0ebb-47d3-476a-8b19-33d875cd0b1f-kube-api-access-jnmzx\") pod \"50ad0ebb-47d3-476a-8b19-33d875cd0b1f\" (UID: \"50ad0ebb-47d3-476a-8b19-33d875cd0b1f\") "
Oct 30 22:40:46.874278 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:46.874246    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/50ad0ebb-47d3-476a-8b19-33d875cd0b1f-openshift-service-ca" pod "50ad0ebb-47d3-476a-8b19-33d875cd0b1f" (UID: "50ad0ebb-47d3-476a-8b19-33d875cd0b1f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:40:46.876130 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:46.876103    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/50ad0ebb-47d3-476a-8b19-33d875cd0b1f-metrics-certs" pod "50ad0ebb-47d3-476a-8b19-33d875cd0b1f" (UID: "50ad0ebb-47d3-476a-8b19-33d875cd0b1f"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:40:46.876308 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:46.876283    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/50ad0ebb-47d3-476a-8b19-33d875cd0b1f-maas-api-tls" pod "50ad0ebb-47d3-476a-8b19-33d875cd0b1f" (UID: "50ad0ebb-47d3-476a-8b19-33d875cd0b1f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:40:46.876383 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:46.876282    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/50ad0ebb-47d3-476a-8b19-33d875cd0b1f-kube-api-access-jnmzx" pod "50ad0ebb-47d3-476a-8b19-33d875cd0b1f" (UID: "50ad0ebb-47d3-476a-8b19-33d875cd0b1f"). InnerVolumeSpecName "kube-api-access-jnmzx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:40:46.975102 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:46.975068    5510 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/50ad0ebb-47d3-476a-8b19-33d875cd0b1f-metrics-certs\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:40:46.975102 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:46.975097    5510 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jnmzx\" (UniqueName: \"kubernetes.io/projected/50ad0ebb-47d3-476a-8b19-33d875cd0b1f-kube-api-access-jnmzx\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:40:46.975102 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:46.975107    5510 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/50ad0ebb-47d3-476a-8b19-33d875cd0b1f-maas-api-tls\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:40:46.975344 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:46.975118    5510 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/50ad0ebb-47d3-476a-8b19-33d875cd0b1f-openshift-service-ca\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:40:47.409775 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:47.409735    5510 generic.go:356] "Generic (PLEG): container finished" podID="50ad0ebb-47d3-476a-8b19-33d875cd0b1f" containerID="35e95cc60a10e1a0e206f56a33188e2abf680825405c912866a191799285fe67" exitCode=0
Oct 30 22:40:47.410051 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:47.409792    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-fb0b93-65f446cc49-89bdv"
Oct 30 22:40:47.410051 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:47.409824    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-fb0b93-65f446cc49-89bdv" event={"ID":"50ad0ebb-47d3-476a-8b19-33d875cd0b1f","Type":"ContainerDied","Data":"35e95cc60a10e1a0e206f56a33188e2abf680825405c912866a191799285fe67"}
Oct 30 22:40:47.410051 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:47.409889    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-fb0b93-65f446cc49-89bdv" event={"ID":"50ad0ebb-47d3-476a-8b19-33d875cd0b1f","Type":"ContainerDied","Data":"9787158922bf01d3fc06c6654822709e5fc99aa24052453e20f96299d858b68c"}
Oct 30 22:40:47.410051 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:47.409912    5510 scope.go:122] "RemoveContainer" containerID="35e95cc60a10e1a0e206f56a33188e2abf680825405c912866a191799285fe67"
Oct 30 22:40:47.419338 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:47.419314    5510 scope.go:122] "RemoveContainer" containerID="35e95cc60a10e1a0e206f56a33188e2abf680825405c912866a191799285fe67"
Oct 30 22:40:47.419639 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:40:47.419619    5510 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"35e95cc60a10e1a0e206f56a33188e2abf680825405c912866a191799285fe67\": container with ID starting with 35e95cc60a10e1a0e206f56a33188e2abf680825405c912866a191799285fe67 not found: ID does not exist" containerID="35e95cc60a10e1a0e206f56a33188e2abf680825405c912866a191799285fe67"
Oct 30 22:40:47.419722 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:47.419647    5510 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"35e95cc60a10e1a0e206f56a33188e2abf680825405c912866a191799285fe67"} err="failed to get container status \"35e95cc60a10e1a0e206f56a33188e2abf680825405c912866a191799285fe67\": rpc error: code = NotFound desc = could not find container \"35e95cc60a10e1a0e206f56a33188e2abf680825405c912866a191799285fe67\": container with ID starting with 35e95cc60a10e1a0e206f56a33188e2abf680825405c912866a191799285fe67 not found: ID does not exist"
Oct 30 22:40:47.564908 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:47.564869    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-fb0b93-65f446cc49-89bdv"]
Oct 30 22:40:47.611278 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:47.611246    5510 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-fb0b93-65f446cc49-89bdv"]
Oct 30 22:40:48.162154 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:48.162114    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:40:48.436089 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:48.436012    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:40:49.087370 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:49.087337    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:40:49.551086 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:49.551055    5510 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="50ad0ebb-47d3-476a-8b19-33d875cd0b1f" path="/var/lib/kubelet/pods/50ad0ebb-47d3-476a-8b19-33d875cd0b1f/volumes"
Oct 30 22:40:52.419992 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:52.419948    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:40:52.654607 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:52.654574    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:40:53.256968 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:53.256925    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:40:55.421824 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:55.421787    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-a00d9b-d9c558db-ltxtj"]
Oct 30 22:40:55.422224 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:55.421997    5510 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-a00d9b-d9c558db-ltxtj" podUID="32dcff45-2112-479a-847a-43c0c9b26de4" containerName="maas-api" containerID="cri-o://8c64fe4357270d89ad89d124f56e1e7d4be311492acb8ebc8a1f4e7967bb6f76" gracePeriod=30
Oct 30 22:40:55.563363 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:55.563312    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:40:55.666170 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:55.666143    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-a00d9b-d9c558db-ltxtj"
Oct 30 22:40:55.758711 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:55.758616    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/32dcff45-2112-479a-847a-43c0c9b26de4-metrics-certs\" (UniqueName: \"kubernetes.io/secret/32dcff45-2112-479a-847a-43c0c9b26de4-metrics-certs\") pod \"32dcff45-2112-479a-847a-43c0c9b26de4\" (UID: \"32dcff45-2112-479a-847a-43c0c9b26de4\") "
Oct 30 22:40:55.758711 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:55.758683    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/32dcff45-2112-479a-847a-43c0c9b26de4-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/32dcff45-2112-479a-847a-43c0c9b26de4-openshift-service-ca\") pod \"32dcff45-2112-479a-847a-43c0c9b26de4\" (UID: \"32dcff45-2112-479a-847a-43c0c9b26de4\") "
Oct 30 22:40:55.758928 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:55.758737    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/32dcff45-2112-479a-847a-43c0c9b26de4-kube-api-access-985mk\" (UniqueName: \"kubernetes.io/projected/32dcff45-2112-479a-847a-43c0c9b26de4-kube-api-access-985mk\") pod \"32dcff45-2112-479a-847a-43c0c9b26de4\" (UID: \"32dcff45-2112-479a-847a-43c0c9b26de4\") "
Oct 30 22:40:55.758928 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:55.758843    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/32dcff45-2112-479a-847a-43c0c9b26de4-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/32dcff45-2112-479a-847a-43c0c9b26de4-maas-api-tls\") pod \"32dcff45-2112-479a-847a-43c0c9b26de4\" (UID: \"32dcff45-2112-479a-847a-43c0c9b26de4\") "
Oct 30 22:40:55.759089 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:55.759056    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/32dcff45-2112-479a-847a-43c0c9b26de4-openshift-service-ca" pod "32dcff45-2112-479a-847a-43c0c9b26de4" (UID: "32dcff45-2112-479a-847a-43c0c9b26de4"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:40:55.759202 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:55.759182    5510 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/32dcff45-2112-479a-847a-43c0c9b26de4-openshift-service-ca\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:40:55.760984 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:55.760959    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/32dcff45-2112-479a-847a-43c0c9b26de4-maas-api-tls" pod "32dcff45-2112-479a-847a-43c0c9b26de4" (UID: "32dcff45-2112-479a-847a-43c0c9b26de4"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:40:55.760984 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:55.760975    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/32dcff45-2112-479a-847a-43c0c9b26de4-metrics-certs" pod "32dcff45-2112-479a-847a-43c0c9b26de4" (UID: "32dcff45-2112-479a-847a-43c0c9b26de4"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:40:55.761126 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:55.761021    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/32dcff45-2112-479a-847a-43c0c9b26de4-kube-api-access-985mk" pod "32dcff45-2112-479a-847a-43c0c9b26de4" (UID: "32dcff45-2112-479a-847a-43c0c9b26de4"). InnerVolumeSpecName "kube-api-access-985mk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:40:55.859775 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:55.859745    5510 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/32dcff45-2112-479a-847a-43c0c9b26de4-maas-api-tls\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:40:55.859775 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:55.859770    5510 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/32dcff45-2112-479a-847a-43c0c9b26de4-metrics-certs\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:40:55.859775 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:55.859781    5510 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-985mk\" (UniqueName: \"kubernetes.io/projected/32dcff45-2112-479a-847a-43c0c9b26de4-kube-api-access-985mk\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:40:56.455508 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:56.455472    5510 generic.go:356] "Generic (PLEG): container finished" podID="32dcff45-2112-479a-847a-43c0c9b26de4" containerID="8c64fe4357270d89ad89d124f56e1e7d4be311492acb8ebc8a1f4e7967bb6f76" exitCode=0
Oct 30 22:40:56.456017 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:56.455520    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-a00d9b-d9c558db-ltxtj"
Oct 30 22:40:56.456017 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:56.455540    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-a00d9b-d9c558db-ltxtj" event={"ID":"32dcff45-2112-479a-847a-43c0c9b26de4","Type":"ContainerDied","Data":"8c64fe4357270d89ad89d124f56e1e7d4be311492acb8ebc8a1f4e7967bb6f76"}
Oct 30 22:40:56.456017 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:56.455598    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-a00d9b-d9c558db-ltxtj" event={"ID":"32dcff45-2112-479a-847a-43c0c9b26de4","Type":"ContainerDied","Data":"a194793a5514d201d346de0af5437b6f0fc7031fc5c0eee55985d3513bbaeb9d"}
Oct 30 22:40:56.456017 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:56.455632    5510 scope.go:122] "RemoveContainer" containerID="8c64fe4357270d89ad89d124f56e1e7d4be311492acb8ebc8a1f4e7967bb6f76"
Oct 30 22:40:56.465439 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:56.465419    5510 scope.go:122] "RemoveContainer" containerID="8c64fe4357270d89ad89d124f56e1e7d4be311492acb8ebc8a1f4e7967bb6f76"
Oct 30 22:40:56.465724 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:40:56.465706    5510 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8c64fe4357270d89ad89d124f56e1e7d4be311492acb8ebc8a1f4e7967bb6f76\": container with ID starting with 8c64fe4357270d89ad89d124f56e1e7d4be311492acb8ebc8a1f4e7967bb6f76 not found: ID does not exist" containerID="8c64fe4357270d89ad89d124f56e1e7d4be311492acb8ebc8a1f4e7967bb6f76"
Oct 30 22:40:56.465778 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:56.465733    5510 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8c64fe4357270d89ad89d124f56e1e7d4be311492acb8ebc8a1f4e7967bb6f76"} err="failed to get container status \"8c64fe4357270d89ad89d124f56e1e7d4be311492acb8ebc8a1f4e7967bb6f76\": rpc error: code = NotFound desc = could not find container \"8c64fe4357270d89ad89d124f56e1e7d4be311492acb8ebc8a1f4e7967bb6f76\": container with ID starting with 8c64fe4357270d89ad89d124f56e1e7d4be311492acb8ebc8a1f4e7967bb6f76 not found: ID does not exist"
Oct 30 22:40:56.629901 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:56.629863    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-a00d9b-d9c558db-ltxtj"]
Oct 30 22:40:56.676926 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:56.676892    5510 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-a00d9b-d9c558db-ltxtj"]
Oct 30 22:40:56.915008 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:56.914969    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:40:57.135838 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:57.135799    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:40:57.333607 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:57.333576    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:40:57.551797 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:57.551766    5510 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="32dcff45-2112-479a-847a-43c0c9b26de4" path="/var/lib/kubelet/pods/32dcff45-2112-479a-847a-43c0c9b26de4/volumes"
Oct 30 22:40:57.967292 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:57.967256    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:40:59.938424 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:40:59.938380    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:41:03.886631 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:03.886598    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:41:04.094375 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:04.094338    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:41:04.288685 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:04.288636    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:41:08.419820 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:41:08.419782    5510 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6gjbf" containerName="extractor"
Oct 30 22:41:09.549186 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:09.549140    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:41:09.653440 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:09.653403    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-869c494b8f-cck5w"]
Oct 30 22:41:09.653647 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:09.653627    5510 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/maas-controller-869c494b8f-cck5w" podUID="cc77e063-6bda-4c8b-818c-9ba4ac8fb3db" containerName="manager" containerID="cri-o://388b3d6d50b11ecb842131351e150d862312ecd95f9ca3d384ec5aa4bd2c3234" gracePeriod=30
Oct 30 22:41:09.743583 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:09.743550    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:41:09.901432 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:09.901404    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-869c494b8f-cck5w"
Oct 30 22:41:10.095856 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:10.095753    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/cc77e063-6bda-4c8b-818c-9ba4ac8fb3db-metrics-certs\" (UniqueName: \"kubernetes.io/secret/cc77e063-6bda-4c8b-818c-9ba4ac8fb3db-metrics-certs\") pod \"cc77e063-6bda-4c8b-818c-9ba4ac8fb3db\" (UID: \"cc77e063-6bda-4c8b-818c-9ba4ac8fb3db\") "
Oct 30 22:41:10.095856 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:10.095848    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cc77e063-6bda-4c8b-818c-9ba4ac8fb3db-kube-api-access-5c8kh\" (UniqueName: \"kubernetes.io/projected/cc77e063-6bda-4c8b-818c-9ba4ac8fb3db-kube-api-access-5c8kh\") pod \"cc77e063-6bda-4c8b-818c-9ba4ac8fb3db\" (UID: \"cc77e063-6bda-4c8b-818c-9ba4ac8fb3db\") "
Oct 30 22:41:10.096134 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:10.095922    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/cc77e063-6bda-4c8b-818c-9ba4ac8fb3db-webhook-cert\" (UniqueName: \"kubernetes.io/secret/cc77e063-6bda-4c8b-818c-9ba4ac8fb3db-webhook-cert\") pod \"cc77e063-6bda-4c8b-818c-9ba4ac8fb3db\" (UID: \"cc77e063-6bda-4c8b-818c-9ba4ac8fb3db\") "
Oct 30 22:41:10.098102 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:10.098053    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cc77e063-6bda-4c8b-818c-9ba4ac8fb3db-metrics-certs" pod "cc77e063-6bda-4c8b-818c-9ba4ac8fb3db" (UID: "cc77e063-6bda-4c8b-818c-9ba4ac8fb3db"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:41:10.098237 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:10.098097    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cc77e063-6bda-4c8b-818c-9ba4ac8fb3db-webhook-cert" pod "cc77e063-6bda-4c8b-818c-9ba4ac8fb3db" (UID: "cc77e063-6bda-4c8b-818c-9ba4ac8fb3db"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:41:10.098237 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:10.098126    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cc77e063-6bda-4c8b-818c-9ba4ac8fb3db-kube-api-access-5c8kh" pod "cc77e063-6bda-4c8b-818c-9ba4ac8fb3db" (UID: "cc77e063-6bda-4c8b-818c-9ba4ac8fb3db"). InnerVolumeSpecName "kube-api-access-5c8kh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:41:10.197055 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:10.197019    5510 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5c8kh\" (UniqueName: \"kubernetes.io/projected/cc77e063-6bda-4c8b-818c-9ba4ac8fb3db-kube-api-access-5c8kh\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:41:10.197055 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:10.197049    5510 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cc77e063-6bda-4c8b-818c-9ba4ac8fb3db-webhook-cert\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:41:10.197055 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:10.197059    5510 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cc77e063-6bda-4c8b-818c-9ba4ac8fb3db-metrics-certs\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:41:10.524463 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:10.524427    5510 generic.go:356] "Generic (PLEG): container finished" podID="cc77e063-6bda-4c8b-818c-9ba4ac8fb3db" containerID="388b3d6d50b11ecb842131351e150d862312ecd95f9ca3d384ec5aa4bd2c3234" exitCode=0
Oct 30 22:41:10.524626 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:10.524479    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-869c494b8f-cck5w"
Oct 30 22:41:10.524626 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:10.524513    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-869c494b8f-cck5w" event={"ID":"cc77e063-6bda-4c8b-818c-9ba4ac8fb3db","Type":"ContainerDied","Data":"388b3d6d50b11ecb842131351e150d862312ecd95f9ca3d384ec5aa4bd2c3234"}
Oct 30 22:41:10.524626 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:10.524567    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-869c494b8f-cck5w" event={"ID":"cc77e063-6bda-4c8b-818c-9ba4ac8fb3db","Type":"ContainerDied","Data":"190983fb99b4d1bbb400a0a6d995204d68fd2c61e80cb81eea46a04652730f49"}
Oct 30 22:41:10.524626 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:10.524596    5510 scope.go:122] "RemoveContainer" containerID="388b3d6d50b11ecb842131351e150d862312ecd95f9ca3d384ec5aa4bd2c3234"
Oct 30 22:41:10.534002 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:10.533981    5510 scope.go:122] "RemoveContainer" containerID="388b3d6d50b11ecb842131351e150d862312ecd95f9ca3d384ec5aa4bd2c3234"
Oct 30 22:41:10.534271 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:41:10.534253    5510 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"388b3d6d50b11ecb842131351e150d862312ecd95f9ca3d384ec5aa4bd2c3234\": container with ID starting with 388b3d6d50b11ecb842131351e150d862312ecd95f9ca3d384ec5aa4bd2c3234 not found: ID does not exist" containerID="388b3d6d50b11ecb842131351e150d862312ecd95f9ca3d384ec5aa4bd2c3234"
Oct 30 22:41:10.534316 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:10.534281    5510 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"388b3d6d50b11ecb842131351e150d862312ecd95f9ca3d384ec5aa4bd2c3234"} err="failed to get container status \"388b3d6d50b11ecb842131351e150d862312ecd95f9ca3d384ec5aa4bd2c3234\": rpc error: code = NotFound desc = could not find container \"388b3d6d50b11ecb842131351e150d862312ecd95f9ca3d384ec5aa4bd2c3234\": container with ID starting with 388b3d6d50b11ecb842131351e150d862312ecd95f9ca3d384ec5aa4bd2c3234 not found: ID does not exist"
Oct 30 22:41:10.680600 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:10.680565    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-869c494b8f-cck5w"]
Oct 30 22:41:10.726911 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:10.726877    5510 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-869c494b8f-cck5w"]
Oct 30 22:41:11.469124 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:11.469084    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:41:11.531468 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:11.531436    5510 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-869c494b8f-w9w5n"]
Oct 30 22:41:11.531823 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:11.531810    5510 cpu_manager.go:414] "containerMap: removing container" podUID="dcbe788e-cc5c-4807-af27-b98a507ea810" containerName="istio-proxy"
Oct 30 22:41:11.531876 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:11.531825    5510 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="dcbe788e-cc5c-4807-af27-b98a507ea810" containerName="istio-proxy"
Oct 30 22:41:11.531876 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:11.531837    5510 cpu_manager.go:414] "containerMap: removing container" podUID="8ee6579b-acea-4492-862b-9883f70d00c9" containerName="main"
Oct 30 22:41:11.531876 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:11.531842    5510 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8ee6579b-acea-4492-862b-9883f70d00c9" containerName="main"
Oct 30 22:41:11.531876 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:11.531859    5510 cpu_manager.go:414] "containerMap: removing container" podUID="61fdf6bd-8922-4db6-a4cf-4fa9ea967b20" containerName="istio-proxy"
Oct 30 22:41:11.531876 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:11.531865    5510 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="61fdf6bd-8922-4db6-a4cf-4fa9ea967b20" containerName="istio-proxy"
Oct 30 22:41:11.531876 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:11.531875    5510 cpu_manager.go:414] "containerMap: removing container" podUID="32dcff45-2112-479a-847a-43c0c9b26de4" containerName="maas-api"
Oct 30 22:41:11.532088 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:11.531880    5510 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="32dcff45-2112-479a-847a-43c0c9b26de4" containerName="maas-api"
Oct 30 22:41:11.532088 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:11.531887    5510 cpu_manager.go:414] "containerMap: removing container" podUID="a3a8e626-b7fa-4fe1-8d74-0d8bef87fb63" containerName="istio-proxy"
Oct 30 22:41:11.532088 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:11.531892    5510 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a3a8e626-b7fa-4fe1-8d74-0d8bef87fb63" containerName="istio-proxy"
Oct 30 22:41:11.532088 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:11.531900    5510 cpu_manager.go:414] "containerMap: removing container" podUID="85f900f2-2722-4143-8974-d1c7ab81770b" containerName="istio-proxy"
Oct 30 22:41:11.532088 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:11.531905    5510 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="85f900f2-2722-4143-8974-d1c7ab81770b" containerName="istio-proxy"
Oct 30 22:41:11.532088 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:11.531912    5510 cpu_manager.go:414] "containerMap: removing container" podUID="cc069342-244c-467a-9d89-2b7fb41338e9" containerName="istio-proxy"
Oct 30 22:41:11.532088 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:11.531917    5510 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cc069342-244c-467a-9d89-2b7fb41338e9" containerName="istio-proxy"
Oct 30 22:41:11.532088 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:11.531925    5510 cpu_manager.go:414] "containerMap: removing container" podUID="195dcf2f-1af0-4404-ac3c-2e3424b9b266" containerName="maas-api"
Oct 30 22:41:11.532088 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:11.531929    5510 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="195dcf2f-1af0-4404-ac3c-2e3424b9b266" containerName="maas-api"
Oct 30 22:41:11.532088 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:11.531936    5510 cpu_manager.go:414] "containerMap: removing container" podUID="eb115651-ae12-43d8-8022-766c30575782" containerName="istio-proxy"
Oct 30 22:41:11.532088 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:11.531941    5510 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="eb115651-ae12-43d8-8022-766c30575782" containerName="istio-proxy"
Oct 30 22:41:11.532088 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:11.531949    5510 cpu_manager.go:414] "containerMap: removing container" podUID="6824a63b-432d-4967-9b83-742599c9cd94" containerName="main"
Oct 30 22:41:11.532088 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:11.531954    5510 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6824a63b-432d-4967-9b83-742599c9cd94" containerName="main"
Oct 30 22:41:11.532088 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:11.531959    5510 cpu_manager.go:414] "containerMap: removing container" podUID="b3eaa015-ceb2-41a4-b52f-67dd79e72b17" containerName="main"
Oct 30 22:41:11.532088 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:11.531964    5510 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b3eaa015-ceb2-41a4-b52f-67dd79e72b17" containerName="main"
Oct 30 22:41:11.532088 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:11.531976    5510 cpu_manager.go:414] "containerMap: removing container" podUID="4fd056ae-943a-43c1-8682-8edb3fe3b5ed" containerName="payload-pre-processing"
Oct 30 22:41:11.532088 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:11.531981    5510 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4fd056ae-943a-43c1-8682-8edb3fe3b5ed" containerName="payload-pre-processing"
Oct 30 22:41:11.532088 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:11.531987    5510 cpu_manager.go:414] "containerMap: removing container" podUID="f789c7e9-26e6-4b2d-b1d7-536dc5d526d8" containerName="istio-proxy"
Oct 30 22:41:11.532088 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:11.531991    5510 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f789c7e9-26e6-4b2d-b1d7-536dc5d526d8" containerName="istio-proxy"
Oct 30 22:41:11.532088 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:11.532001    5510 cpu_manager.go:414] "containerMap: removing container" podUID="290ae7a2-de37-4af3-82ae-a8e855b83bb9" containerName="main"
Oct 30 22:41:11.532088 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:11.532005    5510 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="290ae7a2-de37-4af3-82ae-a8e855b83bb9" containerName="main"
Oct 30 22:41:11.532088 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:11.532015    5510 cpu_manager.go:414] "containerMap: removing container" podUID="4a00e5f4-7118-41ce-b4c7-eb0f74da8237" containerName="maas-api"
Oct 30 22:41:11.532088 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:11.532021    5510 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4a00e5f4-7118-41ce-b4c7-eb0f74da8237" containerName="maas-api"
Oct 30 22:41:11.532088 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:11.532030    5510 cpu_manager.go:414] "containerMap: removing container" podUID="d303451d-0d81-4a9b-8bc6-91f019cfc4a0" containerName="payload-pre-processing"
Oct 30 22:41:11.532088 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:11.532037    5510 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d303451d-0d81-4a9b-8bc6-91f019cfc4a0" containerName="payload-pre-processing"
Oct 30 22:41:11.532088 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:11.532048    5510 cpu_manager.go:414] "containerMap: removing container" podUID="61fdf6bd-8922-4db6-a4cf-4fa9ea967b20" containerName="istio-proxy"
Oct 30 22:41:11.532088 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:11.532053    5510 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="61fdf6bd-8922-4db6-a4cf-4fa9ea967b20" containerName="istio-proxy"
Oct 30 22:41:11.532088 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:11.532059    5510 cpu_manager.go:414] "containerMap: removing container" podUID="1cd4d179-c02d-439e-afea-7dbf8b89d3ea" containerName="payload-processing"
Oct 30 22:41:11.532088 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:11.532064    5510 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1cd4d179-c02d-439e-afea-7dbf8b89d3ea" containerName="payload-processing"
Oct 30 22:41:11.532088 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:11.532072    5510 cpu_manager.go:414] "containerMap: removing container" podUID="50ad0ebb-47d3-476a-8b19-33d875cd0b1f" containerName="maas-api"
Oct 30 22:41:11.532088 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:11.532077    5510 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="50ad0ebb-47d3-476a-8b19-33d875cd0b1f" containerName="maas-api"
Oct 30 22:41:11.532088 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:11.532084    5510 cpu_manager.go:414] "containerMap: removing container" podUID="19d32fc2-60a2-4040-813c-8b2127047510" containerName="payload-processing"
Oct 30 22:41:11.532088 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:11.532088    5510 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="19d32fc2-60a2-4040-813c-8b2127047510" containerName="payload-processing"
Oct 30 22:41:11.532088 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:11.532095    5510 cpu_manager.go:414] "containerMap: removing container" podUID="cc77e063-6bda-4c8b-818c-9ba4ac8fb3db" containerName="manager"
Oct 30 22:41:11.532088 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:11.532099    5510 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cc77e063-6bda-4c8b-818c-9ba4ac8fb3db" containerName="manager"
Oct 30 22:41:11.532088 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:11.532105    5510 cpu_manager.go:414] "containerMap: removing container" podUID="7554b329-1324-4e03-96ac-f8545413fbe3" containerName="payload-pre-processing"
Oct 30 22:41:11.533205 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:11.532110    5510 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7554b329-1324-4e03-96ac-f8545413fbe3" containerName="payload-pre-processing"
Oct 30 22:41:11.533205 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:11.532116    5510 cpu_manager.go:414] "containerMap: removing container" podUID="d29d64dd-be94-4d3b-9888-3196c0c8b5f0" containerName="payload-processing"
Oct 30 22:41:11.533205 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:11.532120    5510 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d29d64dd-be94-4d3b-9888-3196c0c8b5f0" containerName="payload-processing"
Oct 30 22:41:11.533205 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:11.532176    5510 memory_manager.go:365] "RemoveStaleState removing state" podUID="61fdf6bd-8922-4db6-a4cf-4fa9ea967b20" containerName="istio-proxy"
Oct 30 22:41:11.533205 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:11.532183    5510 memory_manager.go:365] "RemoveStaleState removing state" podUID="32dcff45-2112-479a-847a-43c0c9b26de4" containerName="maas-api"
Oct 30 22:41:11.533205 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:11.532190    5510 memory_manager.go:365] "RemoveStaleState removing state" podUID="a3a8e626-b7fa-4fe1-8d74-0d8bef87fb63" containerName="istio-proxy"
Oct 30 22:41:11.533205 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:11.532197    5510 memory_manager.go:365] "RemoveStaleState removing state" podUID="85f900f2-2722-4143-8974-d1c7ab81770b" containerName="istio-proxy"
Oct 30 22:41:11.533205 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:11.532203    5510 memory_manager.go:365] "RemoveStaleState removing state" podUID="cc069342-244c-467a-9d89-2b7fb41338e9" containerName="istio-proxy"
Oct 30 22:41:11.533205 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:11.532209    5510 memory_manager.go:365] "RemoveStaleState removing state" podUID="195dcf2f-1af0-4404-ac3c-2e3424b9b266" containerName="maas-api"
Oct 30 22:41:11.533205 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:11.532216    5510 memory_manager.go:365] "RemoveStaleState removing state" podUID="eb115651-ae12-43d8-8022-766c30575782" containerName="istio-proxy"
Oct 30 22:41:11.533205 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:11.532223    5510 memory_manager.go:365] "RemoveStaleState removing state" podUID="6824a63b-432d-4967-9b83-742599c9cd94" containerName="main"
Oct 30 22:41:11.533205 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:11.532230    5510 memory_manager.go:365] "RemoveStaleState removing state" podUID="b3eaa015-ceb2-41a4-b52f-67dd79e72b17" containerName="main"
Oct 30 22:41:11.533205 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:11.532237    5510 memory_manager.go:365] "RemoveStaleState removing state" podUID="4fd056ae-943a-43c1-8682-8edb3fe3b5ed" containerName="payload-pre-processing"
Oct 30 22:41:11.533205 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:11.532241    5510 memory_manager.go:365] "RemoveStaleState removing state" podUID="f789c7e9-26e6-4b2d-b1d7-536dc5d526d8" containerName="istio-proxy"
Oct 30 22:41:11.533205 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:11.532248    5510 memory_manager.go:365] "RemoveStaleState removing state" podUID="dcbe788e-cc5c-4807-af27-b98a507ea810" containerName="istio-proxy"
Oct 30 22:41:11.533205 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:11.532252    5510 memory_manager.go:365] "RemoveStaleState removing state" podUID="290ae7a2-de37-4af3-82ae-a8e855b83bb9" containerName="main"
Oct 30 22:41:11.533205 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:11.532259    5510 memory_manager.go:365] "RemoveStaleState removing state" podUID="4a00e5f4-7118-41ce-b4c7-eb0f74da8237" containerName="maas-api"
Oct 30 22:41:11.533205 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:11.532264    5510 memory_manager.go:365] "RemoveStaleState removing state" podUID="d303451d-0d81-4a9b-8bc6-91f019cfc4a0" containerName="payload-pre-processing"
Oct 30 22:41:11.533205 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:11.532268    5510 memory_manager.go:365] "RemoveStaleState removing state" podUID="61fdf6bd-8922-4db6-a4cf-4fa9ea967b20" containerName="istio-proxy"
Oct 30 22:41:11.533205 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:11.532273    5510 memory_manager.go:365] "RemoveStaleState removing state" podUID="1cd4d179-c02d-439e-afea-7dbf8b89d3ea" containerName="payload-processing"
Oct 30 22:41:11.533205 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:11.532277    5510 memory_manager.go:365] "RemoveStaleState removing state" podUID="50ad0ebb-47d3-476a-8b19-33d875cd0b1f" containerName="maas-api"
Oct 30 22:41:11.533205 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:11.532283    5510 memory_manager.go:365] "RemoveStaleState removing state" podUID="19d32fc2-60a2-4040-813c-8b2127047510" containerName="payload-processing"
Oct 30 22:41:11.533205 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:11.532287    5510 memory_manager.go:365] "RemoveStaleState removing state" podUID="cc77e063-6bda-4c8b-818c-9ba4ac8fb3db" containerName="manager"
Oct 30 22:41:11.533205 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:11.532292    5510 memory_manager.go:365] "RemoveStaleState removing state" podUID="7554b329-1324-4e03-96ac-f8545413fbe3" containerName="payload-pre-processing"
Oct 30 22:41:11.533205 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:11.532297    5510 memory_manager.go:365] "RemoveStaleState removing state" podUID="d29d64dd-be94-4d3b-9888-3196c0c8b5f0" containerName="payload-processing"
Oct 30 22:41:11.533205 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:11.532304    5510 memory_manager.go:365] "RemoveStaleState removing state" podUID="8ee6579b-acea-4492-862b-9883f70d00c9" containerName="main"
Oct 30 22:41:11.536641 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:11.536625    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-869c494b8f-w9w5n"
Oct 30 22:41:11.550693 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:11.550674    5510 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="cc77e063-6bda-4c8b-818c-9ba4ac8fb3db" path="/var/lib/kubelet/pods/cc77e063-6bda-4c8b-818c-9ba4ac8fb3db/volumes"
Oct 30 22:41:11.586080 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:11.586055    5510 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Oct 30 22:41:11.632519 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:11.632485    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Oct 30 22:41:11.632519 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:11.632511    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-fzht9\""
Oct 30 22:41:11.632761 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:11.632676    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Oct 30 22:41:11.634006 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:11.633986    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-869c494b8f-w9w5n"]
Oct 30 22:41:11.711115 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:11.711078    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b6c7b840-e876-4170-88d1-77e9374dc9aa-webhook-cert\") pod \"maas-controller-869c494b8f-w9w5n\" (UID: \"b6c7b840-e876-4170-88d1-77e9374dc9aa\") " pod="opendatahub/maas-controller-869c494b8f-w9w5n"
Oct 30 22:41:11.711536 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:11.711293    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b6c7b840-e876-4170-88d1-77e9374dc9aa-metrics-certs\") pod \"maas-controller-869c494b8f-w9w5n\" (UID: \"b6c7b840-e876-4170-88d1-77e9374dc9aa\") " pod="opendatahub/maas-controller-869c494b8f-w9w5n"
Oct 30 22:41:11.711536 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:11.711376    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nhk7z\" (UniqueName: \"kubernetes.io/projected/b6c7b840-e876-4170-88d1-77e9374dc9aa-kube-api-access-nhk7z\") pod \"maas-controller-869c494b8f-w9w5n\" (UID: \"b6c7b840-e876-4170-88d1-77e9374dc9aa\") " pod="opendatahub/maas-controller-869c494b8f-w9w5n"
Oct 30 22:41:11.812374 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:11.812339    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b6c7b840-e876-4170-88d1-77e9374dc9aa-webhook-cert\") pod \"maas-controller-869c494b8f-w9w5n\" (UID: \"b6c7b840-e876-4170-88d1-77e9374dc9aa\") " pod="opendatahub/maas-controller-869c494b8f-w9w5n"
Oct 30 22:41:11.812539 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:11.812396    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b6c7b840-e876-4170-88d1-77e9374dc9aa-metrics-certs\") pod \"maas-controller-869c494b8f-w9w5n\" (UID: \"b6c7b840-e876-4170-88d1-77e9374dc9aa\") " pod="opendatahub/maas-controller-869c494b8f-w9w5n"
Oct 30 22:41:11.812539 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:11.812428    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nhk7z\" (UniqueName: \"kubernetes.io/projected/b6c7b840-e876-4170-88d1-77e9374dc9aa-kube-api-access-nhk7z\") pod \"maas-controller-869c494b8f-w9w5n\" (UID: \"b6c7b840-e876-4170-88d1-77e9374dc9aa\") " pod="opendatahub/maas-controller-869c494b8f-w9w5n"
Oct 30 22:41:11.814927 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:11.814900    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b6c7b840-e876-4170-88d1-77e9374dc9aa-webhook-cert\") pod \"maas-controller-869c494b8f-w9w5n\" (UID: \"b6c7b840-e876-4170-88d1-77e9374dc9aa\") " pod="opendatahub/maas-controller-869c494b8f-w9w5n"
Oct 30 22:41:11.815048 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:11.814922    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b6c7b840-e876-4170-88d1-77e9374dc9aa-metrics-certs\") pod \"maas-controller-869c494b8f-w9w5n\" (UID: \"b6c7b840-e876-4170-88d1-77e9374dc9aa\") " pod="opendatahub/maas-controller-869c494b8f-w9w5n"
Oct 30 22:41:11.871150 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:11.871111    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nhk7z\" (UniqueName: \"kubernetes.io/projected/b6c7b840-e876-4170-88d1-77e9374dc9aa-kube-api-access-nhk7z\") pod \"maas-controller-869c494b8f-w9w5n\" (UID: \"b6c7b840-e876-4170-88d1-77e9374dc9aa\") " pod="opendatahub/maas-controller-869c494b8f-w9w5n"
Oct 30 22:41:12.148937 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:12.148854    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-869c494b8f-w9w5n"
Oct 30 22:41:12.519327 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:12.519297    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-869c494b8f-w9w5n"]
Oct 30 22:41:12.549449 ip-10-0-136-210 kubenswrapper[5510]: W1030 22:41:12.549421    5510 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6c7b840_e876_4170_88d1_77e9374dc9aa.slice/crio-5090be2e58a83993b46f031b95f5fccdd9ec4a89d6c4bc618f25dc185781a854 WatchSource:0}: Error finding container 5090be2e58a83993b46f031b95f5fccdd9ec4a89d6c4bc618f25dc185781a854: Status 404 returned error can't find the container with id 5090be2e58a83993b46f031b95f5fccdd9ec4a89d6c4bc618f25dc185781a854
Oct 30 22:41:12.550556 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:12.550540    5510 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 22:41:13.539862 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:13.539829    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-869c494b8f-w9w5n" event={"ID":"b6c7b840-e876-4170-88d1-77e9374dc9aa","Type":"ContainerStarted","Data":"5090be2e58a83993b46f031b95f5fccdd9ec4a89d6c4bc618f25dc185781a854"}
Oct 30 22:41:14.546637 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:14.546596    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-869c494b8f-w9w5n" event={"ID":"b6c7b840-e876-4170-88d1-77e9374dc9aa","Type":"ContainerStarted","Data":"8ea5426517863ca5ca39797740e244681f452cf7cb8d0bb038ff26e3e77fe91d"}
Oct 30 22:41:14.547094 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:14.546749    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-869c494b8f-w9w5n"
Oct 30 22:41:14.662906 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:14.662852    5510 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/maas-controller-869c494b8f-w9w5n" podStartSLOduration=2.675257135 podStartE2EDuration="3.662839772s" totalImagesPullingTime="987.582637ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:41:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:41:14.661230803 +0000 UTC m=+3055.222040819" watchObservedRunningTime="2026-10-30 22:41:14.662839772 +0000 UTC m=+3055.223649819"
Oct 30 22:41:17.861957 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:17.861915    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:41:19.350512 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:19.350464    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:41:19.545066 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:19.545021    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:41:23.293581 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:23.293548    5510 scope.go:122] "RemoveContainer" containerID="2667979b3f05d9ae75d6558234e3020cbfacdf3ebc0af93dec147a881b9c013b"
Oct 30 22:41:25.554082 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:25.554054    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-869c494b8f-w9w5n"
Oct 30 22:41:35.846244 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:35.846200    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:41:38.419761 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:41:38.419726    5510 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6gjbf" containerName="extractor"
Oct 30 22:41:40.881587 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:40.881545    5510 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-hfrbk"]
Oct 30 22:41:40.882106 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:40.881984    5510 cpu_manager.go:414] "containerMap: removing container" podUID="dcbe788e-cc5c-4807-af27-b98a507ea810" containerName="istio-proxy"
Oct 30 22:41:40.882106 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:40.881998    5510 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="dcbe788e-cc5c-4807-af27-b98a507ea810" containerName="istio-proxy"
Oct 30 22:41:40.882106 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:40.882081    5510 memory_manager.go:365] "RemoveStaleState removing state" podUID="dcbe788e-cc5c-4807-af27-b98a507ea810" containerName="istio-proxy"
Oct 30 22:41:40.887019 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:40.886996    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-hfrbk"
Oct 30 22:41:40.928145 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:40.928079    5510 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-llmh8"]
Oct 30 22:41:40.931502 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:40.931486    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-llmh8"
Oct 30 22:41:40.936064 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:40.936040    5510 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Oct 30 22:41:40.957936 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:40.957914    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-v8cwr\""
Oct 30 22:41:40.982455 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:40.982428    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-hfrbk"]
Oct 30 22:41:40.982857 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:40.982831    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/474fc2c2-03ae-420b-a831-c8e98a3e5ed8-istio-data\") pod \"maas-default-gateway-openshift-default-85544d4cd7-hfrbk\" (UID: \"474fc2c2-03ae-420b-a831-c8e98a3e5ed8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-hfrbk"
Oct 30 22:41:40.982963 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:40.982867    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/474fc2c2-03ae-420b-a831-c8e98a3e5ed8-workload-socket\") pod \"maas-default-gateway-openshift-default-85544d4cd7-hfrbk\" (UID: \"474fc2c2-03ae-420b-a831-c8e98a3e5ed8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-hfrbk"
Oct 30 22:41:40.983017 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:40.982969    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sjcnz\" (UniqueName: \"kubernetes.io/projected/474fc2c2-03ae-420b-a831-c8e98a3e5ed8-kube-api-access-sjcnz\") pod \"maas-default-gateway-openshift-default-85544d4cd7-hfrbk\" (UID: \"474fc2c2-03ae-420b-a831-c8e98a3e5ed8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-hfrbk"
Oct 30 22:41:40.983017 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:40.982995    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/474fc2c2-03ae-420b-a831-c8e98a3e5ed8-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-85544d4cd7-hfrbk\" (UID: \"474fc2c2-03ae-420b-a831-c8e98a3e5ed8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-hfrbk"
Oct 30 22:41:40.983126 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:40.983059    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/474fc2c2-03ae-420b-a831-c8e98a3e5ed8-istio-podinfo\") pod \"maas-default-gateway-openshift-default-85544d4cd7-hfrbk\" (UID: \"474fc2c2-03ae-420b-a831-c8e98a3e5ed8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-hfrbk"
Oct 30 22:41:40.983126 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:40.983117    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/474fc2c2-03ae-420b-a831-c8e98a3e5ed8-istio-envoy\") pod \"maas-default-gateway-openshift-default-85544d4cd7-hfrbk\" (UID: \"474fc2c2-03ae-420b-a831-c8e98a3e5ed8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-hfrbk"
Oct 30 22:41:40.983206 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:40.983174    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/474fc2c2-03ae-420b-a831-c8e98a3e5ed8-credential-socket\") pod \"maas-default-gateway-openshift-default-85544d4cd7-hfrbk\" (UID: \"474fc2c2-03ae-420b-a831-c8e98a3e5ed8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-hfrbk"
Oct 30 22:41:40.983255 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:40.983204    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/474fc2c2-03ae-420b-a831-c8e98a3e5ed8-workload-certs\") pod \"maas-default-gateway-openshift-default-85544d4cd7-hfrbk\" (UID: \"474fc2c2-03ae-420b-a831-c8e98a3e5ed8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-hfrbk"
Oct 30 22:41:40.983255 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:40.983227    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/474fc2c2-03ae-420b-a831-c8e98a3e5ed8-istio-token\") pod \"maas-default-gateway-openshift-default-85544d4cd7-hfrbk\" (UID: \"474fc2c2-03ae-420b-a831-c8e98a3e5ed8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-hfrbk"
Oct 30 22:41:41.048302 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:41.048268    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-llmh8"]
Oct 30 22:41:41.084747 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:41.084714    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nw7pv\" (UniqueName: \"kubernetes.io/projected/1123817a-b255-47e2-9188-91650a8b4378-kube-api-access-nw7pv\") pod \"maas-default-gateway-openshift-default-85544d4cd7-llmh8\" (UID: \"1123817a-b255-47e2-9188-91650a8b4378\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-llmh8"
Oct 30 22:41:41.084909 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:41.084755    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/474fc2c2-03ae-420b-a831-c8e98a3e5ed8-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-85544d4cd7-hfrbk\" (UID: \"474fc2c2-03ae-420b-a831-c8e98a3e5ed8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-hfrbk"
Oct 30 22:41:41.084909 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:41.084851    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1123817a-b255-47e2-9188-91650a8b4378-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-85544d4cd7-llmh8\" (UID: \"1123817a-b255-47e2-9188-91650a8b4378\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-llmh8"
Oct 30 22:41:41.084909 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:41.084885    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1123817a-b255-47e2-9188-91650a8b4378-istio-envoy\") pod \"maas-default-gateway-openshift-default-85544d4cd7-llmh8\" (UID: \"1123817a-b255-47e2-9188-91650a8b4378\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-llmh8"
Oct 30 22:41:41.085091 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:41.084913    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/474fc2c2-03ae-420b-a831-c8e98a3e5ed8-istio-podinfo\") pod \"maas-default-gateway-openshift-default-85544d4cd7-hfrbk\" (UID: \"474fc2c2-03ae-420b-a831-c8e98a3e5ed8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-hfrbk"
Oct 30 22:41:41.085091 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:41.084937    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1123817a-b255-47e2-9188-91650a8b4378-workload-socket\") pod \"maas-default-gateway-openshift-default-85544d4cd7-llmh8\" (UID: \"1123817a-b255-47e2-9188-91650a8b4378\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-llmh8"
Oct 30 22:41:41.085091 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:41.085026    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1123817a-b255-47e2-9188-91650a8b4378-istio-podinfo\") pod \"maas-default-gateway-openshift-default-85544d4cd7-llmh8\" (UID: \"1123817a-b255-47e2-9188-91650a8b4378\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-llmh8"
Oct 30 22:41:41.085317 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:41.085131    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/474fc2c2-03ae-420b-a831-c8e98a3e5ed8-istio-envoy\") pod \"maas-default-gateway-openshift-default-85544d4cd7-hfrbk\" (UID: \"474fc2c2-03ae-420b-a831-c8e98a3e5ed8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-hfrbk"
Oct 30 22:41:41.085317 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:41.085158    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1123817a-b255-47e2-9188-91650a8b4378-istio-data\") pod \"maas-default-gateway-openshift-default-85544d4cd7-llmh8\" (UID: \"1123817a-b255-47e2-9188-91650a8b4378\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-llmh8"
Oct 30 22:41:41.085317 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:41.085183    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/474fc2c2-03ae-420b-a831-c8e98a3e5ed8-credential-socket\") pod \"maas-default-gateway-openshift-default-85544d4cd7-hfrbk\" (UID: \"474fc2c2-03ae-420b-a831-c8e98a3e5ed8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-hfrbk"
Oct 30 22:41:41.085493 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:41.085319    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1123817a-b255-47e2-9188-91650a8b4378-workload-certs\") pod \"maas-default-gateway-openshift-default-85544d4cd7-llmh8\" (UID: \"1123817a-b255-47e2-9188-91650a8b4378\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-llmh8"
Oct 30 22:41:41.085493 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:41.085355    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/474fc2c2-03ae-420b-a831-c8e98a3e5ed8-workload-certs\") pod \"maas-default-gateway-openshift-default-85544d4cd7-hfrbk\" (UID: \"474fc2c2-03ae-420b-a831-c8e98a3e5ed8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-hfrbk"
Oct 30 22:41:41.085493 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:41.085385    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/474fc2c2-03ae-420b-a831-c8e98a3e5ed8-istio-token\") pod \"maas-default-gateway-openshift-default-85544d4cd7-hfrbk\" (UID: \"474fc2c2-03ae-420b-a831-c8e98a3e5ed8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-hfrbk"
Oct 30 22:41:41.085493 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:41.085482    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1123817a-b255-47e2-9188-91650a8b4378-credential-socket\") pod \"maas-default-gateway-openshift-default-85544d4cd7-llmh8\" (UID: \"1123817a-b255-47e2-9188-91650a8b4378\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-llmh8"
Oct 30 22:41:41.085729 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:41.085525    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/474fc2c2-03ae-420b-a831-c8e98a3e5ed8-credential-socket\") pod \"maas-default-gateway-openshift-default-85544d4cd7-hfrbk\" (UID: \"474fc2c2-03ae-420b-a831-c8e98a3e5ed8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-hfrbk"
Oct 30 22:41:41.085729 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:41.085564    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/474fc2c2-03ae-420b-a831-c8e98a3e5ed8-istio-data\") pod \"maas-default-gateway-openshift-default-85544d4cd7-hfrbk\" (UID: \"474fc2c2-03ae-420b-a831-c8e98a3e5ed8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-hfrbk"
Oct 30 22:41:41.085729 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:41.085585    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/474fc2c2-03ae-420b-a831-c8e98a3e5ed8-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-85544d4cd7-hfrbk\" (UID: \"474fc2c2-03ae-420b-a831-c8e98a3e5ed8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-hfrbk"
Oct 30 22:41:41.085729 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:41.085597    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1123817a-b255-47e2-9188-91650a8b4378-istio-token\") pod \"maas-default-gateway-openshift-default-85544d4cd7-llmh8\" (UID: \"1123817a-b255-47e2-9188-91650a8b4378\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-llmh8"
Oct 30 22:41:41.085729 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:41.085691    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/474fc2c2-03ae-420b-a831-c8e98a3e5ed8-workload-socket\") pod \"maas-default-gateway-openshift-default-85544d4cd7-hfrbk\" (UID: \"474fc2c2-03ae-420b-a831-c8e98a3e5ed8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-hfrbk"
Oct 30 22:41:41.085918 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:41.085748    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/474fc2c2-03ae-420b-a831-c8e98a3e5ed8-workload-certs\") pod \"maas-default-gateway-openshift-default-85544d4cd7-hfrbk\" (UID: \"474fc2c2-03ae-420b-a831-c8e98a3e5ed8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-hfrbk"
Oct 30 22:41:41.085918 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:41.085754    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sjcnz\" (UniqueName: \"kubernetes.io/projected/474fc2c2-03ae-420b-a831-c8e98a3e5ed8-kube-api-access-sjcnz\") pod \"maas-default-gateway-openshift-default-85544d4cd7-hfrbk\" (UID: \"474fc2c2-03ae-420b-a831-c8e98a3e5ed8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-hfrbk"
Oct 30 22:41:41.085918 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:41.085825    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/474fc2c2-03ae-420b-a831-c8e98a3e5ed8-istio-data\") pod \"maas-default-gateway-openshift-default-85544d4cd7-hfrbk\" (UID: \"474fc2c2-03ae-420b-a831-c8e98a3e5ed8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-hfrbk"
Oct 30 22:41:41.086017 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:41.085964    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/474fc2c2-03ae-420b-a831-c8e98a3e5ed8-workload-socket\") pod \"maas-default-gateway-openshift-default-85544d4cd7-hfrbk\" (UID: \"474fc2c2-03ae-420b-a831-c8e98a3e5ed8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-hfrbk"
Oct 30 22:41:41.087359 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:41.087331    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/474fc2c2-03ae-420b-a831-c8e98a3e5ed8-istio-envoy\") pod \"maas-default-gateway-openshift-default-85544d4cd7-hfrbk\" (UID: \"474fc2c2-03ae-420b-a831-c8e98a3e5ed8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-hfrbk"
Oct 30 22:41:41.087481 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:41.087463    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/474fc2c2-03ae-420b-a831-c8e98a3e5ed8-istio-podinfo\") pod \"maas-default-gateway-openshift-default-85544d4cd7-hfrbk\" (UID: \"474fc2c2-03ae-420b-a831-c8e98a3e5ed8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-hfrbk"
Oct 30 22:41:41.144481 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:41.144408    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/474fc2c2-03ae-420b-a831-c8e98a3e5ed8-istio-token\") pod \"maas-default-gateway-openshift-default-85544d4cd7-hfrbk\" (UID: \"474fc2c2-03ae-420b-a831-c8e98a3e5ed8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-hfrbk"
Oct 30 22:41:41.144481 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:41.144433    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sjcnz\" (UniqueName: \"kubernetes.io/projected/474fc2c2-03ae-420b-a831-c8e98a3e5ed8-kube-api-access-sjcnz\") pod \"maas-default-gateway-openshift-default-85544d4cd7-hfrbk\" (UID: \"474fc2c2-03ae-420b-a831-c8e98a3e5ed8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-hfrbk"
Oct 30 22:41:41.186566 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:41.186536    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1123817a-b255-47e2-9188-91650a8b4378-workload-certs\") pod \"maas-default-gateway-openshift-default-85544d4cd7-llmh8\" (UID: \"1123817a-b255-47e2-9188-91650a8b4378\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-llmh8"
Oct 30 22:41:41.186746 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:41.186575    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1123817a-b255-47e2-9188-91650a8b4378-credential-socket\") pod \"maas-default-gateway-openshift-default-85544d4cd7-llmh8\" (UID: \"1123817a-b255-47e2-9188-91650a8b4378\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-llmh8"
Oct 30 22:41:41.186746 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:41.186711    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1123817a-b255-47e2-9188-91650a8b4378-istio-token\") pod \"maas-default-gateway-openshift-default-85544d4cd7-llmh8\" (UID: \"1123817a-b255-47e2-9188-91650a8b4378\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-llmh8"
Oct 30 22:41:41.186839 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:41.186804    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nw7pv\" (UniqueName: \"kubernetes.io/projected/1123817a-b255-47e2-9188-91650a8b4378-kube-api-access-nw7pv\") pod \"maas-default-gateway-openshift-default-85544d4cd7-llmh8\" (UID: \"1123817a-b255-47e2-9188-91650a8b4378\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-llmh8"
Oct 30 22:41:41.186892 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:41.186874    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1123817a-b255-47e2-9188-91650a8b4378-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-85544d4cd7-llmh8\" (UID: \"1123817a-b255-47e2-9188-91650a8b4378\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-llmh8"
Oct 30 22:41:41.186953 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:41.186903    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1123817a-b255-47e2-9188-91650a8b4378-istio-envoy\") pod \"maas-default-gateway-openshift-default-85544d4cd7-llmh8\" (UID: \"1123817a-b255-47e2-9188-91650a8b4378\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-llmh8"
Oct 30 22:41:41.186953 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:41.186910    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1123817a-b255-47e2-9188-91650a8b4378-credential-socket\") pod \"maas-default-gateway-openshift-default-85544d4cd7-llmh8\" (UID: \"1123817a-b255-47e2-9188-91650a8b4378\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-llmh8"
Oct 30 22:41:41.186953 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:41.186941    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1123817a-b255-47e2-9188-91650a8b4378-workload-socket\") pod \"maas-default-gateway-openshift-default-85544d4cd7-llmh8\" (UID: \"1123817a-b255-47e2-9188-91650a8b4378\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-llmh8"
Oct 30 22:41:41.187109 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:41.186967    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1123817a-b255-47e2-9188-91650a8b4378-workload-certs\") pod \"maas-default-gateway-openshift-default-85544d4cd7-llmh8\" (UID: \"1123817a-b255-47e2-9188-91650a8b4378\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-llmh8"
Oct 30 22:41:41.187109 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:41.187014    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1123817a-b255-47e2-9188-91650a8b4378-istio-podinfo\") pod \"maas-default-gateway-openshift-default-85544d4cd7-llmh8\" (UID: \"1123817a-b255-47e2-9188-91650a8b4378\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-llmh8"
Oct 30 22:41:41.187109 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:41.187057    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1123817a-b255-47e2-9188-91650a8b4378-istio-data\") pod \"maas-default-gateway-openshift-default-85544d4cd7-llmh8\" (UID: \"1123817a-b255-47e2-9188-91650a8b4378\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-llmh8"
Oct 30 22:41:41.187243 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:41.187192    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1123817a-b255-47e2-9188-91650a8b4378-workload-socket\") pod \"maas-default-gateway-openshift-default-85544d4cd7-llmh8\" (UID: \"1123817a-b255-47e2-9188-91650a8b4378\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-llmh8"
Oct 30 22:41:41.187370 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:41.187351    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1123817a-b255-47e2-9188-91650a8b4378-istio-data\") pod \"maas-default-gateway-openshift-default-85544d4cd7-llmh8\" (UID: \"1123817a-b255-47e2-9188-91650a8b4378\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-llmh8"
Oct 30 22:41:41.187510 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:41.187492    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1123817a-b255-47e2-9188-91650a8b4378-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-85544d4cd7-llmh8\" (UID: \"1123817a-b255-47e2-9188-91650a8b4378\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-llmh8"
Oct 30 22:41:41.189026 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:41.189003    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1123817a-b255-47e2-9188-91650a8b4378-istio-envoy\") pod \"maas-default-gateway-openshift-default-85544d4cd7-llmh8\" (UID: \"1123817a-b255-47e2-9188-91650a8b4378\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-llmh8"
Oct 30 22:41:41.189240 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:41.189223    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1123817a-b255-47e2-9188-91650a8b4378-istio-podinfo\") pod \"maas-default-gateway-openshift-default-85544d4cd7-llmh8\" (UID: \"1123817a-b255-47e2-9188-91650a8b4378\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-llmh8"
Oct 30 22:41:41.199118 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:41.199099    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-hfrbk"
Oct 30 22:41:41.245322 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:41.245254    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1123817a-b255-47e2-9188-91650a8b4378-istio-token\") pod \"maas-default-gateway-openshift-default-85544d4cd7-llmh8\" (UID: \"1123817a-b255-47e2-9188-91650a8b4378\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-llmh8"
Oct 30 22:41:41.245322 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:41.245291    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nw7pv\" (UniqueName: \"kubernetes.io/projected/1123817a-b255-47e2-9188-91650a8b4378-kube-api-access-nw7pv\") pod \"maas-default-gateway-openshift-default-85544d4cd7-llmh8\" (UID: \"1123817a-b255-47e2-9188-91650a8b4378\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-llmh8"
Oct 30 22:41:41.544516 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:41.544479    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-llmh8"
Oct 30 22:41:41.614252 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:41.614230    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-hfrbk"]
Oct 30 22:41:41.617250 ip-10-0-136-210 kubenswrapper[5510]: W1030 22:41:41.617223    5510 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod474fc2c2_03ae_420b_a831_c8e98a3e5ed8.slice/crio-63a8a58dada5458a22aeccf29e7a6686edd0a4fb1e3063211b41bb9c07338b2a WatchSource:0}: Error finding container 63a8a58dada5458a22aeccf29e7a6686edd0a4fb1e3063211b41bb9c07338b2a: Status 404 returned error can't find the container with id 63a8a58dada5458a22aeccf29e7a6686edd0a4fb1e3063211b41bb9c07338b2a
Oct 30 22:41:41.619623 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:41.619577    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:41:41.619738 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:41.619703    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:41:41.619815 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:41.619752    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:41:41.666291 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:41.666265    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-hfrbk" event={"ID":"474fc2c2-03ae-420b-a831-c8e98a3e5ed8","Type":"ContainerStarted","Data":"63a8a58dada5458a22aeccf29e7a6686edd0a4fb1e3063211b41bb9c07338b2a"}
Oct 30 22:41:41.916837 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:41.916811    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-llmh8"]
Oct 30 22:41:41.920220 ip-10-0-136-210 kubenswrapper[5510]: W1030 22:41:41.920190    5510 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1123817a_b255_47e2_9188_91650a8b4378.slice/crio-30d0f78c711e502a68ca4635d63b433e0031c76ab6127eb1eef886145322cb99 WatchSource:0}: Error finding container 30d0f78c711e502a68ca4635d63b433e0031c76ab6127eb1eef886145322cb99: Status 404 returned error can't find the container with id 30d0f78c711e502a68ca4635d63b433e0031c76ab6127eb1eef886145322cb99
Oct 30 22:41:41.922460 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:41.922424    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:41:41.922547 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:41.922492    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:41:41.922547 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:41.922520    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:41:42.695773 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:42.695735    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-llmh8" event={"ID":"1123817a-b255-47e2-9188-91650a8b4378","Type":"ContainerStarted","Data":"d5a748623b076d55c2f272ed972c8446748754f666d6206ac80e8e302d74ab9a"}
Oct 30 22:41:42.696018 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:42.696000    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-llmh8" event={"ID":"1123817a-b255-47e2-9188-91650a8b4378","Type":"ContainerStarted","Data":"30d0f78c711e502a68ca4635d63b433e0031c76ab6127eb1eef886145322cb99"}
Oct 30 22:41:42.697465 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:42.697441    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-hfrbk" event={"ID":"474fc2c2-03ae-420b-a831-c8e98a3e5ed8","Type":"ContainerStarted","Data":"65734bdbcffa99bcfb135460f0eea2abb029605d61261c8c304ce4f5a1fe4c00"}
Oct 30 22:41:42.888690 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:42.888594    5510 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-llmh8" podStartSLOduration=2.888568246 podStartE2EDuration="2.888568246s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:41:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:41:42.866437424 +0000 UTC m=+3083.427247445" watchObservedRunningTime="2026-10-30 22:41:42.888568246 +0000 UTC m=+3083.449378253"
Oct 30 22:41:43.199955 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:43.199907    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-hfrbk"
Oct 30 22:41:43.201420 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:43.201393    5510 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-85544d4cd7-hfrbk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.149:15021/healthz/ready\": dial tcp 10.133.0.149:15021: connect: connection refused" start-of-body=
Oct 30 22:41:43.201572 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:43.201444    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-hfrbk" podUID="474fc2c2-03ae-420b-a831-c8e98a3e5ed8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.149:15021/healthz/ready\": dial tcp 10.133.0.149:15021: connect: connection refused"
Oct 30 22:41:43.558681 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:43.553436    5510 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-85544d4cd7-llmh8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.150:15021/healthz/ready\": dial tcp 10.133.0.150:15021: connect: connection refused" start-of-body=
Oct 30 22:41:43.558681 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:43.553499    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-llmh8" podUID="1123817a-b255-47e2-9188-91650a8b4378" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.150:15021/healthz/ready\": dial tcp 10.133.0.150:15021: connect: connection refused"
Oct 30 22:41:43.578753 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:43.578714    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-llmh8"
Oct 30 22:41:44.200237 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:44.200185    5510 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-85544d4cd7-hfrbk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.149:15021/healthz/ready\": dial tcp 10.133.0.149:15021: connect: connection refused" start-of-body=
Oct 30 22:41:44.200679 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:44.200261    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-hfrbk" podUID="474fc2c2-03ae-420b-a831-c8e98a3e5ed8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.149:15021/healthz/ready\": dial tcp 10.133.0.149:15021: connect: connection refused"
Oct 30 22:41:44.549682 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:44.548786    5510 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-85544d4cd7-llmh8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.150:15021/healthz/ready\": dial tcp 10.133.0.150:15021: connect: connection refused" start-of-body=
Oct 30 22:41:44.549682 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:44.548954    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-llmh8" podUID="1123817a-b255-47e2-9188-91650a8b4378" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.150:15021/healthz/ready\": dial tcp 10.133.0.150:15021: connect: connection refused"
Oct 30 22:41:44.895342 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:44.895261    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:41:44.982681 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:44.981778    5510 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-hfrbk" podStartSLOduration=4.981756349 podStartE2EDuration="4.981756349s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:41:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:41:42.993627514 +0000 UTC m=+3083.554437540" watchObservedRunningTime="2026-10-30 22:41:44.981756349 +0000 UTC m=+3085.542566355"
Oct 30 22:41:44.992682 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:44.985168    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-869c494b8f-w9w5n"]
Oct 30 22:41:44.992682 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:44.985827    5510 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/maas-controller-869c494b8f-w9w5n" podUID="b6c7b840-e876-4170-88d1-77e9374dc9aa" containerName="manager" containerID="cri-o://8ea5426517863ca5ca39797740e244681f452cf7cb8d0bb038ff26e3e77fe91d" gracePeriod=30
Oct 30 22:41:45.199993 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:45.199772    5510 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-85544d4cd7-hfrbk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.149:15021/healthz/ready\": dial tcp 10.133.0.149:15021: connect: connection refused" start-of-body=
Oct 30 22:41:45.199993 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:45.199839    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-hfrbk" podUID="474fc2c2-03ae-420b-a831-c8e98a3e5ed8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.149:15021/healthz/ready\": dial tcp 10.133.0.149:15021: connect: connection refused"
Oct 30 22:41:45.323251 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:45.323223    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-869c494b8f-w9w5n"
Oct 30 22:41:45.426848 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:45.426808    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b6c7b840-e876-4170-88d1-77e9374dc9aa-metrics-certs\" (UniqueName: \"kubernetes.io/secret/b6c7b840-e876-4170-88d1-77e9374dc9aa-metrics-certs\") pod \"b6c7b840-e876-4170-88d1-77e9374dc9aa\" (UID: \"b6c7b840-e876-4170-88d1-77e9374dc9aa\") "
Oct 30 22:41:45.427011 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:45.426876    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b6c7b840-e876-4170-88d1-77e9374dc9aa-webhook-cert\" (UniqueName: \"kubernetes.io/secret/b6c7b840-e876-4170-88d1-77e9374dc9aa-webhook-cert\") pod \"b6c7b840-e876-4170-88d1-77e9374dc9aa\" (UID: \"b6c7b840-e876-4170-88d1-77e9374dc9aa\") "
Oct 30 22:41:45.427011 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:45.426941    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b6c7b840-e876-4170-88d1-77e9374dc9aa-kube-api-access-nhk7z\" (UniqueName: \"kubernetes.io/projected/b6c7b840-e876-4170-88d1-77e9374dc9aa-kube-api-access-nhk7z\") pod \"b6c7b840-e876-4170-88d1-77e9374dc9aa\" (UID: \"b6c7b840-e876-4170-88d1-77e9374dc9aa\") "
Oct 30 22:41:45.428856 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:45.428821    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b6c7b840-e876-4170-88d1-77e9374dc9aa-metrics-certs" pod "b6c7b840-e876-4170-88d1-77e9374dc9aa" (UID: "b6c7b840-e876-4170-88d1-77e9374dc9aa"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:41:45.429190 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:45.429169    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b6c7b840-e876-4170-88d1-77e9374dc9aa-kube-api-access-nhk7z" pod "b6c7b840-e876-4170-88d1-77e9374dc9aa" (UID: "b6c7b840-e876-4170-88d1-77e9374dc9aa"). InnerVolumeSpecName "kube-api-access-nhk7z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:41:45.429768 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:45.429751    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b6c7b840-e876-4170-88d1-77e9374dc9aa-webhook-cert" pod "b6c7b840-e876-4170-88d1-77e9374dc9aa" (UID: "b6c7b840-e876-4170-88d1-77e9374dc9aa"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:41:45.528496 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:45.528466    5510 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b6c7b840-e876-4170-88d1-77e9374dc9aa-metrics-certs\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:41:45.528496 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:45.528496    5510 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b6c7b840-e876-4170-88d1-77e9374dc9aa-webhook-cert\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:41:45.528716 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:45.528506    5510 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nhk7z\" (UniqueName: \"kubernetes.io/projected/b6c7b840-e876-4170-88d1-77e9374dc9aa-kube-api-access-nhk7z\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:41:45.548152 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:45.548112    5510 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-85544d4cd7-llmh8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.150:15021/healthz/ready\": dial tcp 10.133.0.150:15021: connect: connection refused" start-of-body=
Oct 30 22:41:45.576845 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:45.576796    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-llmh8" podUID="1123817a-b255-47e2-9188-91650a8b4378" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.150:15021/healthz/ready\": dial tcp 10.133.0.150:15021: connect: connection refused"
Oct 30 22:41:45.721422 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:45.721386    5510 generic.go:356] "Generic (PLEG): container finished" podID="b6c7b840-e876-4170-88d1-77e9374dc9aa" containerID="8ea5426517863ca5ca39797740e244681f452cf7cb8d0bb038ff26e3e77fe91d" exitCode=0
Oct 30 22:41:45.721634 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:45.721438    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-869c494b8f-w9w5n" event={"ID":"b6c7b840-e876-4170-88d1-77e9374dc9aa","Type":"ContainerDied","Data":"8ea5426517863ca5ca39797740e244681f452cf7cb8d0bb038ff26e3e77fe91d"}
Oct 30 22:41:45.721634 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:45.721439    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-869c494b8f-w9w5n"
Oct 30 22:41:45.721634 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:45.721491    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-869c494b8f-w9w5n" event={"ID":"b6c7b840-e876-4170-88d1-77e9374dc9aa","Type":"ContainerDied","Data":"5090be2e58a83993b46f031b95f5fccdd9ec4a89d6c4bc618f25dc185781a854"}
Oct 30 22:41:45.721634 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:45.721519    5510 scope.go:122] "RemoveContainer" containerID="8ea5426517863ca5ca39797740e244681f452cf7cb8d0bb038ff26e3e77fe91d"
Oct 30 22:41:45.729939 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:45.729920    5510 scope.go:122] "RemoveContainer" containerID="8ea5426517863ca5ca39797740e244681f452cf7cb8d0bb038ff26e3e77fe91d"
Oct 30 22:41:45.730230 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:41:45.730212    5510 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8ea5426517863ca5ca39797740e244681f452cf7cb8d0bb038ff26e3e77fe91d\": container with ID starting with 8ea5426517863ca5ca39797740e244681f452cf7cb8d0bb038ff26e3e77fe91d not found: ID does not exist" containerID="8ea5426517863ca5ca39797740e244681f452cf7cb8d0bb038ff26e3e77fe91d"
Oct 30 22:41:45.730299 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:45.730235    5510 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8ea5426517863ca5ca39797740e244681f452cf7cb8d0bb038ff26e3e77fe91d"} err="failed to get container status \"8ea5426517863ca5ca39797740e244681f452cf7cb8d0bb038ff26e3e77fe91d\": rpc error: code = NotFound desc = could not find container \"8ea5426517863ca5ca39797740e244681f452cf7cb8d0bb038ff26e3e77fe91d\": container with ID starting with 8ea5426517863ca5ca39797740e244681f452cf7cb8d0bb038ff26e3e77fe91d not found: ID does not exist"
Oct 30 22:41:45.880739 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:45.880374    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-869c494b8f-w9w5n"]
Oct 30 22:41:45.895751 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:45.895704    5510 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-869c494b8f-w9w5n"]
Oct 30 22:41:46.199921 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:46.199831    5510 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-85544d4cd7-hfrbk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.149:15021/healthz/ready\": dial tcp 10.133.0.149:15021: connect: connection refused" start-of-body=
Oct 30 22:41:46.199921 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:46.199879    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-hfrbk" podUID="474fc2c2-03ae-420b-a831-c8e98a3e5ed8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.149:15021/healthz/ready\": dial tcp 10.133.0.149:15021: connect: connection refused"
Oct 30 22:41:46.551683 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:46.548187    5510 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-85544d4cd7-llmh8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.150:15021/healthz/ready\": dial tcp 10.133.0.150:15021: connect: connection refused" start-of-body=
Oct 30 22:41:46.551683 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:46.548253    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-llmh8" podUID="1123817a-b255-47e2-9188-91650a8b4378" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.150:15021/healthz/ready\": dial tcp 10.133.0.150:15021: connect: connection refused"
Oct 30 22:41:47.199856 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:47.199811    5510 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-85544d4cd7-hfrbk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.149:15021/healthz/ready\": dial tcp 10.133.0.149:15021: connect: connection refused" start-of-body=
Oct 30 22:41:47.200055 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:47.199874    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-hfrbk" podUID="474fc2c2-03ae-420b-a831-c8e98a3e5ed8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.149:15021/healthz/ready\": dial tcp 10.133.0.149:15021: connect: connection refused"
Oct 30 22:41:47.306058 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:47.306023    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:41:47.394682 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:47.389294    5510 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-869c494b8f-27qzn"]
Oct 30 22:41:47.394682 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:47.389962    5510 cpu_manager.go:414] "containerMap: removing container" podUID="b6c7b840-e876-4170-88d1-77e9374dc9aa" containerName="manager"
Oct 30 22:41:47.394682 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:47.389983    5510 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b6c7b840-e876-4170-88d1-77e9374dc9aa" containerName="manager"
Oct 30 22:41:47.394682 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:47.390089    5510 memory_manager.go:365] "RemoveStaleState removing state" podUID="b6c7b840-e876-4170-88d1-77e9374dc9aa" containerName="manager"
Oct 30 22:41:47.398679 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:47.398214    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-869c494b8f-27qzn"
Oct 30 22:41:47.464737 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:47.463058    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Oct 30 22:41:47.464737 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:47.463188    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-fzht9\""
Oct 30 22:41:47.465861 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:47.465558    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/41b8b39f-9e91-463b-bc79-330e7d62d699-webhook-cert\") pod \"maas-controller-869c494b8f-27qzn\" (UID: \"41b8b39f-9e91-463b-bc79-330e7d62d699\") " pod="opendatahub/maas-controller-869c494b8f-27qzn"
Oct 30 22:41:47.465861 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:47.465644    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/41b8b39f-9e91-463b-bc79-330e7d62d699-metrics-certs\") pod \"maas-controller-869c494b8f-27qzn\" (UID: \"41b8b39f-9e91-463b-bc79-330e7d62d699\") " pod="opendatahub/maas-controller-869c494b8f-27qzn"
Oct 30 22:41:47.465861 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:47.465713    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fc4f4\" (UniqueName: \"kubernetes.io/projected/41b8b39f-9e91-463b-bc79-330e7d62d699-kube-api-access-fc4f4\") pod \"maas-controller-869c494b8f-27qzn\" (UID: \"41b8b39f-9e91-463b-bc79-330e7d62d699\") " pod="opendatahub/maas-controller-869c494b8f-27qzn"
Oct 30 22:41:47.509549 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:47.508990    5510 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Oct 30 22:41:47.509549 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:47.509224    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Oct 30 22:41:47.510872 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:47.510841    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-869c494b8f-27qzn"]
Oct 30 22:41:47.572693 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:47.571232    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fc4f4\" (UniqueName: \"kubernetes.io/projected/41b8b39f-9e91-463b-bc79-330e7d62d699-kube-api-access-fc4f4\") pod \"maas-controller-869c494b8f-27qzn\" (UID: \"41b8b39f-9e91-463b-bc79-330e7d62d699\") " pod="opendatahub/maas-controller-869c494b8f-27qzn"
Oct 30 22:41:47.572693 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:47.571386    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/41b8b39f-9e91-463b-bc79-330e7d62d699-webhook-cert\") pod \"maas-controller-869c494b8f-27qzn\" (UID: \"41b8b39f-9e91-463b-bc79-330e7d62d699\") " pod="opendatahub/maas-controller-869c494b8f-27qzn"
Oct 30 22:41:47.572693 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:47.571465    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/41b8b39f-9e91-463b-bc79-330e7d62d699-metrics-certs\") pod \"maas-controller-869c494b8f-27qzn\" (UID: \"41b8b39f-9e91-463b-bc79-330e7d62d699\") " pod="opendatahub/maas-controller-869c494b8f-27qzn"
Oct 30 22:41:47.580680 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:47.579094    5510 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-85544d4cd7-llmh8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.150:15021/healthz/ready\": dial tcp 10.133.0.150:15021: connect: connection refused" start-of-body=
Oct 30 22:41:47.580680 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:47.579167    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-llmh8" podUID="1123817a-b255-47e2-9188-91650a8b4378" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.150:15021/healthz/ready\": dial tcp 10.133.0.150:15021: connect: connection refused"
Oct 30 22:41:47.594678 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:47.584264    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/41b8b39f-9e91-463b-bc79-330e7d62d699-metrics-certs\") pod \"maas-controller-869c494b8f-27qzn\" (UID: \"41b8b39f-9e91-463b-bc79-330e7d62d699\") " pod="opendatahub/maas-controller-869c494b8f-27qzn"
Oct 30 22:41:47.597767 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:47.597719    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/41b8b39f-9e91-463b-bc79-330e7d62d699-webhook-cert\") pod \"maas-controller-869c494b8f-27qzn\" (UID: \"41b8b39f-9e91-463b-bc79-330e7d62d699\") " pod="opendatahub/maas-controller-869c494b8f-27qzn"
Oct 30 22:41:47.602594 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:47.602567    5510 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b6c7b840-e876-4170-88d1-77e9374dc9aa" path="/var/lib/kubelet/pods/b6c7b840-e876-4170-88d1-77e9374dc9aa/volumes"
Oct 30 22:41:47.630769 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:47.630736    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fc4f4\" (UniqueName: \"kubernetes.io/projected/41b8b39f-9e91-463b-bc79-330e7d62d699-kube-api-access-fc4f4\") pod \"maas-controller-869c494b8f-27qzn\" (UID: \"41b8b39f-9e91-463b-bc79-330e7d62d699\") " pod="opendatahub/maas-controller-869c494b8f-27qzn"
Oct 30 22:41:47.711284 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:47.711234    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-869c494b8f-27qzn"
Oct 30 22:41:48.106941 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:48.106908    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-869c494b8f-27qzn"]
Oct 30 22:41:48.110368 ip-10-0-136-210 kubenswrapper[5510]: W1030 22:41:48.110326    5510 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41b8b39f_9e91_463b_bc79_330e7d62d699.slice/crio-c40c7f15b5d9312223c41571440cd719d3749aaef954dd6f478ff81a83ae6881 WatchSource:0}: Error finding container c40c7f15b5d9312223c41571440cd719d3749aaef954dd6f478ff81a83ae6881: Status 404 returned error can't find the container with id c40c7f15b5d9312223c41571440cd719d3749aaef954dd6f478ff81a83ae6881
Oct 30 22:41:48.200282 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:48.200239    5510 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-85544d4cd7-hfrbk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.149:15021/healthz/ready\": dial tcp 10.133.0.149:15021: connect: connection refused" start-of-body=
Oct 30 22:41:48.200461 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:48.200313    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-hfrbk" podUID="474fc2c2-03ae-420b-a831-c8e98a3e5ed8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.149:15021/healthz/ready\": dial tcp 10.133.0.149:15021: connect: connection refused"
Oct 30 22:41:48.545415 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:48.545371    5510 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-85544d4cd7-llmh8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.150:15021/healthz/ready\": dial tcp 10.133.0.150:15021: connect: connection refused" start-of-body=
Oct 30 22:41:48.545605 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:48.545445    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-llmh8" podUID="1123817a-b255-47e2-9188-91650a8b4378" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.150:15021/healthz/ready\": dial tcp 10.133.0.150:15021: connect: connection refused"
Oct 30 22:41:48.739419 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:48.739376    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-869c494b8f-27qzn" event={"ID":"41b8b39f-9e91-463b-bc79-330e7d62d699","Type":"ContainerStarted","Data":"c40c7f15b5d9312223c41571440cd719d3749aaef954dd6f478ff81a83ae6881"}
Oct 30 22:41:49.199983 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:49.199933    5510 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-85544d4cd7-hfrbk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.149:15021/healthz/ready\": dial tcp 10.133.0.149:15021: connect: connection refused" start-of-body=
Oct 30 22:41:49.200179 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:49.200009    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-hfrbk" podUID="474fc2c2-03ae-420b-a831-c8e98a3e5ed8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.149:15021/healthz/ready\": dial tcp 10.133.0.149:15021: connect: connection refused"
Oct 30 22:41:49.749114 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:49.749060    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-869c494b8f-27qzn" event={"ID":"41b8b39f-9e91-463b-bc79-330e7d62d699","Type":"ContainerStarted","Data":"cdddb814ece96dcab03d93df6e13bd0fd31090349209f42a0ec5440ac7443ae5"}
Oct 30 22:41:49.749732 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:49.749143    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-869c494b8f-27qzn"
Oct 30 22:41:49.864711 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:49.863220    5510 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/maas-controller-869c494b8f-27qzn" podStartSLOduration=1.919448027 podStartE2EDuration="2.863185368s" totalImagesPullingTime="943.737341ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:41:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:41:49.862134318 +0000 UTC m=+3090.422944337" watchObservedRunningTime="2026-10-30 22:41:49.863185368 +0000 UTC m=+3090.423995374"
Oct 30 22:41:50.204351 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:50.204311    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-hfrbk"
Oct 30 22:41:50.204864 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:50.204836    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-hfrbk"
Oct 30 22:41:50.205849 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:50.205830    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-hfrbk"
Oct 30 22:41:50.357873 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:50.357832    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-llmh8"
Oct 30 22:41:50.358206 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:50.358181    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-llmh8"
Oct 30 22:41:50.359485 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:41:50.359465    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-llmh8"
Oct 30 22:42:00.757522 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:42:00.757484    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-869c494b8f-27qzn"
Oct 30 22:42:04.889016 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:42:04.888979    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:42:08.419957 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:42:08.419913    5510 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6gjbf" containerName="extractor"
Oct 30 22:42:19.184715 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:42:19.184679    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:42:19.382543 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:42:19.382499    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:42:36.191854 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:42:36.191816    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:42:38.418809 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:42:38.418773    5510 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6gjbf" containerName="extractor"
Oct 30 22:42:56.494044 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:42:56.494003    5510 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-q6cg4"]
Oct 30 22:42:56.498359 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:42:56.498331    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-q6cg4"
Oct 30 22:42:56.552379 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:42:56.552336    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/46c98868-6890-4614-93b4-55493087df27-workload-socket\") pod \"maas-default-gateway-openshift-default-85544d4cd7-q6cg4\" (UID: \"46c98868-6890-4614-93b4-55493087df27\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-q6cg4"
Oct 30 22:42:56.552604 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:42:56.552463    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/46c98868-6890-4614-93b4-55493087df27-istio-token\") pod \"maas-default-gateway-openshift-default-85544d4cd7-q6cg4\" (UID: \"46c98868-6890-4614-93b4-55493087df27\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-q6cg4"
Oct 30 22:42:56.552604 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:42:56.552536    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/46c98868-6890-4614-93b4-55493087df27-istio-data\") pod \"maas-default-gateway-openshift-default-85544d4cd7-q6cg4\" (UID: \"46c98868-6890-4614-93b4-55493087df27\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-q6cg4"
Oct 30 22:42:56.552604 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:42:56.552592    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/46c98868-6890-4614-93b4-55493087df27-istio-envoy\") pod \"maas-default-gateway-openshift-default-85544d4cd7-q6cg4\" (UID: \"46c98868-6890-4614-93b4-55493087df27\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-q6cg4"
Oct 30 22:42:56.552997 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:42:56.552622    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/46c98868-6890-4614-93b4-55493087df27-workload-certs\") pod \"maas-default-gateway-openshift-default-85544d4cd7-q6cg4\" (UID: \"46c98868-6890-4614-93b4-55493087df27\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-q6cg4"
Oct 30 22:42:56.552997 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:42:56.552646    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/46c98868-6890-4614-93b4-55493087df27-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-85544d4cd7-q6cg4\" (UID: \"46c98868-6890-4614-93b4-55493087df27\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-q6cg4"
Oct 30 22:42:56.552997 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:42:56.552720    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/46c98868-6890-4614-93b4-55493087df27-istio-podinfo\") pod \"maas-default-gateway-openshift-default-85544d4cd7-q6cg4\" (UID: \"46c98868-6890-4614-93b4-55493087df27\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-q6cg4"
Oct 30 22:42:56.552997 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:42:56.552825    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hh2zg\" (UniqueName: \"kubernetes.io/projected/46c98868-6890-4614-93b4-55493087df27-kube-api-access-hh2zg\") pod \"maas-default-gateway-openshift-default-85544d4cd7-q6cg4\" (UID: \"46c98868-6890-4614-93b4-55493087df27\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-q6cg4"
Oct 30 22:42:56.553799 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:42:56.553774    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/46c98868-6890-4614-93b4-55493087df27-credential-socket\") pod \"maas-default-gateway-openshift-default-85544d4cd7-q6cg4\" (UID: \"46c98868-6890-4614-93b4-55493087df27\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-q6cg4"
Oct 30 22:42:56.654846 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:42:56.654799    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hh2zg\" (UniqueName: \"kubernetes.io/projected/46c98868-6890-4614-93b4-55493087df27-kube-api-access-hh2zg\") pod \"maas-default-gateway-openshift-default-85544d4cd7-q6cg4\" (UID: \"46c98868-6890-4614-93b4-55493087df27\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-q6cg4"
Oct 30 22:42:56.655047 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:42:56.654876    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/46c98868-6890-4614-93b4-55493087df27-credential-socket\") pod \"maas-default-gateway-openshift-default-85544d4cd7-q6cg4\" (UID: \"46c98868-6890-4614-93b4-55493087df27\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-q6cg4"
Oct 30 22:42:56.655047 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:42:56.654917    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/46c98868-6890-4614-93b4-55493087df27-workload-socket\") pod \"maas-default-gateway-openshift-default-85544d4cd7-q6cg4\" (UID: \"46c98868-6890-4614-93b4-55493087df27\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-q6cg4"
Oct 30 22:42:56.655047 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:42:56.654968    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/46c98868-6890-4614-93b4-55493087df27-istio-token\") pod \"maas-default-gateway-openshift-default-85544d4cd7-q6cg4\" (UID: \"46c98868-6890-4614-93b4-55493087df27\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-q6cg4"
Oct 30 22:42:56.655047 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:42:56.655014    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/46c98868-6890-4614-93b4-55493087df27-istio-data\") pod \"maas-default-gateway-openshift-default-85544d4cd7-q6cg4\" (UID: \"46c98868-6890-4614-93b4-55493087df27\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-q6cg4"
Oct 30 22:42:56.655304 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:42:56.655046    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/46c98868-6890-4614-93b4-55493087df27-istio-envoy\") pod \"maas-default-gateway-openshift-default-85544d4cd7-q6cg4\" (UID: \"46c98868-6890-4614-93b4-55493087df27\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-q6cg4"
Oct 30 22:42:56.655304 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:42:56.655074    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/46c98868-6890-4614-93b4-55493087df27-workload-certs\") pod \"maas-default-gateway-openshift-default-85544d4cd7-q6cg4\" (UID: \"46c98868-6890-4614-93b4-55493087df27\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-q6cg4"
Oct 30 22:42:56.655304 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:42:56.655100    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/46c98868-6890-4614-93b4-55493087df27-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-85544d4cd7-q6cg4\" (UID: \"46c98868-6890-4614-93b4-55493087df27\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-q6cg4"
Oct 30 22:42:56.655304 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:42:56.655138    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/46c98868-6890-4614-93b4-55493087df27-istio-podinfo\") pod \"maas-default-gateway-openshift-default-85544d4cd7-q6cg4\" (UID: \"46c98868-6890-4614-93b4-55493087df27\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-q6cg4"
Oct 30 22:42:56.655542 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:42:56.655342    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/46c98868-6890-4614-93b4-55493087df27-credential-socket\") pod \"maas-default-gateway-openshift-default-85544d4cd7-q6cg4\" (UID: \"46c98868-6890-4614-93b4-55493087df27\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-q6cg4"
Oct 30 22:42:56.655542 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:42:56.655427    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/46c98868-6890-4614-93b4-55493087df27-istio-data\") pod \"maas-default-gateway-openshift-default-85544d4cd7-q6cg4\" (UID: \"46c98868-6890-4614-93b4-55493087df27\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-q6cg4"
Oct 30 22:42:56.655706 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:42:56.655585    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/46c98868-6890-4614-93b4-55493087df27-workload-certs\") pod \"maas-default-gateway-openshift-default-85544d4cd7-q6cg4\" (UID: \"46c98868-6890-4614-93b4-55493087df27\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-q6cg4"
Oct 30 22:42:56.655706 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:42:56.655695    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/46c98868-6890-4614-93b4-55493087df27-workload-socket\") pod \"maas-default-gateway-openshift-default-85544d4cd7-q6cg4\" (UID: \"46c98868-6890-4614-93b4-55493087df27\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-q6cg4"
Oct 30 22:42:56.656492 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:42:56.656465    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/46c98868-6890-4614-93b4-55493087df27-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-85544d4cd7-q6cg4\" (UID: \"46c98868-6890-4614-93b4-55493087df27\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-q6cg4"
Oct 30 22:42:56.658330 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:42:56.658287    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/46c98868-6890-4614-93b4-55493087df27-istio-envoy\") pod \"maas-default-gateway-openshift-default-85544d4cd7-q6cg4\" (UID: \"46c98868-6890-4614-93b4-55493087df27\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-q6cg4"
Oct 30 22:42:56.658448 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:42:56.658382    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/46c98868-6890-4614-93b4-55493087df27-istio-podinfo\") pod \"maas-default-gateway-openshift-default-85544d4cd7-q6cg4\" (UID: \"46c98868-6890-4614-93b4-55493087df27\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-q6cg4"
Oct 30 22:42:56.691618 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:42:56.691551    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-q6cg4"]
Oct 30 22:42:56.713255 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:42:56.713220    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hh2zg\" (UniqueName: \"kubernetes.io/projected/46c98868-6890-4614-93b4-55493087df27-kube-api-access-hh2zg\") pod \"maas-default-gateway-openshift-default-85544d4cd7-q6cg4\" (UID: \"46c98868-6890-4614-93b4-55493087df27\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-q6cg4"
Oct 30 22:42:56.713949 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:42:56.713923    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/46c98868-6890-4614-93b4-55493087df27-istio-token\") pod \"maas-default-gateway-openshift-default-85544d4cd7-q6cg4\" (UID: \"46c98868-6890-4614-93b4-55493087df27\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-q6cg4"
Oct 30 22:42:56.815257 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:42:56.815210    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-q6cg4"
Oct 30 22:42:57.237251 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:42:57.237223    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-q6cg4"]
Oct 30 22:42:57.239965 ip-10-0-136-210 kubenswrapper[5510]: W1030 22:42:57.239927    5510 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46c98868_6890_4614_93b4_55493087df27.slice/crio-2e6fc34bf92052490fcec486c9c32dcd4535b287455a2caf8231e18ead1b13ff WatchSource:0}: Error finding container 2e6fc34bf92052490fcec486c9c32dcd4535b287455a2caf8231e18ead1b13ff: Status 404 returned error can't find the container with id 2e6fc34bf92052490fcec486c9c32dcd4535b287455a2caf8231e18ead1b13ff
Oct 30 22:42:57.242082 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:42:57.242047    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:42:57.242193 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:42:57.242114    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:42:57.242193 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:42:57.242142    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:42:58.072420 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:42:58.072387    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-q6cg4" event={"ID":"46c98868-6890-4614-93b4-55493087df27","Type":"ContainerStarted","Data":"bdc7c324b153dd091b490a50fbc9b6881afe0e51e03456333626ca783442f0a2"}
Oct 30 22:42:58.072890 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:42:58.072437    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-q6cg4" event={"ID":"46c98868-6890-4614-93b4-55493087df27","Type":"ContainerStarted","Data":"2e6fc34bf92052490fcec486c9c32dcd4535b287455a2caf8231e18ead1b13ff"}
Oct 30 22:42:58.189203 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:42:58.189149    5510 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-q6cg4" podStartSLOduration=2.18913419 podStartE2EDuration="2.18913419s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:42:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:42:58.188088247 +0000 UTC m=+3158.748898252" watchObservedRunningTime="2026-10-30 22:42:58.18913419 +0000 UTC m=+3158.749944194"
Oct 30 22:42:58.815835 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:42:58.815796    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-q6cg4"
Oct 30 22:42:58.817370 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:42:58.817341    5510 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-85544d4cd7-q6cg4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.152:15021/healthz/ready\": dial tcp 10.133.0.152:15021: connect: connection refused" start-of-body=
Oct 30 22:42:58.817504 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:42:58.817396    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-q6cg4" podUID="46c98868-6890-4614-93b4-55493087df27" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.152:15021/healthz/ready\": dial tcp 10.133.0.152:15021: connect: connection refused"
Oct 30 22:42:59.815884 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:42:59.815845    5510 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-85544d4cd7-q6cg4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.152:15021/healthz/ready\": dial tcp 10.133.0.152:15021: connect: connection refused" start-of-body=
Oct 30 22:42:59.816313 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:42:59.815914    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-q6cg4" podUID="46c98868-6890-4614-93b4-55493087df27" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.152:15021/healthz/ready\": dial tcp 10.133.0.152:15021: connect: connection refused"
Oct 30 22:43:00.816452 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:43:00.816415    5510 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-85544d4cd7-q6cg4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.152:15021/healthz/ready\": dial tcp 10.133.0.152:15021: connect: connection refused" start-of-body=
Oct 30 22:43:00.816894 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:43:00.816482    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-q6cg4" podUID="46c98868-6890-4614-93b4-55493087df27" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.152:15021/healthz/ready\": dial tcp 10.133.0.152:15021: connect: connection refused"
Oct 30 22:43:01.816625 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:43:01.816582    5510 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-85544d4cd7-q6cg4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.152:15021/healthz/ready\": dial tcp 10.133.0.152:15021: connect: connection refused" start-of-body=
Oct 30 22:43:01.817294 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:43:01.817259    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-q6cg4" podUID="46c98868-6890-4614-93b4-55493087df27" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.152:15021/healthz/ready\": dial tcp 10.133.0.152:15021: connect: connection refused"
Oct 30 22:43:02.819402 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:43:02.819366    5510 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-85544d4cd7-q6cg4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.152:15021/healthz/ready\": dial tcp 10.133.0.152:15021: connect: connection refused" start-of-body=
Oct 30 22:43:02.820109 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:43:02.820076    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-q6cg4" podUID="46c98868-6890-4614-93b4-55493087df27" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.152:15021/healthz/ready\": dial tcp 10.133.0.152:15021: connect: connection refused"
Oct 30 22:43:03.815805 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:43:03.815769    5510 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-85544d4cd7-q6cg4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.152:15021/healthz/ready\": dial tcp 10.133.0.152:15021: connect: connection refused" start-of-body=
Oct 30 22:43:03.815988 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:43:03.815845    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-q6cg4" podUID="46c98868-6890-4614-93b4-55493087df27" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.152:15021/healthz/ready\": dial tcp 10.133.0.152:15021: connect: connection refused"
Oct 30 22:43:05.817164 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:43:05.817116    5510 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-85544d4cd7-q6cg4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.152:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 22:43:05.817618 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:43:05.817181    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-q6cg4" podUID="46c98868-6890-4614-93b4-55493087df27" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.152:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:43:06.858856 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:43:06.858817    5510 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-85544d4cd7-q6cg4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.152:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 22:43:06.859415 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:43:06.858876    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-q6cg4" podUID="46c98868-6890-4614-93b4-55493087df27" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.152:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:43:07.496577 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:43:07.496548    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-q6cg4"
Oct 30 22:43:07.496845 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:43:07.496825    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-q6cg4"
Oct 30 22:43:07.497474 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:43:07.497459    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-q6cg4"
Oct 30 22:43:08.419638 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:43:08.419602    5510 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6gjbf" containerName="extractor"
Oct 30 22:43:26.740556 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:43:26.740522    5510 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-m5nkk"]
Oct 30 22:43:26.744143 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:43:26.744113    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-m5nkk"
Oct 30 22:43:26.841645 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:43:26.841611    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-m5nkk"]
Oct 30 22:43:26.867174 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:43:26.867124    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6fb59fa4-0f8d-4111-87e3-e0a2e29982e4-istio-data\") pod \"maas-default-gateway-openshift-default-85544d4cd7-m5nkk\" (UID: \"6fb59fa4-0f8d-4111-87e3-e0a2e29982e4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-m5nkk"
Oct 30 22:43:26.867349 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:43:26.867197    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6fb59fa4-0f8d-4111-87e3-e0a2e29982e4-workload-certs\") pod \"maas-default-gateway-openshift-default-85544d4cd7-m5nkk\" (UID: \"6fb59fa4-0f8d-4111-87e3-e0a2e29982e4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-m5nkk"
Oct 30 22:43:26.867393 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:43:26.867337    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6fb59fa4-0f8d-4111-87e3-e0a2e29982e4-istio-envoy\") pod \"maas-default-gateway-openshift-default-85544d4cd7-m5nkk\" (UID: \"6fb59fa4-0f8d-4111-87e3-e0a2e29982e4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-m5nkk"
Oct 30 22:43:26.867429 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:43:26.867396    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6fb59fa4-0f8d-4111-87e3-e0a2e29982e4-credential-socket\") pod \"maas-default-gateway-openshift-default-85544d4cd7-m5nkk\" (UID: \"6fb59fa4-0f8d-4111-87e3-e0a2e29982e4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-m5nkk"
Oct 30 22:43:26.867499 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:43:26.867472    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vckfb\" (UniqueName: \"kubernetes.io/projected/6fb59fa4-0f8d-4111-87e3-e0a2e29982e4-kube-api-access-vckfb\") pod \"maas-default-gateway-openshift-default-85544d4cd7-m5nkk\" (UID: \"6fb59fa4-0f8d-4111-87e3-e0a2e29982e4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-m5nkk"
Oct 30 22:43:26.867616 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:43:26.867595    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6fb59fa4-0f8d-4111-87e3-e0a2e29982e4-workload-socket\") pod \"maas-default-gateway-openshift-default-85544d4cd7-m5nkk\" (UID: \"6fb59fa4-0f8d-4111-87e3-e0a2e29982e4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-m5nkk"
Oct 30 22:43:26.867691 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:43:26.867640    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6fb59fa4-0f8d-4111-87e3-e0a2e29982e4-istio-podinfo\") pod \"maas-default-gateway-openshift-default-85544d4cd7-m5nkk\" (UID: \"6fb59fa4-0f8d-4111-87e3-e0a2e29982e4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-m5nkk"
Oct 30 22:43:26.867730 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:43:26.867707    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6fb59fa4-0f8d-4111-87e3-e0a2e29982e4-istio-token\") pod \"maas-default-gateway-openshift-default-85544d4cd7-m5nkk\" (UID: \"6fb59fa4-0f8d-4111-87e3-e0a2e29982e4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-m5nkk"
Oct 30 22:43:26.867880 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:43:26.867856    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6fb59fa4-0f8d-4111-87e3-e0a2e29982e4-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-85544d4cd7-m5nkk\" (UID: \"6fb59fa4-0f8d-4111-87e3-e0a2e29982e4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-m5nkk"
Oct 30 22:43:26.968578 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:43:26.968535    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6fb59fa4-0f8d-4111-87e3-e0a2e29982e4-credential-socket\") pod \"maas-default-gateway-openshift-default-85544d4cd7-m5nkk\" (UID: \"6fb59fa4-0f8d-4111-87e3-e0a2e29982e4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-m5nkk"
Oct 30 22:43:26.968578 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:43:26.968578    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vckfb\" (UniqueName: \"kubernetes.io/projected/6fb59fa4-0f8d-4111-87e3-e0a2e29982e4-kube-api-access-vckfb\") pod \"maas-default-gateway-openshift-default-85544d4cd7-m5nkk\" (UID: \"6fb59fa4-0f8d-4111-87e3-e0a2e29982e4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-m5nkk"
Oct 30 22:43:26.968875 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:43:26.968613    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6fb59fa4-0f8d-4111-87e3-e0a2e29982e4-workload-socket\") pod \"maas-default-gateway-openshift-default-85544d4cd7-m5nkk\" (UID: \"6fb59fa4-0f8d-4111-87e3-e0a2e29982e4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-m5nkk"
Oct 30 22:43:26.968875 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:43:26.968736    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6fb59fa4-0f8d-4111-87e3-e0a2e29982e4-istio-podinfo\") pod \"maas-default-gateway-openshift-default-85544d4cd7-m5nkk\" (UID: \"6fb59fa4-0f8d-4111-87e3-e0a2e29982e4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-m5nkk"
Oct 30 22:43:26.968875 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:43:26.968786    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6fb59fa4-0f8d-4111-87e3-e0a2e29982e4-istio-token\") pod \"maas-default-gateway-openshift-default-85544d4cd7-m5nkk\" (UID: \"6fb59fa4-0f8d-4111-87e3-e0a2e29982e4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-m5nkk"
Oct 30 22:43:26.969057 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:43:26.968879    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6fb59fa4-0f8d-4111-87e3-e0a2e29982e4-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-85544d4cd7-m5nkk\" (UID: \"6fb59fa4-0f8d-4111-87e3-e0a2e29982e4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-m5nkk"
Oct 30 22:43:26.969057 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:43:26.969006    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6fb59fa4-0f8d-4111-87e3-e0a2e29982e4-istio-data\") pod \"maas-default-gateway-openshift-default-85544d4cd7-m5nkk\" (UID: \"6fb59fa4-0f8d-4111-87e3-e0a2e29982e4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-m5nkk"
Oct 30 22:43:26.969057 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:43:26.969025    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6fb59fa4-0f8d-4111-87e3-e0a2e29982e4-workload-socket\") pod \"maas-default-gateway-openshift-default-85544d4cd7-m5nkk\" (UID: \"6fb59fa4-0f8d-4111-87e3-e0a2e29982e4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-m5nkk"
Oct 30 22:43:26.969225 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:43:26.969056    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6fb59fa4-0f8d-4111-87e3-e0a2e29982e4-workload-certs\") pod \"maas-default-gateway-openshift-default-85544d4cd7-m5nkk\" (UID: \"6fb59fa4-0f8d-4111-87e3-e0a2e29982e4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-m5nkk"
Oct 30 22:43:26.969225 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:43:26.969110    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6fb59fa4-0f8d-4111-87e3-e0a2e29982e4-istio-envoy\") pod \"maas-default-gateway-openshift-default-85544d4cd7-m5nkk\" (UID: \"6fb59fa4-0f8d-4111-87e3-e0a2e29982e4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-m5nkk"
Oct 30 22:43:26.969506 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:43:26.969478    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6fb59fa4-0f8d-4111-87e3-e0a2e29982e4-istio-data\") pod \"maas-default-gateway-openshift-default-85544d4cd7-m5nkk\" (UID: \"6fb59fa4-0f8d-4111-87e3-e0a2e29982e4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-m5nkk"
Oct 30 22:43:26.969719 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:43:26.969697    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6fb59fa4-0f8d-4111-87e3-e0a2e29982e4-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-85544d4cd7-m5nkk\" (UID: \"6fb59fa4-0f8d-4111-87e3-e0a2e29982e4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-m5nkk"
Oct 30 22:43:26.969867 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:43:26.969776    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6fb59fa4-0f8d-4111-87e3-e0a2e29982e4-credential-socket\") pod \"maas-default-gateway-openshift-default-85544d4cd7-m5nkk\" (UID: \"6fb59fa4-0f8d-4111-87e3-e0a2e29982e4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-m5nkk"
Oct 30 22:43:26.970049 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:43:26.970023    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6fb59fa4-0f8d-4111-87e3-e0a2e29982e4-workload-certs\") pod \"maas-default-gateway-openshift-default-85544d4cd7-m5nkk\" (UID: \"6fb59fa4-0f8d-4111-87e3-e0a2e29982e4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-m5nkk"
Oct 30 22:43:26.971503 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:43:26.971478    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6fb59fa4-0f8d-4111-87e3-e0a2e29982e4-istio-envoy\") pod \"maas-default-gateway-openshift-default-85544d4cd7-m5nkk\" (UID: \"6fb59fa4-0f8d-4111-87e3-e0a2e29982e4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-m5nkk"
Oct 30 22:43:26.972071 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:43:26.972049    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6fb59fa4-0f8d-4111-87e3-e0a2e29982e4-istio-podinfo\") pod \"maas-default-gateway-openshift-default-85544d4cd7-m5nkk\" (UID: \"6fb59fa4-0f8d-4111-87e3-e0a2e29982e4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-m5nkk"
Oct 30 22:43:27.026726 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:43:27.026683    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vckfb\" (UniqueName: \"kubernetes.io/projected/6fb59fa4-0f8d-4111-87e3-e0a2e29982e4-kube-api-access-vckfb\") pod \"maas-default-gateway-openshift-default-85544d4cd7-m5nkk\" (UID: \"6fb59fa4-0f8d-4111-87e3-e0a2e29982e4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-m5nkk"
Oct 30 22:43:27.031097 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:43:27.031065    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6fb59fa4-0f8d-4111-87e3-e0a2e29982e4-istio-token\") pod \"maas-default-gateway-openshift-default-85544d4cd7-m5nkk\" (UID: \"6fb59fa4-0f8d-4111-87e3-e0a2e29982e4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-m5nkk"
Oct 30 22:43:27.059185 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:43:27.059132    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-m5nkk"
Oct 30 22:43:27.484021 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:43:27.483988    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-m5nkk"]
Oct 30 22:43:27.485927 ip-10-0-136-210 kubenswrapper[5510]: W1030 22:43:27.485897    5510 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6fb59fa4_0f8d_4111_87e3_e0a2e29982e4.slice/crio-5fe6216f6c5c471a621043e871e0b14127df27eec0bfe298ec54f63cfef10275 WatchSource:0}: Error finding container 5fe6216f6c5c471a621043e871e0b14127df27eec0bfe298ec54f63cfef10275: Status 404 returned error can't find the container with id 5fe6216f6c5c471a621043e871e0b14127df27eec0bfe298ec54f63cfef10275
Oct 30 22:43:27.488579 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:43:27.488534    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:43:27.488712 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:43:27.488684    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:43:27.488797 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:43:27.488739    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:43:28.232360 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:43:28.232325    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-m5nkk" event={"ID":"6fb59fa4-0f8d-4111-87e3-e0a2e29982e4","Type":"ContainerStarted","Data":"514767a795dc75f667719c7dca6abf88db429fd00bfd02110c0a27e9c7d9f616"}
Oct 30 22:43:28.232864 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:43:28.232411    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-m5nkk" event={"ID":"6fb59fa4-0f8d-4111-87e3-e0a2e29982e4","Type":"ContainerStarted","Data":"5fe6216f6c5c471a621043e871e0b14127df27eec0bfe298ec54f63cfef10275"}
Oct 30 22:43:28.396140 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:43:28.396068    5510 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-m5nkk" podStartSLOduration=2.396050906 podStartE2EDuration="2.396050906s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:43:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:43:28.395159503 +0000 UTC m=+3188.955969509" watchObservedRunningTime="2026-10-30 22:43:28.396050906 +0000 UTC m=+3188.956860913"
Oct 30 22:43:29.067757 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:43:29.067717    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-m5nkk"
Oct 30 22:43:29.068780 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:43:29.068742    5510 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-85544d4cd7-m5nkk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.153:15021/healthz/ready\": dial tcp 10.133.0.153:15021: connect: connection refused" start-of-body=
Oct 30 22:43:29.068919 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:43:29.068806    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-m5nkk" podUID="6fb59fa4-0f8d-4111-87e3-e0a2e29982e4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.153:15021/healthz/ready\": dial tcp 10.133.0.153:15021: connect: connection refused"
Oct 30 22:43:30.065681 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:43:30.065167    5510 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-85544d4cd7-m5nkk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.153:15021/healthz/ready\": dial tcp 10.133.0.153:15021: connect: connection refused" start-of-body=
Oct 30 22:43:30.065681 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:43:30.065243    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-m5nkk" podUID="6fb59fa4-0f8d-4111-87e3-e0a2e29982e4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.153:15021/healthz/ready\": dial tcp 10.133.0.153:15021: connect: connection refused"
Oct 30 22:43:31.064139 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:43:31.064097    5510 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-85544d4cd7-m5nkk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.153:15021/healthz/ready\": dial tcp 10.133.0.153:15021: connect: connection refused" start-of-body=
Oct 30 22:43:31.064442 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:43:31.064409    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-m5nkk" podUID="6fb59fa4-0f8d-4111-87e3-e0a2e29982e4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.153:15021/healthz/ready\": dial tcp 10.133.0.153:15021: connect: connection refused"
Oct 30 22:43:32.065187 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:43:32.065141    5510 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-85544d4cd7-m5nkk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.153:15021/healthz/ready\": dial tcp 10.133.0.153:15021: connect: connection refused" start-of-body=
Oct 30 22:43:32.065841 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:43:32.065807    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-m5nkk" podUID="6fb59fa4-0f8d-4111-87e3-e0a2e29982e4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.153:15021/healthz/ready\": dial tcp 10.133.0.153:15021: connect: connection refused"
Oct 30 22:43:33.064190 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:43:33.064151    5510 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-85544d4cd7-m5nkk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.153:15021/healthz/ready\": dial tcp 10.133.0.153:15021: connect: connection refused" start-of-body=
Oct 30 22:43:33.064500 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:43:33.064472    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-m5nkk" podUID="6fb59fa4-0f8d-4111-87e3-e0a2e29982e4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.153:15021/healthz/ready\": dial tcp 10.133.0.153:15021: connect: connection refused"
Oct 30 22:43:34.059590 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:43:34.059557    5510 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-85544d4cd7-m5nkk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.153:15021/healthz/ready\": dial tcp 10.133.0.153:15021: connect: connection refused" start-of-body=
Oct 30 22:43:34.059983 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:43:34.059623    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-m5nkk" podUID="6fb59fa4-0f8d-4111-87e3-e0a2e29982e4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.153:15021/healthz/ready\": dial tcp 10.133.0.153:15021: connect: connection refused"
Oct 30 22:43:35.555569 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:43:35.555533    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:43:35.763998 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:43:35.763964    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-m5nkk"
Oct 30 22:43:35.764386 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:43:35.764362    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-m5nkk"
Oct 30 22:43:35.765186 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:43:35.765154    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-m5nkk"
Oct 30 22:43:36.515218 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:43:36.515179    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:43:38.419143 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:43:38.419108    5510 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6gjbf" containerName="extractor"
Oct 30 22:44:08.419287 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:44:08.419246    5510 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6gjbf" containerName="extractor"
Oct 30 22:44:30.665737 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:44:30.663591    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:44:36.850262 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:44:36.850221    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:44:38.418636 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:44:38.418595    5510 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6gjbf" containerName="extractor"
Oct 30 22:45:06.256854 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:45:06.256776    5510 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-85544d4cd7-hfrbk container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.149:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 22:45:06.265928 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:45:06.257427    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-hfrbk" podUID="474fc2c2-03ae-420b-a831-c8e98a3e5ed8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.149:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:45:06.405681 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:45:06.404741    5510 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-85544d4cd7-llmh8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.150:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 22:45:06.405681 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:45:06.404804    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-llmh8" podUID="1123817a-b255-47e2-9188-91650a8b4378" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.150:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:45:08.419805 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:45:08.419765    5510 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6gjbf" containerName="extractor"
Oct 30 22:45:18.918076 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:45:18.918041    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:45:36.259808 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:45:36.259763    5510 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-85544d4cd7-hfrbk container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.149:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 22:45:36.272775 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:45:36.272732    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-hfrbk" podUID="474fc2c2-03ae-420b-a831-c8e98a3e5ed8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.149:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:45:36.389965 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:45:36.389927    5510 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-85544d4cd7-llmh8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.150:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 22:45:36.390267 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:45:36.390233    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-llmh8" podUID="1123817a-b255-47e2-9188-91650a8b4378" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.150:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:45:37.175580 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:45:37.175538    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:45:37.375280 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:45:37.375246    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:45:38.418565 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:45:38.418528    5510 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6gjbf" containerName="extractor"
Oct 30 22:46:08.419676 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:46:08.419567    5510 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6gjbf" containerName="extractor"
Oct 30 22:46:23.550457 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:46:23.549978    5510 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-85544d4cd7-q6cg4 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.152:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 22:46:23.550457 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:46:23.550055    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-q6cg4" podUID="46c98868-6890-4614-93b4-55493087df27" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.152:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:46:35.446535 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:46:35.446492    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:46:37.541899 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:46:37.541860    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:46:38.419461 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:46:38.419423    5510 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6gjbf" containerName="extractor"
Oct 30 22:47:08.418718 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:47:08.418675    5510 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6gjbf" containerName="extractor"
Oct 30 22:47:37.904685 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:47:37.904632    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:47:38.418562 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:47:38.418526    5510 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6gjbf" containerName="extractor"
Oct 30 22:48:08.420993 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:48:08.420952    5510 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6gjbf" containerName="extractor"
Oct 30 22:48:38.230705 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:48:38.230644    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:48:38.419383 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:48:38.419340    5510 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6gjbf" containerName="extractor"
Oct 30 22:48:38.427238 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:48:38.427201    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:48:44.403055 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:48:44.403012    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:48:44.848387 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:48:44.848349    5510 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-single-main-88e685-openshift-default-94f47957-dn4cw"]
Oct 30 22:48:44.852148 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:48:44.852129    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-single-main-88e685-openshift-default-94f47957-dn4cw"
Oct 30 22:48:44.902842 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:48:44.902808    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-single-main-88e685-openshift-default-dockercfg-rtm6h\""
Oct 30 22:48:44.946427 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:48:44.946385    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sfb5r\" (UniqueName: \"kubernetes.io/projected/31c93ac4-77c9-4231-845d-b13607c94688-kube-api-access-sfb5r\") pod \"e2e-worker-single-main-88e685-openshift-default-94f47957-dn4cw\" (UID: \"31c93ac4-77c9-4231-845d-b13607c94688\") " pod="openshift-ingress/e2e-worker-single-main-88e685-openshift-default-94f47957-dn4cw"
Oct 30 22:48:44.946427 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:48:44.946426    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/31c93ac4-77c9-4231-845d-b13607c94688-istio-data\") pod \"e2e-worker-single-main-88e685-openshift-default-94f47957-dn4cw\" (UID: \"31c93ac4-77c9-4231-845d-b13607c94688\") " pod="openshift-ingress/e2e-worker-single-main-88e685-openshift-default-94f47957-dn4cw"
Oct 30 22:48:44.946640 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:48:44.946490    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/31c93ac4-77c9-4231-845d-b13607c94688-istio-envoy\") pod \"e2e-worker-single-main-88e685-openshift-default-94f47957-dn4cw\" (UID: \"31c93ac4-77c9-4231-845d-b13607c94688\") " pod="openshift-ingress/e2e-worker-single-main-88e685-openshift-default-94f47957-dn4cw"
Oct 30 22:48:44.946640 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:48:44.946588    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/31c93ac4-77c9-4231-845d-b13607c94688-workload-socket\") pod \"e2e-worker-single-main-88e685-openshift-default-94f47957-dn4cw\" (UID: \"31c93ac4-77c9-4231-845d-b13607c94688\") " pod="openshift-ingress/e2e-worker-single-main-88e685-openshift-default-94f47957-dn4cw"
Oct 30 22:48:44.946751 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:48:44.946638    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/31c93ac4-77c9-4231-845d-b13607c94688-istiod-ca-cert\") pod \"e2e-worker-single-main-88e685-openshift-default-94f47957-dn4cw\" (UID: \"31c93ac4-77c9-4231-845d-b13607c94688\") " pod="openshift-ingress/e2e-worker-single-main-88e685-openshift-default-94f47957-dn4cw"
Oct 30 22:48:44.946751 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:48:44.946735    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/31c93ac4-77c9-4231-845d-b13607c94688-workload-certs\") pod \"e2e-worker-single-main-88e685-openshift-default-94f47957-dn4cw\" (UID: \"31c93ac4-77c9-4231-845d-b13607c94688\") " pod="openshift-ingress/e2e-worker-single-main-88e685-openshift-default-94f47957-dn4cw"
Oct 30 22:48:44.946827 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:48:44.946770    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/31c93ac4-77c9-4231-845d-b13607c94688-istio-podinfo\") pod \"e2e-worker-single-main-88e685-openshift-default-94f47957-dn4cw\" (UID: \"31c93ac4-77c9-4231-845d-b13607c94688\") " pod="openshift-ingress/e2e-worker-single-main-88e685-openshift-default-94f47957-dn4cw"
Oct 30 22:48:44.946897 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:48:44.946877    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/31c93ac4-77c9-4231-845d-b13607c94688-credential-socket\") pod \"e2e-worker-single-main-88e685-openshift-default-94f47957-dn4cw\" (UID: \"31c93ac4-77c9-4231-845d-b13607c94688\") " pod="openshift-ingress/e2e-worker-single-main-88e685-openshift-default-94f47957-dn4cw"
Oct 30 22:48:44.946940 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:48:44.946921    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/31c93ac4-77c9-4231-845d-b13607c94688-istio-token\") pod \"e2e-worker-single-main-88e685-openshift-default-94f47957-dn4cw\" (UID: \"31c93ac4-77c9-4231-845d-b13607c94688\") " pod="openshift-ingress/e2e-worker-single-main-88e685-openshift-default-94f47957-dn4cw"
Oct 30 22:48:44.951961 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:48:44.951929    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-single-main-88e685-openshift-default-94f47957-dn4cw"]
Oct 30 22:48:45.048080 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:48:45.048034    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/31c93ac4-77c9-4231-845d-b13607c94688-credential-socket\") pod \"e2e-worker-single-main-88e685-openshift-default-94f47957-dn4cw\" (UID: \"31c93ac4-77c9-4231-845d-b13607c94688\") " pod="openshift-ingress/e2e-worker-single-main-88e685-openshift-default-94f47957-dn4cw"
Oct 30 22:48:45.048299 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:48:45.048090    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/31c93ac4-77c9-4231-845d-b13607c94688-istio-token\") pod \"e2e-worker-single-main-88e685-openshift-default-94f47957-dn4cw\" (UID: \"31c93ac4-77c9-4231-845d-b13607c94688\") " pod="openshift-ingress/e2e-worker-single-main-88e685-openshift-default-94f47957-dn4cw"
Oct 30 22:48:45.048299 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:48:45.048140    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sfb5r\" (UniqueName: \"kubernetes.io/projected/31c93ac4-77c9-4231-845d-b13607c94688-kube-api-access-sfb5r\") pod \"e2e-worker-single-main-88e685-openshift-default-94f47957-dn4cw\" (UID: \"31c93ac4-77c9-4231-845d-b13607c94688\") " pod="openshift-ingress/e2e-worker-single-main-88e685-openshift-default-94f47957-dn4cw"
Oct 30 22:48:45.048299 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:48:45.048158    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/31c93ac4-77c9-4231-845d-b13607c94688-istio-data\") pod \"e2e-worker-single-main-88e685-openshift-default-94f47957-dn4cw\" (UID: \"31c93ac4-77c9-4231-845d-b13607c94688\") " pod="openshift-ingress/e2e-worker-single-main-88e685-openshift-default-94f47957-dn4cw"
Oct 30 22:48:45.048299 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:48:45.048187    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/31c93ac4-77c9-4231-845d-b13607c94688-istio-envoy\") pod \"e2e-worker-single-main-88e685-openshift-default-94f47957-dn4cw\" (UID: \"31c93ac4-77c9-4231-845d-b13607c94688\") " pod="openshift-ingress/e2e-worker-single-main-88e685-openshift-default-94f47957-dn4cw"
Oct 30 22:48:45.048299 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:48:45.048219    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/31c93ac4-77c9-4231-845d-b13607c94688-workload-socket\") pod \"e2e-worker-single-main-88e685-openshift-default-94f47957-dn4cw\" (UID: \"31c93ac4-77c9-4231-845d-b13607c94688\") " pod="openshift-ingress/e2e-worker-single-main-88e685-openshift-default-94f47957-dn4cw"
Oct 30 22:48:45.048299 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:48:45.048243    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/31c93ac4-77c9-4231-845d-b13607c94688-istiod-ca-cert\") pod \"e2e-worker-single-main-88e685-openshift-default-94f47957-dn4cw\" (UID: \"31c93ac4-77c9-4231-845d-b13607c94688\") " pod="openshift-ingress/e2e-worker-single-main-88e685-openshift-default-94f47957-dn4cw"
Oct 30 22:48:45.048299 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:48:45.048275    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/31c93ac4-77c9-4231-845d-b13607c94688-workload-certs\") pod \"e2e-worker-single-main-88e685-openshift-default-94f47957-dn4cw\" (UID: \"31c93ac4-77c9-4231-845d-b13607c94688\") " pod="openshift-ingress/e2e-worker-single-main-88e685-openshift-default-94f47957-dn4cw"
Oct 30 22:48:45.048747 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:48:45.048382    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/31c93ac4-77c9-4231-845d-b13607c94688-istio-podinfo\") pod \"e2e-worker-single-main-88e685-openshift-default-94f47957-dn4cw\" (UID: \"31c93ac4-77c9-4231-845d-b13607c94688\") " pod="openshift-ingress/e2e-worker-single-main-88e685-openshift-default-94f47957-dn4cw"
Oct 30 22:48:45.048747 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:48:45.048608    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/31c93ac4-77c9-4231-845d-b13607c94688-istio-data\") pod \"e2e-worker-single-main-88e685-openshift-default-94f47957-dn4cw\" (UID: \"31c93ac4-77c9-4231-845d-b13607c94688\") " pod="openshift-ingress/e2e-worker-single-main-88e685-openshift-default-94f47957-dn4cw"
Oct 30 22:48:45.048747 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:48:45.048716    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/31c93ac4-77c9-4231-845d-b13607c94688-workload-socket\") pod \"e2e-worker-single-main-88e685-openshift-default-94f47957-dn4cw\" (UID: \"31c93ac4-77c9-4231-845d-b13607c94688\") " pod="openshift-ingress/e2e-worker-single-main-88e685-openshift-default-94f47957-dn4cw"
Oct 30 22:48:45.048879 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:48:45.048754    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/31c93ac4-77c9-4231-845d-b13607c94688-workload-certs\") pod \"e2e-worker-single-main-88e685-openshift-default-94f47957-dn4cw\" (UID: \"31c93ac4-77c9-4231-845d-b13607c94688\") " pod="openshift-ingress/e2e-worker-single-main-88e685-openshift-default-94f47957-dn4cw"
Oct 30 22:48:45.048879 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:48:45.048756    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/31c93ac4-77c9-4231-845d-b13607c94688-credential-socket\") pod \"e2e-worker-single-main-88e685-openshift-default-94f47957-dn4cw\" (UID: \"31c93ac4-77c9-4231-845d-b13607c94688\") " pod="openshift-ingress/e2e-worker-single-main-88e685-openshift-default-94f47957-dn4cw"
Oct 30 22:48:45.049037 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:48:45.049018    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/31c93ac4-77c9-4231-845d-b13607c94688-istiod-ca-cert\") pod \"e2e-worker-single-main-88e685-openshift-default-94f47957-dn4cw\" (UID: \"31c93ac4-77c9-4231-845d-b13607c94688\") " pod="openshift-ingress/e2e-worker-single-main-88e685-openshift-default-94f47957-dn4cw"
Oct 30 22:48:45.050683 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:48:45.050648    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/31c93ac4-77c9-4231-845d-b13607c94688-istio-envoy\") pod \"e2e-worker-single-main-88e685-openshift-default-94f47957-dn4cw\" (UID: \"31c93ac4-77c9-4231-845d-b13607c94688\") " pod="openshift-ingress/e2e-worker-single-main-88e685-openshift-default-94f47957-dn4cw"
Oct 30 22:48:45.050745 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:48:45.050725    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/31c93ac4-77c9-4231-845d-b13607c94688-istio-podinfo\") pod \"e2e-worker-single-main-88e685-openshift-default-94f47957-dn4cw\" (UID: \"31c93ac4-77c9-4231-845d-b13607c94688\") " pod="openshift-ingress/e2e-worker-single-main-88e685-openshift-default-94f47957-dn4cw"
Oct 30 22:48:45.111478 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:48:45.111405    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sfb5r\" (UniqueName: \"kubernetes.io/projected/31c93ac4-77c9-4231-845d-b13607c94688-kube-api-access-sfb5r\") pod \"e2e-worker-single-main-88e685-openshift-default-94f47957-dn4cw\" (UID: \"31c93ac4-77c9-4231-845d-b13607c94688\") " pod="openshift-ingress/e2e-worker-single-main-88e685-openshift-default-94f47957-dn4cw"
Oct 30 22:48:45.120131 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:48:45.120100    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/31c93ac4-77c9-4231-845d-b13607c94688-istio-token\") pod \"e2e-worker-single-main-88e685-openshift-default-94f47957-dn4cw\" (UID: \"31c93ac4-77c9-4231-845d-b13607c94688\") " pod="openshift-ingress/e2e-worker-single-main-88e685-openshift-default-94f47957-dn4cw"
Oct 30 22:48:45.165854 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:48:45.165814    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-single-main-88e685-openshift-default-94f47957-dn4cw"
Oct 30 22:48:45.190107 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:48:45.190072    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:48:45.387013 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:48:45.386924    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:48:45.586817 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:48:45.586790    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-single-main-88e685-openshift-default-94f47957-dn4cw"]
Oct 30 22:48:45.589015 ip-10-0-136-210 kubenswrapper[5510]: W1030 22:48:45.588986    5510 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod31c93ac4_77c9_4231_845d_b13607c94688.slice/crio-bfa7cb1cd8b9b167552c1832b566faa328ed68e9af315da42ab25beebe54e102 WatchSource:0}: Error finding container bfa7cb1cd8b9b167552c1832b566faa328ed68e9af315da42ab25beebe54e102: Status 404 returned error can't find the container with id bfa7cb1cd8b9b167552c1832b566faa328ed68e9af315da42ab25beebe54e102
Oct 30 22:48:45.591181 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:48:45.591149    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:48:45.591248 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:48:45.591217    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:48:45.591248 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:48:45.591243    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:48:46.182832 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:48:46.182793    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-88e685-openshift-default-94f47957-dn4cw" event={"ID":"31c93ac4-77c9-4231-845d-b13607c94688","Type":"ContainerStarted","Data":"b1e4702ea8b25b90b10aa8ee9331855d05c3639329595cb9c429d2bd2ab280f6"}
Oct 30 22:48:46.183046 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:48:46.182853    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-88e685-openshift-default-94f47957-dn4cw" event={"ID":"31c93ac4-77c9-4231-845d-b13607c94688","Type":"ContainerStarted","Data":"bfa7cb1cd8b9b167552c1832b566faa328ed68e9af315da42ab25beebe54e102"}
Oct 30 22:48:46.298841 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:48:46.298790    5510 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-single-main-88e685-openshift-default-94f47957-dn4cw" podStartSLOduration=2.298774654 podStartE2EDuration="2.298774654s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:48:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:48:46.296940986 +0000 UTC m=+3506.857751002" watchObservedRunningTime="2026-10-30 22:48:46.298774654 +0000 UTC m=+3506.859584660"
Oct 30 22:48:47.166567 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:48:47.166531    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-single-main-88e685-openshift-default-94f47957-dn4cw"
Oct 30 22:48:47.171540 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:48:47.171515    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-single-main-88e685-openshift-default-94f47957-dn4cw"
Oct 30 22:48:47.187833 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:48:47.187804    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-single-main-88e685-openshift-default-94f47957-dn4cw"
Oct 30 22:48:47.188744 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:48:47.188728    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-single-main-88e685-openshift-default-94f47957-dn4cw"
Oct 30 22:48:49.408293 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:48:49.408252    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:48:49.646811 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:48:49.646769    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:48:52.605236 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:48:52.605191    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:48:52.795219 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:48:52.795175    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:48:52.964127 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:48:52.964035    5510 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-88e685-59954dqthn"]
Oct 30 22:48:52.967405 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:48:52.967385    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-88e685-59954dqthn"
Oct 30 22:48:53.028411 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:48:53.028360    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fbvcq\" (UniqueName: \"kubernetes.io/projected/acebd8f6-36f7-4152-9154-0b8b186d2a81-kube-api-access-fbvcq\") pod \"payload-pre-processing-e2e-worker-single-main-88e685-59954dqthn\" (UID: \"acebd8f6-36f7-4152-9154-0b8b186d2a81\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-88e685-59954dqthn"
Oct 30 22:48:53.028611 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:48:53.028573    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/acebd8f6-36f7-4152-9154-0b8b186d2a81-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-single-main-88e685-59954dqthn\" (UID: \"acebd8f6-36f7-4152-9154-0b8b186d2a81\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-88e685-59954dqthn"
Oct 30 22:48:53.063229 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:48:53.063189    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-single-m-c9de1d19-dockercfg-8trtw\""
Oct 30 22:48:53.063429 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:48:53.063239    5510 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-single-main-88e685\""
Oct 30 22:48:53.064944 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:48:53.064922    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-88e685-59954dqthn"]
Oct 30 22:48:53.119392 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:48:53.119358    5510 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-88e685-77594c5967zjvx"]
Oct 30 22:48:53.123038 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:48:53.123015    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-single-main-88e685-77594c5967zjvx"
Oct 30 22:48:53.129304 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:48:53.129270    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/acebd8f6-36f7-4152-9154-0b8b186d2a81-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-single-main-88e685-59954dqthn\" (UID: \"acebd8f6-36f7-4152-9154-0b8b186d2a81\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-88e685-59954dqthn"
Oct 30 22:48:53.129450 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:48:53.129380    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fbvcq\" (UniqueName: \"kubernetes.io/projected/acebd8f6-36f7-4152-9154-0b8b186d2a81-kube-api-access-fbvcq\") pod \"payload-pre-processing-e2e-worker-single-main-88e685-59954dqthn\" (UID: \"acebd8f6-36f7-4152-9154-0b8b186d2a81\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-88e685-59954dqthn"
Oct 30 22:48:53.130062 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:48:53.130040    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/acebd8f6-36f7-4152-9154-0b8b186d2a81-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-single-main-88e685-59954dqthn\" (UID: \"acebd8f6-36f7-4152-9154-0b8b186d2a81\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-88e685-59954dqthn"
Oct 30 22:48:53.178216 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:48:53.178188    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-88e685-77594c5967zjvx"]
Oct 30 22:48:53.187834 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:48:53.187807    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fbvcq\" (UniqueName: \"kubernetes.io/projected/acebd8f6-36f7-4152-9154-0b8b186d2a81-kube-api-access-fbvcq\") pod \"payload-pre-processing-e2e-worker-single-main-88e685-59954dqthn\" (UID: \"acebd8f6-36f7-4152-9154-0b8b186d2a81\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-88e685-59954dqthn"
Oct 30 22:48:53.230118 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:48:53.230025    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2p2sg\" (UniqueName: \"kubernetes.io/projected/920155b2-a008-4621-8019-de154b8a1fea-kube-api-access-2p2sg\") pod \"payload-processing-e2e-worker-single-main-88e685-77594c5967zjvx\" (UID: \"920155b2-a008-4621-8019-de154b8a1fea\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-88e685-77594c5967zjvx"
Oct 30 22:48:53.230118 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:48:53.230077    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/920155b2-a008-4621-8019-de154b8a1fea-plugins-config-volume\") pod \"payload-processing-e2e-worker-single-main-88e685-77594c5967zjvx\" (UID: \"920155b2-a008-4621-8019-de154b8a1fea\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-88e685-77594c5967zjvx"
Oct 30 22:48:53.280907 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:48:53.280869    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-88e685-59954dqthn"
Oct 30 22:48:53.331048 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:48:53.331017    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2p2sg\" (UniqueName: \"kubernetes.io/projected/920155b2-a008-4621-8019-de154b8a1fea-kube-api-access-2p2sg\") pod \"payload-processing-e2e-worker-single-main-88e685-77594c5967zjvx\" (UID: \"920155b2-a008-4621-8019-de154b8a1fea\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-88e685-77594c5967zjvx"
Oct 30 22:48:53.331265 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:48:53.331075    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/920155b2-a008-4621-8019-de154b8a1fea-plugins-config-volume\") pod \"payload-processing-e2e-worker-single-main-88e685-77594c5967zjvx\" (UID: \"920155b2-a008-4621-8019-de154b8a1fea\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-88e685-77594c5967zjvx"
Oct 30 22:48:53.331844 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:48:53.331809    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/920155b2-a008-4621-8019-de154b8a1fea-plugins-config-volume\") pod \"payload-processing-e2e-worker-single-main-88e685-77594c5967zjvx\" (UID: \"920155b2-a008-4621-8019-de154b8a1fea\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-88e685-77594c5967zjvx"
Oct 30 22:48:53.390168 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:48:53.390126    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2p2sg\" (UniqueName: \"kubernetes.io/projected/920155b2-a008-4621-8019-de154b8a1fea-kube-api-access-2p2sg\") pod \"payload-processing-e2e-worker-single-main-88e685-77594c5967zjvx\" (UID: \"920155b2-a008-4621-8019-de154b8a1fea\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-88e685-77594c5967zjvx"
Oct 30 22:48:53.436088 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:48:53.436058    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-single-main-88e685-77594c5967zjvx"
Oct 30 22:48:53.650339 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:48:53.650310    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-88e685-59954dqthn"]
Oct 30 22:48:53.652606 ip-10-0-136-210 kubenswrapper[5510]: W1030 22:48:53.652580    5510 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podacebd8f6_36f7_4152_9154_0b8b186d2a81.slice/crio-f5868bb368a69f2ff875c8f9d9901ef8842bf289f77df84994021f487a1fdcbb WatchSource:0}: Error finding container f5868bb368a69f2ff875c8f9d9901ef8842bf289f77df84994021f487a1fdcbb: Status 404 returned error can't find the container with id f5868bb368a69f2ff875c8f9d9901ef8842bf289f77df84994021f487a1fdcbb
Oct 30 22:48:53.801674 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:48:53.801625    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-88e685-77594c5967zjvx"]
Oct 30 22:48:53.804277 ip-10-0-136-210 kubenswrapper[5510]: W1030 22:48:53.804250    5510 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod920155b2_a008_4621_8019_de154b8a1fea.slice/crio-598ed122f0e4e7562d22b2872d64f5512dfad6310f70de7aec6f737df0eec124 WatchSource:0}: Error finding container 598ed122f0e4e7562d22b2872d64f5512dfad6310f70de7aec6f737df0eec124: Status 404 returned error can't find the container with id 598ed122f0e4e7562d22b2872d64f5512dfad6310f70de7aec6f737df0eec124
Oct 30 22:48:54.222365 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:48:54.222270    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-88e685-59954dqthn" event={"ID":"acebd8f6-36f7-4152-9154-0b8b186d2a81","Type":"ContainerStarted","Data":"d617393d0278be031079d40dbf5f79c7b35d653e5174e75ca4bde672e2c921fb"}
Oct 30 22:48:54.222365 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:48:54.222338    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-88e685-59954dqthn" event={"ID":"acebd8f6-36f7-4152-9154-0b8b186d2a81","Type":"ContainerStarted","Data":"f5868bb368a69f2ff875c8f9d9901ef8842bf289f77df84994021f487a1fdcbb"}
Oct 30 22:48:54.222596 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:48:54.222508    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-88e685-59954dqthn"
Oct 30 22:48:54.224590 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:48:54.224562    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-single-main-88e685-77594c5967zjvx" event={"ID":"920155b2-a008-4621-8019-de154b8a1fea","Type":"ContainerStarted","Data":"d5e58440d61fe8b4c597310353441f773842c2cc873c8789c76a2999a75d142e"}
Oct 30 22:48:54.224741 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:48:54.224601    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-single-main-88e685-77594c5967zjvx" event={"ID":"920155b2-a008-4621-8019-de154b8a1fea","Type":"ContainerStarted","Data":"598ed122f0e4e7562d22b2872d64f5512dfad6310f70de7aec6f737df0eec124"}
Oct 30 22:48:54.224741 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:48:54.224668    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-single-main-88e685-77594c5967zjvx"
Oct 30 22:48:54.334715 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:48:54.334632    5510 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-88e685-59954dqthn" podStartSLOduration=2.33461875 podStartE2EDuration="2.33461875s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:48:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:48:54.332594265 +0000 UTC m=+3514.893404270" watchObservedRunningTime="2026-10-30 22:48:54.33461875 +0000 UTC m=+3514.895428755"
Oct 30 22:48:54.445714 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:48:54.444146    5510 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-single-main-88e685-77594c5967zjvx" podStartSLOduration=1.444126415 podStartE2EDuration="1.444126415s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:48:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:48:54.442016627 +0000 UTC m=+3515.002826657" watchObservedRunningTime="2026-10-30 22:48:54.444126415 +0000 UTC m=+3515.004936423"
Oct 30 22:48:55.500334 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:48:55.500296    5510 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-88e685-69dbb8b6bb-29796"]
Oct 30 22:48:55.504183 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:48:55.504165    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-88e685-69dbb8b6bb-29796"
Oct 30 22:48:55.553692 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:48:55.553643    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-single-main-88e685-serving-cert\""
Oct 30 22:48:55.556715 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:48:55.556680    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fc669e80-db80-4479-93d5-79c6651b405d-metrics-certs\") pod \"maas-api-e2e-worker-single-main-88e685-69dbb8b6bb-29796\" (UID: \"fc669e80-db80-4479-93d5-79c6651b405d\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-88e685-69dbb8b6bb-29796"
Oct 30 22:48:55.556882 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:48:55.556738    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fc669e80-db80-4479-93d5-79c6651b405d-maas-api-tls\") pod \"maas-api-e2e-worker-single-main-88e685-69dbb8b6bb-29796\" (UID: \"fc669e80-db80-4479-93d5-79c6651b405d\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-88e685-69dbb8b6bb-29796"
Oct 30 22:48:55.556882 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:48:55.556803    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vkpb4\" (UniqueName: \"kubernetes.io/projected/fc669e80-db80-4479-93d5-79c6651b405d-kube-api-access-vkpb4\") pod \"maas-api-e2e-worker-single-main-88e685-69dbb8b6bb-29796\" (UID: \"fc669e80-db80-4479-93d5-79c6651b405d\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-88e685-69dbb8b6bb-29796"
Oct 30 22:48:55.556998 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:48:55.556891    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fc669e80-db80-4479-93d5-79c6651b405d-openshift-service-ca\") pod \"maas-api-e2e-worker-single-main-88e685-69dbb8b6bb-29796\" (UID: \"fc669e80-db80-4479-93d5-79c6651b405d\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-88e685-69dbb8b6bb-29796"
Oct 30 22:48:55.583421 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:48:55.583386    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-88e685-69dbb8b6bb-29796"]
Oct 30 22:48:55.658435 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:48:55.658398    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fc669e80-db80-4479-93d5-79c6651b405d-openshift-service-ca\") pod \"maas-api-e2e-worker-single-main-88e685-69dbb8b6bb-29796\" (UID: \"fc669e80-db80-4479-93d5-79c6651b405d\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-88e685-69dbb8b6bb-29796"
Oct 30 22:48:55.658642 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:48:55.658481    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fc669e80-db80-4479-93d5-79c6651b405d-metrics-certs\") pod \"maas-api-e2e-worker-single-main-88e685-69dbb8b6bb-29796\" (UID: \"fc669e80-db80-4479-93d5-79c6651b405d\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-88e685-69dbb8b6bb-29796"
Oct 30 22:48:55.658642 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:48:55.658503    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fc669e80-db80-4479-93d5-79c6651b405d-maas-api-tls\") pod \"maas-api-e2e-worker-single-main-88e685-69dbb8b6bb-29796\" (UID: \"fc669e80-db80-4479-93d5-79c6651b405d\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-88e685-69dbb8b6bb-29796"
Oct 30 22:48:55.658642 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:48:55.658543    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vkpb4\" (UniqueName: \"kubernetes.io/projected/fc669e80-db80-4479-93d5-79c6651b405d-kube-api-access-vkpb4\") pod \"maas-api-e2e-worker-single-main-88e685-69dbb8b6bb-29796\" (UID: \"fc669e80-db80-4479-93d5-79c6651b405d\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-88e685-69dbb8b6bb-29796"
Oct 30 22:48:55.658875 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:48:55.658670    5510 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-single-main-88e685-serving-cert: secret "maas-api-e2e-worker-single-main-88e685-serving-cert" not found
Oct 30 22:48:55.658875 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:48:55.658750    5510 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fc669e80-db80-4479-93d5-79c6651b405d-maas-api-tls podName:fc669e80-db80-4479-93d5-79c6651b405d nodeName:}" failed. No retries permitted until 2026-10-30 22:48:56.158729045 +0000 UTC m=+3516.719539028 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/fc669e80-db80-4479-93d5-79c6651b405d-maas-api-tls") pod "maas-api-e2e-worker-single-main-88e685-69dbb8b6bb-29796" (UID: "fc669e80-db80-4479-93d5-79c6651b405d") : secret "maas-api-e2e-worker-single-main-88e685-serving-cert" not found
Oct 30 22:48:55.659176 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:48:55.659150    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fc669e80-db80-4479-93d5-79c6651b405d-openshift-service-ca\") pod \"maas-api-e2e-worker-single-main-88e685-69dbb8b6bb-29796\" (UID: \"fc669e80-db80-4479-93d5-79c6651b405d\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-88e685-69dbb8b6bb-29796"
Oct 30 22:48:55.661158 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:48:55.661132    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fc669e80-db80-4479-93d5-79c6651b405d-metrics-certs\") pod \"maas-api-e2e-worker-single-main-88e685-69dbb8b6bb-29796\" (UID: \"fc669e80-db80-4479-93d5-79c6651b405d\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-88e685-69dbb8b6bb-29796"
Oct 30 22:48:55.716729 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:48:55.716696    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vkpb4\" (UniqueName: \"kubernetes.io/projected/fc669e80-db80-4479-93d5-79c6651b405d-kube-api-access-vkpb4\") pod \"maas-api-e2e-worker-single-main-88e685-69dbb8b6bb-29796\" (UID: \"fc669e80-db80-4479-93d5-79c6651b405d\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-88e685-69dbb8b6bb-29796"
Oct 30 22:48:55.739460 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:48:55.739422    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:48:55.933775 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:48:55.933716    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:48:56.121464 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:48:56.121424    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:48:56.164186 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:48:56.164150    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fc669e80-db80-4479-93d5-79c6651b405d-maas-api-tls\") pod \"maas-api-e2e-worker-single-main-88e685-69dbb8b6bb-29796\" (UID: \"fc669e80-db80-4479-93d5-79c6651b405d\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-88e685-69dbb8b6bb-29796"
Oct 30 22:48:56.166554 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:48:56.166525    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fc669e80-db80-4479-93d5-79c6651b405d-maas-api-tls\") pod \"maas-api-e2e-worker-single-main-88e685-69dbb8b6bb-29796\" (UID: \"fc669e80-db80-4479-93d5-79c6651b405d\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-88e685-69dbb8b6bb-29796"
Oct 30 22:48:56.418184 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:48:56.418153    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-88e685-69dbb8b6bb-29796"
Oct 30 22:48:56.787456 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:48:56.787424    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-88e685-69dbb8b6bb-29796"]
Oct 30 22:48:56.789735 ip-10-0-136-210 kubenswrapper[5510]: W1030 22:48:56.789706    5510 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc669e80_db80_4479_93d5_79c6651b405d.slice/crio-a7ab5577b1bd76f2c2dcba5692fb6381da13d56b1bddab7bf54a9e3be174e00c WatchSource:0}: Error finding container a7ab5577b1bd76f2c2dcba5692fb6381da13d56b1bddab7bf54a9e3be174e00c: Status 404 returned error can't find the container with id a7ab5577b1bd76f2c2dcba5692fb6381da13d56b1bddab7bf54a9e3be174e00c
Oct 30 22:48:56.791327 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:48:56.791302    5510 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 22:48:57.240648 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:48:57.240563    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-88e685-69dbb8b6bb-29796" event={"ID":"fc669e80-db80-4479-93d5-79c6651b405d","Type":"ContainerStarted","Data":"a7ab5577b1bd76f2c2dcba5692fb6381da13d56b1bddab7bf54a9e3be174e00c"}
Oct 30 22:48:58.246877 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:48:58.246837    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-88e685-69dbb8b6bb-29796" event={"ID":"fc669e80-db80-4479-93d5-79c6651b405d","Type":"ContainerStarted","Data":"7327acd8f1b3e94c968235bb4d86f905fedb0b6e162aec5e601f36bd1863ba90"}
Oct 30 22:48:58.247243 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:48:58.247024    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-88e685-69dbb8b6bb-29796"
Oct 30 22:48:58.358976 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:48:58.358916    5510 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-88e685-69dbb8b6bb-29796" podStartSLOduration=2.530562158 podStartE2EDuration="3.358893492s" totalImagesPullingTime="828.331334ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:48:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:48:58.358837072 +0000 UTC m=+3518.919647082" watchObservedRunningTime="2026-10-30 22:48:58.358893492 +0000 UTC m=+3518.919703494"
Oct 30 22:49:00.806302 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:49:00.806204    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:49:04.272928 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:49:04.272897    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-88e685-69dbb8b6bb-29796"
Oct 30 22:49:05.232176 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:49:05.232146    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-88e685-59954dqthn"
Oct 30 22:49:05.232334 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:49:05.232308    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-single-main-88e685-77594c5967zjvx"
Oct 30 22:49:05.849683 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:49:05.848347    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:49:08.419202 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:49:08.419160    5510 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6gjbf" containerName="extractor"
Oct 30 22:49:10.821141 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:49:10.821098    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:49:13.319843 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:49:13.319806    5510 generic.go:356] "Generic (PLEG): container finished" podID="31c93ac4-77c9-4231-845d-b13607c94688" containerID="b1e4702ea8b25b90b10aa8ee9331855d05c3639329595cb9c429d2bd2ab280f6" exitCode=0
Oct 30 22:49:13.320300 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:49:13.319878    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-88e685-openshift-default-94f47957-dn4cw" event={"ID":"31c93ac4-77c9-4231-845d-b13607c94688","Type":"ContainerDied","Data":"b1e4702ea8b25b90b10aa8ee9331855d05c3639329595cb9c429d2bd2ab280f6"}
Oct 30 22:49:13.320345 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:49:13.320328    5510 scope.go:122] "RemoveContainer" containerID="b1e4702ea8b25b90b10aa8ee9331855d05c3639329595cb9c429d2bd2ab280f6"
Oct 30 22:49:13.321634 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:49:13.321603    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:49:13.321719 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:49:13.321705    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:49:13.321760 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:49:13.321742    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:49:14.201276 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:49:14.201234    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:49:14.326810 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:49:14.326776    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-88e685-openshift-default-94f47957-dn4cw" event={"ID":"31c93ac4-77c9-4231-845d-b13607c94688","Type":"ContainerStarted","Data":"d7e2ccbbb52bd53c00b5409705cfc10e0f1934b469a11a335608ef6168e10013"}
Oct 30 22:49:14.884236 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:49:14.884199    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:49:15.115869 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:49:15.115829    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:49:15.168923 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:49:15.168823    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-single-main-88e685-openshift-default-94f47957-dn4cw"
Oct 30 22:49:15.170299 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:49:15.170266    5510 patch_prober.go:29] interesting pod/e2e-worker-single-main-88e685-openshift-default-94f47957-dn4cw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.154:15021/healthz/ready\": dial tcp 10.133.0.154:15021: connect: connection refused" start-of-body=
Oct 30 22:49:15.170444 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:49:15.170337    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-88e685-openshift-default-94f47957-dn4cw" podUID="31c93ac4-77c9-4231-845d-b13607c94688" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.154:15021/healthz/ready\": dial tcp 10.133.0.154:15021: connect: connection refused"
Oct 30 22:49:16.169068 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:49:16.169023    5510 patch_prober.go:29] interesting pod/e2e-worker-single-main-88e685-openshift-default-94f47957-dn4cw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.154:15021/healthz/ready\": dial tcp 10.133.0.154:15021: connect: connection refused" start-of-body=
Oct 30 22:49:16.169561 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:49:16.169093    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-88e685-openshift-default-94f47957-dn4cw" podUID="31c93ac4-77c9-4231-845d-b13607c94688" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.154:15021/healthz/ready\": dial tcp 10.133.0.154:15021: connect: connection refused"
Oct 30 22:49:16.303858 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:49:16.303824    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:49:17.169100 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:49:17.169060    5510 patch_prober.go:29] interesting pod/e2e-worker-single-main-88e685-openshift-default-94f47957-dn4cw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.154:15021/healthz/ready\": dial tcp 10.133.0.154:15021: connect: connection refused" start-of-body=
Oct 30 22:49:17.169524 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:49:17.169129    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-88e685-openshift-default-94f47957-dn4cw" podUID="31c93ac4-77c9-4231-845d-b13607c94688" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.154:15021/healthz/ready\": dial tcp 10.133.0.154:15021: connect: connection refused"
Oct 30 22:49:17.188717 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:49:17.188689    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-single-main-88e685-openshift-default-94f47957-dn4cw"
Oct 30 22:49:17.861680 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:49:17.850755    5510 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-single-main-88e685/simulator-single-main-88e685-kserve-5cfcc964fd-4d4xn"]
Oct 30 22:49:17.871548 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:49:17.871516    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-single-main-88e685/simulator-single-main-88e685-kserve-5cfcc964fd-4d4xn"
Oct 30 22:49:17.913111 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:49:17.913083    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/491c333b-c990-4332-930c-a1fe707644ca-dshm\") pod \"simulator-single-main-88e685-kserve-5cfcc964fd-4d4xn\" (UID: \"491c333b-c990-4332-930c-a1fe707644ca\") " pod="e2e-models-e2e-worker-single-main-88e685/simulator-single-main-88e685-kserve-5cfcc964fd-4d4xn"
Oct 30 22:49:17.913326 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:49:17.913206    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/491c333b-c990-4332-930c-a1fe707644ca-model-cache\") pod \"simulator-single-main-88e685-kserve-5cfcc964fd-4d4xn\" (UID: \"491c333b-c990-4332-930c-a1fe707644ca\") " pod="e2e-models-e2e-worker-single-main-88e685/simulator-single-main-88e685-kserve-5cfcc964fd-4d4xn"
Oct 30 22:49:17.913486 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:49:17.913327    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/491c333b-c990-4332-930c-a1fe707644ca-tls-certs\") pod \"simulator-single-main-88e685-kserve-5cfcc964fd-4d4xn\" (UID: \"491c333b-c990-4332-930c-a1fe707644ca\") " pod="e2e-models-e2e-worker-single-main-88e685/simulator-single-main-88e685-kserve-5cfcc964fd-4d4xn"
Oct 30 22:49:17.913486 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:49:17.913383    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9fgw2\" (UniqueName: \"kubernetes.io/projected/491c333b-c990-4332-930c-a1fe707644ca-kube-api-access-9fgw2\") pod \"simulator-single-main-88e685-kserve-5cfcc964fd-4d4xn\" (UID: \"491c333b-c990-4332-930c-a1fe707644ca\") " pod="e2e-models-e2e-worker-single-main-88e685/simulator-single-main-88e685-kserve-5cfcc964fd-4d4xn"
Oct 30 22:49:17.913486 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:49:17.913451    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/491c333b-c990-4332-930c-a1fe707644ca-tmp-dir\") pod \"simulator-single-main-88e685-kserve-5cfcc964fd-4d4xn\" (UID: \"491c333b-c990-4332-930c-a1fe707644ca\") " pod="e2e-models-e2e-worker-single-main-88e685/simulator-single-main-88e685-kserve-5cfcc964fd-4d4xn"
Oct 30 22:49:17.913775 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:49:17.913553    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/491c333b-c990-4332-930c-a1fe707644ca-home\") pod \"simulator-single-main-88e685-kserve-5cfcc964fd-4d4xn\" (UID: \"491c333b-c990-4332-930c-a1fe707644ca\") " pod="e2e-models-e2e-worker-single-main-88e685/simulator-single-main-88e685-kserve-5cfcc964fd-4d4xn"
Oct 30 22:49:17.915892 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:49:17.915866    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-single-main-88e685/simulator-single-main-88e685-kserve-5cfcc964fd-4d4xn"]
Oct 30 22:49:17.968000 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:49:17.967963    5510 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-single-main-88e685\"/\"kube-root-ca.crt\""
Oct 30 22:49:17.968744 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:49:17.968428    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-single-main-88e685\"/\"simulator-single-main-88e685-kserve-self-signed-certs\""
Oct 30 22:49:17.968744 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:49:17.968549    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-single-main-88e685\"/\"default-dockercfg-jhwq2\""
Oct 30 22:49:17.968911 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:49:17.968803    5510 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-single-main-88e685\"/\"openshift-service-ca.crt\""
Oct 30 22:49:17.981099 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:49:17.981071    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:49:18.014725 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:49:18.014693    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/491c333b-c990-4332-930c-a1fe707644ca-model-cache\") pod \"simulator-single-main-88e685-kserve-5cfcc964fd-4d4xn\" (UID: \"491c333b-c990-4332-930c-a1fe707644ca\") " pod="e2e-models-e2e-worker-single-main-88e685/simulator-single-main-88e685-kserve-5cfcc964fd-4d4xn"
Oct 30 22:49:18.014924 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:49:18.014771    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/491c333b-c990-4332-930c-a1fe707644ca-tls-certs\") pod \"simulator-single-main-88e685-kserve-5cfcc964fd-4d4xn\" (UID: \"491c333b-c990-4332-930c-a1fe707644ca\") " pod="e2e-models-e2e-worker-single-main-88e685/simulator-single-main-88e685-kserve-5cfcc964fd-4d4xn"
Oct 30 22:49:18.014924 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:49:18.014803    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9fgw2\" (UniqueName: \"kubernetes.io/projected/491c333b-c990-4332-930c-a1fe707644ca-kube-api-access-9fgw2\") pod \"simulator-single-main-88e685-kserve-5cfcc964fd-4d4xn\" (UID: \"491c333b-c990-4332-930c-a1fe707644ca\") " pod="e2e-models-e2e-worker-single-main-88e685/simulator-single-main-88e685-kserve-5cfcc964fd-4d4xn"
Oct 30 22:49:18.014924 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:49:18.014825    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/491c333b-c990-4332-930c-a1fe707644ca-tmp-dir\") pod \"simulator-single-main-88e685-kserve-5cfcc964fd-4d4xn\" (UID: \"491c333b-c990-4332-930c-a1fe707644ca\") " pod="e2e-models-e2e-worker-single-main-88e685/simulator-single-main-88e685-kserve-5cfcc964fd-4d4xn"
Oct 30 22:49:18.014924 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:49:18.014868    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/491c333b-c990-4332-930c-a1fe707644ca-home\") pod \"simulator-single-main-88e685-kserve-5cfcc964fd-4d4xn\" (UID: \"491c333b-c990-4332-930c-a1fe707644ca\") " pod="e2e-models-e2e-worker-single-main-88e685/simulator-single-main-88e685-kserve-5cfcc964fd-4d4xn"
Oct 30 22:49:18.014924 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:49:18.014900    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/491c333b-c990-4332-930c-a1fe707644ca-dshm\") pod \"simulator-single-main-88e685-kserve-5cfcc964fd-4d4xn\" (UID: \"491c333b-c990-4332-930c-a1fe707644ca\") " pod="e2e-models-e2e-worker-single-main-88e685/simulator-single-main-88e685-kserve-5cfcc964fd-4d4xn"
Oct 30 22:49:18.015234 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:49:18.015174    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/491c333b-c990-4332-930c-a1fe707644ca-model-cache\") pod \"simulator-single-main-88e685-kserve-5cfcc964fd-4d4xn\" (UID: \"491c333b-c990-4332-930c-a1fe707644ca\") " pod="e2e-models-e2e-worker-single-main-88e685/simulator-single-main-88e685-kserve-5cfcc964fd-4d4xn"
Oct 30 22:49:18.015234 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:49:18.015194    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/491c333b-c990-4332-930c-a1fe707644ca-tmp-dir\") pod \"simulator-single-main-88e685-kserve-5cfcc964fd-4d4xn\" (UID: \"491c333b-c990-4332-930c-a1fe707644ca\") " pod="e2e-models-e2e-worker-single-main-88e685/simulator-single-main-88e685-kserve-5cfcc964fd-4d4xn"
Oct 30 22:49:18.015308 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:49:18.015259    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/491c333b-c990-4332-930c-a1fe707644ca-home\") pod \"simulator-single-main-88e685-kserve-5cfcc964fd-4d4xn\" (UID: \"491c333b-c990-4332-930c-a1fe707644ca\") " pod="e2e-models-e2e-worker-single-main-88e685/simulator-single-main-88e685-kserve-5cfcc964fd-4d4xn"
Oct 30 22:49:18.017103 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:49:18.017085    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/491c333b-c990-4332-930c-a1fe707644ca-dshm\") pod \"simulator-single-main-88e685-kserve-5cfcc964fd-4d4xn\" (UID: \"491c333b-c990-4332-930c-a1fe707644ca\") " pod="e2e-models-e2e-worker-single-main-88e685/simulator-single-main-88e685-kserve-5cfcc964fd-4d4xn"
Oct 30 22:49:18.017359 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:49:18.017342    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/491c333b-c990-4332-930c-a1fe707644ca-tls-certs\") pod \"simulator-single-main-88e685-kserve-5cfcc964fd-4d4xn\" (UID: \"491c333b-c990-4332-930c-a1fe707644ca\") " pod="e2e-models-e2e-worker-single-main-88e685/simulator-single-main-88e685-kserve-5cfcc964fd-4d4xn"
Oct 30 22:49:18.073705 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:49:18.073635    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9fgw2\" (UniqueName: \"kubernetes.io/projected/491c333b-c990-4332-930c-a1fe707644ca-kube-api-access-9fgw2\") pod \"simulator-single-main-88e685-kserve-5cfcc964fd-4d4xn\" (UID: \"491c333b-c990-4332-930c-a1fe707644ca\") " pod="e2e-models-e2e-worker-single-main-88e685/simulator-single-main-88e685-kserve-5cfcc964fd-4d4xn"
Oct 30 22:49:18.168742 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:49:18.168122    5510 patch_prober.go:29] interesting pod/e2e-worker-single-main-88e685-openshift-default-94f47957-dn4cw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.154:15021/healthz/ready\": dial tcp 10.133.0.154:15021: connect: connection refused" start-of-body=
Oct 30 22:49:18.168742 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:49:18.168195    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-88e685-openshift-default-94f47957-dn4cw" podUID="31c93ac4-77c9-4231-845d-b13607c94688" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.154:15021/healthz/ready\": dial tcp 10.133.0.154:15021: connect: connection refused"
Oct 30 22:49:18.184458 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:49:18.184415    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-single-main-88e685/simulator-single-main-88e685-kserve-5cfcc964fd-4d4xn"
Oct 30 22:49:18.570129 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:49:18.569111    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-single-main-88e685/simulator-single-main-88e685-kserve-5cfcc964fd-4d4xn"]
Oct 30 22:49:18.572887 ip-10-0-136-210 kubenswrapper[5510]: W1030 22:49:18.572849    5510 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod491c333b_c990_4332_930c_a1fe707644ca.slice/crio-a5d0c7e6232b299dde153c1f7dbb5f8beb8cd94fb90196d1493e8882f57d122e WatchSource:0}: Error finding container a5d0c7e6232b299dde153c1f7dbb5f8beb8cd94fb90196d1493e8882f57d122e: Status 404 returned error can't find the container with id a5d0c7e6232b299dde153c1f7dbb5f8beb8cd94fb90196d1493e8882f57d122e
Oct 30 22:49:19.169232 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:49:19.169130    5510 patch_prober.go:29] interesting pod/e2e-worker-single-main-88e685-openshift-default-94f47957-dn4cw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.154:15021/healthz/ready\": dial tcp 10.133.0.154:15021: connect: connection refused" start-of-body=
Oct 30 22:49:19.169232 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:49:19.169202    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-88e685-openshift-default-94f47957-dn4cw" podUID="31c93ac4-77c9-4231-845d-b13607c94688" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.154:15021/healthz/ready\": dial tcp 10.133.0.154:15021: connect: connection refused"
Oct 30 22:49:19.379894 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:49:19.379861    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-single-main-88e685/simulator-single-main-88e685-kserve-5cfcc964fd-4d4xn" event={"ID":"491c333b-c990-4332-930c-a1fe707644ca","Type":"ContainerStarted","Data":"d087eb35627ec82d556636db386151019b6fb942790b3ede2d668d82ef00bae6"}
Oct 30 22:49:19.379894 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:49:19.379900    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-single-main-88e685/simulator-single-main-88e685-kserve-5cfcc964fd-4d4xn" event={"ID":"491c333b-c990-4332-930c-a1fe707644ca","Type":"ContainerStarted","Data":"a5d0c7e6232b299dde153c1f7dbb5f8beb8cd94fb90196d1493e8882f57d122e"}
Oct 30 22:49:19.491056 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:49:19.490933    5510 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-single-main-88e685/simulator-single-main-88e685-kserve-5cfcc964fd-4d4xn" podStartSLOduration=2.490914875 podStartE2EDuration="2.490914875s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:49:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:49:19.490249643 +0000 UTC m=+3540.051059647" watchObservedRunningTime="2026-10-30 22:49:19.490914875 +0000 UTC m=+3540.051724881"
Oct 30 22:49:20.170057 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:49:20.170025    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-single-main-88e685-openshift-default-94f47957-dn4cw"
Oct 30 22:49:20.171059 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:49:20.171032    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-single-main-88e685-openshift-default-94f47957-dn4cw"
Oct 30 22:49:20.185147 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:49:20.185127    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-single-main-88e685/simulator-single-main-88e685-kserve-5cfcc964fd-4d4xn"
Oct 30 22:49:21.311677 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:49:21.311628    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:49:26.321569 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:49:26.321454    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:49:28.185060 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:49:28.185027    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-single-main-88e685/simulator-single-main-88e685-kserve-5cfcc964fd-4d4xn"
Oct 30 22:49:28.197793 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:49:28.197771    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-single-main-88e685/simulator-single-main-88e685-kserve-5cfcc964fd-4d4xn"
Oct 30 22:49:28.437976 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:49:28.437895    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-single-main-88e685/simulator-single-main-88e685-kserve-5cfcc964fd-4d4xn"
Oct 30 22:49:31.222004 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:49:31.221939    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:49:36.122648 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:49:36.122599    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:49:38.419107 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:49:38.419066    5510 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6gjbf" containerName="extractor"
Oct 30 22:49:38.558077 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:49:38.558039    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:49:40.974971 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:49:40.974931    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:49:45.928995 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:49:45.928954    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:49:50.820052 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:49:50.820011    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:49:55.819292 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:49:55.819254    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:50:00.735775 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:50:00.735731    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:50:05.616331 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:50:05.616286    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:50:08.419866 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:50:08.419830    5510 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6gjbf" containerName="extractor"
Oct 30 22:50:09.150776 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:50:09.150736    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:50:10.584062 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:50:10.584021    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:50:14.063589 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:50:14.063553    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-single-main-88e685/simulator-single-main-88e685-kserve-5cfcc964fd-4d4xn"]
Oct 30 22:50:14.189537 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:50:14.189500    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:50:15.915266 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:50:15.915203    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:50:20.017187 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:50:20.017152    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-88e685-77594c5967zjvx"]
Oct 30 22:50:20.118783 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:50:20.118734    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-88e685-59954dqthn"]
Oct 30 22:50:20.428086 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:50:20.427993    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:50:20.620100 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:50:20.620061    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:50:20.838875 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:50:20.838833    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:50:24.972338 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:50:24.972302    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-88e685-69dbb8b6bb-29796"]
Oct 30 22:50:24.972962 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:50:24.972515    5510 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-88e685-69dbb8b6bb-29796" podUID="fc669e80-db80-4479-93d5-79c6651b405d" containerName="maas-api" containerID="cri-o://7327acd8f1b3e94c968235bb4d86f905fedb0b6e162aec5e601f36bd1863ba90" gracePeriod=30
Oct 30 22:50:25.018190 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:50:25.018142    5510 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-single-main-88e685-77594c5967zjvx" podUID="920155b2-a008-4621-8019-de154b8a1fea" containerName="payload-processing" containerID="cri-o://d5e58440d61fe8b4c597310353441f773842c2cc873c8789c76a2999a75d142e" gracePeriod=25
Oct 30 22:50:25.116599 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:50:25.116564    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:50:25.119379 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:50:25.119335    5510 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-88e685-59954dqthn" podUID="acebd8f6-36f7-4152-9154-0b8b186d2a81" containerName="payload-pre-processing" containerID="cri-o://d617393d0278be031079d40dbf5f79c7b35d653e5174e75ca4bde672e2c921fb" gracePeriod=25
Oct 30 22:50:25.230959 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:50:25.230863    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-single-main-88e685-77594c5967zjvx" podUID="920155b2-a008-4621-8019-de154b8a1fea" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:50:25.231122 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:50:25.230882    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-88e685-59954dqthn" podUID="acebd8f6-36f7-4152-9154-0b8b186d2a81" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:50:26.213642 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:50:26.213611    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-88e685-69dbb8b6bb-29796"
Oct 30 22:50:26.215762 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:50:26.215742    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/fc669e80-db80-4479-93d5-79c6651b405d-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fc669e80-db80-4479-93d5-79c6651b405d-maas-api-tls\") pod \"fc669e80-db80-4479-93d5-79c6651b405d\" (UID: \"fc669e80-db80-4479-93d5-79c6651b405d\") "
Oct 30 22:50:26.215829 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:50:26.215788    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fc669e80-db80-4479-93d5-79c6651b405d-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fc669e80-db80-4479-93d5-79c6651b405d-openshift-service-ca\") pod \"fc669e80-db80-4479-93d5-79c6651b405d\" (UID: \"fc669e80-db80-4479-93d5-79c6651b405d\") "
Oct 30 22:50:26.215876 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:50:26.215862    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fc669e80-db80-4479-93d5-79c6651b405d-kube-api-access-vkpb4\" (UniqueName: \"kubernetes.io/projected/fc669e80-db80-4479-93d5-79c6651b405d-kube-api-access-vkpb4\") pod \"fc669e80-db80-4479-93d5-79c6651b405d\" (UID: \"fc669e80-db80-4479-93d5-79c6651b405d\") "
Oct 30 22:50:26.215920 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:50:26.215894    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/fc669e80-db80-4479-93d5-79c6651b405d-metrics-certs\" (UniqueName: \"kubernetes.io/secret/fc669e80-db80-4479-93d5-79c6651b405d-metrics-certs\") pod \"fc669e80-db80-4479-93d5-79c6651b405d\" (UID: \"fc669e80-db80-4479-93d5-79c6651b405d\") "
Oct 30 22:50:26.216219 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:50:26.216194    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fc669e80-db80-4479-93d5-79c6651b405d-openshift-service-ca" pod "fc669e80-db80-4479-93d5-79c6651b405d" (UID: "fc669e80-db80-4479-93d5-79c6651b405d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:50:26.217930 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:50:26.217898    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fc669e80-db80-4479-93d5-79c6651b405d-metrics-certs" pod "fc669e80-db80-4479-93d5-79c6651b405d" (UID: "fc669e80-db80-4479-93d5-79c6651b405d"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:50:26.218043 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:50:26.217982    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fc669e80-db80-4479-93d5-79c6651b405d-maas-api-tls" pod "fc669e80-db80-4479-93d5-79c6651b405d" (UID: "fc669e80-db80-4479-93d5-79c6651b405d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:50:26.218092 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:50:26.218057    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fc669e80-db80-4479-93d5-79c6651b405d-kube-api-access-vkpb4" pod "fc669e80-db80-4479-93d5-79c6651b405d" (UID: "fc669e80-db80-4479-93d5-79c6651b405d"). InnerVolumeSpecName "kube-api-access-vkpb4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:50:26.316865 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:50:26.316771    5510 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fc669e80-db80-4479-93d5-79c6651b405d-openshift-service-ca\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:50:26.316865 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:50:26.316800    5510 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vkpb4\" (UniqueName: \"kubernetes.io/projected/fc669e80-db80-4479-93d5-79c6651b405d-kube-api-access-vkpb4\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:50:26.316865 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:50:26.316823    5510 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fc669e80-db80-4479-93d5-79c6651b405d-metrics-certs\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:50:26.316865 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:50:26.316837    5510 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fc669e80-db80-4479-93d5-79c6651b405d-maas-api-tls\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:50:26.685808 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:50:26.685710    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:50:26.722583 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:50:26.722541    5510 generic.go:356] "Generic (PLEG): container finished" podID="fc669e80-db80-4479-93d5-79c6651b405d" containerID="7327acd8f1b3e94c968235bb4d86f905fedb0b6e162aec5e601f36bd1863ba90" exitCode=0
Oct 30 22:50:26.722798 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:50:26.722596    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-88e685-69dbb8b6bb-29796"
Oct 30 22:50:26.722798 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:50:26.722604    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-88e685-69dbb8b6bb-29796" event={"ID":"fc669e80-db80-4479-93d5-79c6651b405d","Type":"ContainerDied","Data":"7327acd8f1b3e94c968235bb4d86f905fedb0b6e162aec5e601f36bd1863ba90"}
Oct 30 22:50:26.722798 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:50:26.722670    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-88e685-69dbb8b6bb-29796" event={"ID":"fc669e80-db80-4479-93d5-79c6651b405d","Type":"ContainerDied","Data":"a7ab5577b1bd76f2c2dcba5692fb6381da13d56b1bddab7bf54a9e3be174e00c"}
Oct 30 22:50:26.722798 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:50:26.722691    5510 scope.go:122] "RemoveContainer" containerID="7327acd8f1b3e94c968235bb4d86f905fedb0b6e162aec5e601f36bd1863ba90"
Oct 30 22:50:26.732817 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:50:26.732797    5510 scope.go:122] "RemoveContainer" containerID="7327acd8f1b3e94c968235bb4d86f905fedb0b6e162aec5e601f36bd1863ba90"
Oct 30 22:50:26.733052 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:50:26.733031    5510 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7327acd8f1b3e94c968235bb4d86f905fedb0b6e162aec5e601f36bd1863ba90\": container with ID starting with 7327acd8f1b3e94c968235bb4d86f905fedb0b6e162aec5e601f36bd1863ba90 not found: ID does not exist" containerID="7327acd8f1b3e94c968235bb4d86f905fedb0b6e162aec5e601f36bd1863ba90"
Oct 30 22:50:26.733104 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:50:26.733061    5510 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7327acd8f1b3e94c968235bb4d86f905fedb0b6e162aec5e601f36bd1863ba90"} err="failed to get container status \"7327acd8f1b3e94c968235bb4d86f905fedb0b6e162aec5e601f36bd1863ba90\": rpc error: code = NotFound desc = could not find container \"7327acd8f1b3e94c968235bb4d86f905fedb0b6e162aec5e601f36bd1863ba90\": container with ID starting with 7327acd8f1b3e94c968235bb4d86f905fedb0b6e162aec5e601f36bd1863ba90 not found: ID does not exist"
Oct 30 22:50:26.879430 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:50:26.879381    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-88e685-69dbb8b6bb-29796"]
Oct 30 22:50:26.897050 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:50:26.897021    5510 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-88e685-69dbb8b6bb-29796"]
Oct 30 22:50:26.968610 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:50:26.968523    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:50:27.552951 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:50:27.552912    5510 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fc669e80-db80-4479-93d5-79c6651b405d" path="/var/lib/kubelet/pods/fc669e80-db80-4479-93d5-79c6651b405d/volumes"
Oct 30 22:50:27.786576 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:50:27.786542    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:50:29.083898 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:50:29.083850    5510 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-single-main-88e685/simulator-single-main-88e685-kserve-5cfcc964fd-4d4xn" podUID="491c333b-c990-4332-930c-a1fe707644ca" containerName="main" containerID="cri-o://d087eb35627ec82d556636db386151019b6fb942790b3ede2d668d82ef00bae6" gracePeriod=45
Oct 30 22:50:29.339291 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:50:29.339226    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-single-main-88e685/simulator-single-main-88e685-kserve-5cfcc964fd-4d4xn"
Oct 30 22:50:29.347557 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:50:29.347536    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/491c333b-c990-4332-930c-a1fe707644ca-home\" (UniqueName: \"kubernetes.io/empty-dir/491c333b-c990-4332-930c-a1fe707644ca-home\") pod \"491c333b-c990-4332-930c-a1fe707644ca\" (UID: \"491c333b-c990-4332-930c-a1fe707644ca\") "
Oct 30 22:50:29.347632 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:50:29.347592    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/491c333b-c990-4332-930c-a1fe707644ca-tls-certs\" (UniqueName: \"kubernetes.io/secret/491c333b-c990-4332-930c-a1fe707644ca-tls-certs\") pod \"491c333b-c990-4332-930c-a1fe707644ca\" (UID: \"491c333b-c990-4332-930c-a1fe707644ca\") "
Oct 30 22:50:29.347632 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:50:29.347620    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/491c333b-c990-4332-930c-a1fe707644ca-kube-api-access-9fgw2\" (UniqueName: \"kubernetes.io/projected/491c333b-c990-4332-930c-a1fe707644ca-kube-api-access-9fgw2\") pod \"491c333b-c990-4332-930c-a1fe707644ca\" (UID: \"491c333b-c990-4332-930c-a1fe707644ca\") "
Oct 30 22:50:29.347779 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:50:29.347758    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/491c333b-c990-4332-930c-a1fe707644ca-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/491c333b-c990-4332-930c-a1fe707644ca-model-cache\") pod \"491c333b-c990-4332-930c-a1fe707644ca\" (UID: \"491c333b-c990-4332-930c-a1fe707644ca\") "
Oct 30 22:50:29.347852 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:50:29.347828    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/491c333b-c990-4332-930c-a1fe707644ca-home" pod "491c333b-c990-4332-930c-a1fe707644ca" (UID: "491c333b-c990-4332-930c-a1fe707644ca"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:50:29.347914 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:50:29.347855    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/491c333b-c990-4332-930c-a1fe707644ca-dshm\" (UniqueName: \"kubernetes.io/empty-dir/491c333b-c990-4332-930c-a1fe707644ca-dshm\") pod \"491c333b-c990-4332-930c-a1fe707644ca\" (UID: \"491c333b-c990-4332-930c-a1fe707644ca\") "
Oct 30 22:50:29.347914 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:50:29.347889    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/491c333b-c990-4332-930c-a1fe707644ca-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/491c333b-c990-4332-930c-a1fe707644ca-tmp-dir\") pod \"491c333b-c990-4332-930c-a1fe707644ca\" (UID: \"491c333b-c990-4332-930c-a1fe707644ca\") "
Oct 30 22:50:29.348034 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:50:29.348008    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/491c333b-c990-4332-930c-a1fe707644ca-model-cache" pod "491c333b-c990-4332-930c-a1fe707644ca" (UID: "491c333b-c990-4332-930c-a1fe707644ca"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:50:29.348180 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:50:29.348153    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/491c333b-c990-4332-930c-a1fe707644ca-tmp-dir" pod "491c333b-c990-4332-930c-a1fe707644ca" (UID: "491c333b-c990-4332-930c-a1fe707644ca"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:50:29.348279 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:50:29.348263    5510 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/491c333b-c990-4332-930c-a1fe707644ca-tmp-dir\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:50:29.348279 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:50:29.348276    5510 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/491c333b-c990-4332-930c-a1fe707644ca-home\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:50:29.348365 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:50:29.348289    5510 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/491c333b-c990-4332-930c-a1fe707644ca-model-cache\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:50:29.349913 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:50:29.349888    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/491c333b-c990-4332-930c-a1fe707644ca-tls-certs" pod "491c333b-c990-4332-930c-a1fe707644ca" (UID: "491c333b-c990-4332-930c-a1fe707644ca"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:50:29.350212 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:50:29.350189    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/491c333b-c990-4332-930c-a1fe707644ca-dshm" pod "491c333b-c990-4332-930c-a1fe707644ca" (UID: "491c333b-c990-4332-930c-a1fe707644ca"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:50:29.350301 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:50:29.350228    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/491c333b-c990-4332-930c-a1fe707644ca-kube-api-access-9fgw2" pod "491c333b-c990-4332-930c-a1fe707644ca" (UID: "491c333b-c990-4332-930c-a1fe707644ca"). InnerVolumeSpecName "kube-api-access-9fgw2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:50:29.449470 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:50:29.449440    5510 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/491c333b-c990-4332-930c-a1fe707644ca-dshm\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:50:29.449470 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:50:29.449467    5510 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/491c333b-c990-4332-930c-a1fe707644ca-tls-certs\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:50:29.449470 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:50:29.449476    5510 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9fgw2\" (UniqueName: \"kubernetes.io/projected/491c333b-c990-4332-930c-a1fe707644ca-kube-api-access-9fgw2\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:50:29.740817 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:50:29.740725    5510 generic.go:356] "Generic (PLEG): container finished" podID="491c333b-c990-4332-930c-a1fe707644ca" containerID="d087eb35627ec82d556636db386151019b6fb942790b3ede2d668d82ef00bae6" exitCode=0
Oct 30 22:50:29.740817 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:50:29.740765    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-single-main-88e685/simulator-single-main-88e685-kserve-5cfcc964fd-4d4xn" event={"ID":"491c333b-c990-4332-930c-a1fe707644ca","Type":"ContainerDied","Data":"d087eb35627ec82d556636db386151019b6fb942790b3ede2d668d82ef00bae6"}
Oct 30 22:50:29.740817 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:50:29.740780    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-single-main-88e685/simulator-single-main-88e685-kserve-5cfcc964fd-4d4xn"
Oct 30 22:50:29.740817 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:50:29.740798    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-single-main-88e685/simulator-single-main-88e685-kserve-5cfcc964fd-4d4xn" event={"ID":"491c333b-c990-4332-930c-a1fe707644ca","Type":"ContainerDied","Data":"a5d0c7e6232b299dde153c1f7dbb5f8beb8cd94fb90196d1493e8882f57d122e"}
Oct 30 22:50:29.740817 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:50:29.740814    5510 scope.go:122] "RemoveContainer" containerID="d087eb35627ec82d556636db386151019b6fb942790b3ede2d668d82ef00bae6"
Oct 30 22:50:29.749523 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:50:29.749505    5510 scope.go:122] "RemoveContainer" containerID="d087eb35627ec82d556636db386151019b6fb942790b3ede2d668d82ef00bae6"
Oct 30 22:50:29.749838 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:50:29.749820    5510 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d087eb35627ec82d556636db386151019b6fb942790b3ede2d668d82ef00bae6\": container with ID starting with d087eb35627ec82d556636db386151019b6fb942790b3ede2d668d82ef00bae6 not found: ID does not exist" containerID="d087eb35627ec82d556636db386151019b6fb942790b3ede2d668d82ef00bae6"
Oct 30 22:50:29.749921 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:50:29.749851    5510 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d087eb35627ec82d556636db386151019b6fb942790b3ede2d668d82ef00bae6"} err="failed to get container status \"d087eb35627ec82d556636db386151019b6fb942790b3ede2d668d82ef00bae6\": rpc error: code = NotFound desc = could not find container \"d087eb35627ec82d556636db386151019b6fb942790b3ede2d668d82ef00bae6\": container with ID starting with d087eb35627ec82d556636db386151019b6fb942790b3ede2d668d82ef00bae6 not found: ID does not exist"
Oct 30 22:50:29.896883 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:50:29.896850    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-single-main-88e685/simulator-single-main-88e685-kserve-5cfcc964fd-4d4xn"]
Oct 30 22:50:29.902310 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:50:29.902281    5510 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-single-main-88e685/simulator-single-main-88e685-kserve-5cfcc964fd-4d4xn"]
Oct 30 22:50:31.551576 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:50:31.551539    5510 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="491c333b-c990-4332-930c-a1fe707644ca" path="/var/lib/kubelet/pods/491c333b-c990-4332-930c-a1fe707644ca/volumes"
Oct 30 22:50:33.852983 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:50:33.852952    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:50:34.042424 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:50:34.042385    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:50:34.148303 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:50:34.148225    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-single-main-88e685-openshift-default-94f47957-dn4cw"]
Oct 30 22:50:34.148451 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:50:34.148428    5510 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-single-main-88e685-openshift-default-94f47957-dn4cw" podUID="31c93ac4-77c9-4231-845d-b13607c94688" containerName="istio-proxy" containerID="cri-o://d7e2ccbbb52bd53c00b5409705cfc10e0f1934b469a11a335608ef6168e10013" gracePeriod=30
Oct 30 22:50:34.245616 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:50:34.245577    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:50:35.173372 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:50:35.173279    5510 patch_prober.go:29] interesting pod/e2e-worker-single-main-88e685-openshift-default-94f47957-dn4cw container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 22:50:35.173372 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:50:35.173336    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-single-main-88e685-openshift-default-94f47957-dn4cw" podUID="31c93ac4-77c9-4231-845d-b13607c94688" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 22:50:35.231101 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:50:35.231060    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-single-main-88e685-77594c5967zjvx" podUID="920155b2-a008-4621-8019-de154b8a1fea" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:50:35.231292 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:50:35.231175    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-88e685-59954dqthn" podUID="acebd8f6-36f7-4152-9154-0b8b186d2a81" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:50:38.421921 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:50:38.421883    5510 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6gjbf" containerName="extractor"
Oct 30 22:50:38.888279 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:50:38.888235    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:50:39.403667 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:50:39.403632    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-single-main-88e685-openshift-default-94f47957-dn4cw"
Oct 30 22:50:39.451840 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:50:39.451758    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/31c93ac4-77c9-4231-845d-b13607c94688-istio-token\" (UniqueName: \"kubernetes.io/projected/31c93ac4-77c9-4231-845d-b13607c94688-istio-token\") pod \"31c93ac4-77c9-4231-845d-b13607c94688\" (UID: \"31c93ac4-77c9-4231-845d-b13607c94688\") "
Oct 30 22:50:39.451840 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:50:39.451819    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/31c93ac4-77c9-4231-845d-b13607c94688-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/31c93ac4-77c9-4231-845d-b13607c94688-workload-certs\") pod \"31c93ac4-77c9-4231-845d-b13607c94688\" (UID: \"31c93ac4-77c9-4231-845d-b13607c94688\") "
Oct 30 22:50:39.452393 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:50:39.451859    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/31c93ac4-77c9-4231-845d-b13607c94688-kube-api-access-sfb5r\" (UniqueName: \"kubernetes.io/projected/31c93ac4-77c9-4231-845d-b13607c94688-kube-api-access-sfb5r\") pod \"31c93ac4-77c9-4231-845d-b13607c94688\" (UID: \"31c93ac4-77c9-4231-845d-b13607c94688\") "
Oct 30 22:50:39.452393 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:50:39.451891    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/31c93ac4-77c9-4231-845d-b13607c94688-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/31c93ac4-77c9-4231-845d-b13607c94688-istio-data\") pod \"31c93ac4-77c9-4231-845d-b13607c94688\" (UID: \"31c93ac4-77c9-4231-845d-b13607c94688\") "
Oct 30 22:50:39.452393 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:50:39.451918    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/31c93ac4-77c9-4231-845d-b13607c94688-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/31c93ac4-77c9-4231-845d-b13607c94688-istio-podinfo\") pod \"31c93ac4-77c9-4231-845d-b13607c94688\" (UID: \"31c93ac4-77c9-4231-845d-b13607c94688\") "
Oct 30 22:50:39.452393 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:50:39.451979    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/31c93ac4-77c9-4231-845d-b13607c94688-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/31c93ac4-77c9-4231-845d-b13607c94688-workload-socket\") pod \"31c93ac4-77c9-4231-845d-b13607c94688\" (UID: \"31c93ac4-77c9-4231-845d-b13607c94688\") "
Oct 30 22:50:39.452393 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:50:39.452159    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/31c93ac4-77c9-4231-845d-b13607c94688-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/31c93ac4-77c9-4231-845d-b13607c94688-istiod-ca-cert\") pod \"31c93ac4-77c9-4231-845d-b13607c94688\" (UID: \"31c93ac4-77c9-4231-845d-b13607c94688\") "
Oct 30 22:50:39.452393 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:50:39.452219    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/31c93ac4-77c9-4231-845d-b13607c94688-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/31c93ac4-77c9-4231-845d-b13607c94688-credential-socket\") pod \"31c93ac4-77c9-4231-845d-b13607c94688\" (UID: \"31c93ac4-77c9-4231-845d-b13607c94688\") "
Oct 30 22:50:39.452393 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:50:39.452210    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/31c93ac4-77c9-4231-845d-b13607c94688-workload-certs" pod "31c93ac4-77c9-4231-845d-b13607c94688" (UID: "31c93ac4-77c9-4231-845d-b13607c94688"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:50:39.452393 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:50:39.452281    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/31c93ac4-77c9-4231-845d-b13607c94688-workload-socket" pod "31c93ac4-77c9-4231-845d-b13607c94688" (UID: "31c93ac4-77c9-4231-845d-b13607c94688"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:50:39.452393 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:50:39.452309    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/31c93ac4-77c9-4231-845d-b13607c94688-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/31c93ac4-77c9-4231-845d-b13607c94688-istio-envoy\") pod \"31c93ac4-77c9-4231-845d-b13607c94688\" (UID: \"31c93ac4-77c9-4231-845d-b13607c94688\") "
Oct 30 22:50:39.452949 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:50:39.452469    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/31c93ac4-77c9-4231-845d-b13607c94688-istiod-ca-cert" pod "31c93ac4-77c9-4231-845d-b13607c94688" (UID: "31c93ac4-77c9-4231-845d-b13607c94688"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:50:39.452949 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:50:39.452525    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/31c93ac4-77c9-4231-845d-b13607c94688-credential-socket" pod "31c93ac4-77c9-4231-845d-b13607c94688" (UID: "31c93ac4-77c9-4231-845d-b13607c94688"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:50:39.452949 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:50:39.452599    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/31c93ac4-77c9-4231-845d-b13607c94688-istio-data" pod "31c93ac4-77c9-4231-845d-b13607c94688" (UID: "31c93ac4-77c9-4231-845d-b13607c94688"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:50:39.452949 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:50:39.452782    5510 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/31c93ac4-77c9-4231-845d-b13607c94688-istiod-ca-cert\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:50:39.452949 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:50:39.452796    5510 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/31c93ac4-77c9-4231-845d-b13607c94688-credential-socket\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:50:39.452949 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:50:39.452805    5510 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/31c93ac4-77c9-4231-845d-b13607c94688-workload-certs\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:50:39.452949 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:50:39.452814    5510 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/31c93ac4-77c9-4231-845d-b13607c94688-istio-data\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:50:39.452949 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:50:39.452827    5510 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/31c93ac4-77c9-4231-845d-b13607c94688-workload-socket\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:50:39.454127 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:50:39.454100    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/31c93ac4-77c9-4231-845d-b13607c94688-istio-token" pod "31c93ac4-77c9-4231-845d-b13607c94688" (UID: "31c93ac4-77c9-4231-845d-b13607c94688"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:50:39.454243 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:50:39.454200    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/31c93ac4-77c9-4231-845d-b13607c94688-kube-api-access-sfb5r" pod "31c93ac4-77c9-4231-845d-b13607c94688" (UID: "31c93ac4-77c9-4231-845d-b13607c94688"). InnerVolumeSpecName "kube-api-access-sfb5r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:50:39.454529 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:50:39.454505    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/31c93ac4-77c9-4231-845d-b13607c94688-istio-podinfo" pod "31c93ac4-77c9-4231-845d-b13607c94688" (UID: "31c93ac4-77c9-4231-845d-b13607c94688"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:50:39.454640 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:50:39.454617    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/31c93ac4-77c9-4231-845d-b13607c94688-istio-envoy" pod "31c93ac4-77c9-4231-845d-b13607c94688" (UID: "31c93ac4-77c9-4231-845d-b13607c94688"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:50:39.553647 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:50:39.553612    5510 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/31c93ac4-77c9-4231-845d-b13607c94688-istio-token\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:50:39.553647 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:50:39.553643    5510 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sfb5r\" (UniqueName: \"kubernetes.io/projected/31c93ac4-77c9-4231-845d-b13607c94688-kube-api-access-sfb5r\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:50:39.553878 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:50:39.553681    5510 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/31c93ac4-77c9-4231-845d-b13607c94688-istio-podinfo\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:50:39.553878 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:50:39.553695    5510 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/31c93ac4-77c9-4231-845d-b13607c94688-istio-envoy\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:50:39.794670 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:50:39.794617    5510 generic.go:356] "Generic (PLEG): container finished" podID="31c93ac4-77c9-4231-845d-b13607c94688" containerID="d7e2ccbbb52bd53c00b5409705cfc10e0f1934b469a11a335608ef6168e10013" exitCode=0
Oct 30 22:50:39.794852 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:50:39.794691    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-single-main-88e685-openshift-default-94f47957-dn4cw"
Oct 30 22:50:39.794852 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:50:39.794744    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-88e685-openshift-default-94f47957-dn4cw" event={"ID":"31c93ac4-77c9-4231-845d-b13607c94688","Type":"ContainerDied","Data":"d7e2ccbbb52bd53c00b5409705cfc10e0f1934b469a11a335608ef6168e10013"}
Oct 30 22:50:39.794852 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:50:39.794795    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-88e685-openshift-default-94f47957-dn4cw" event={"ID":"31c93ac4-77c9-4231-845d-b13607c94688","Type":"ContainerDied","Data":"bfa7cb1cd8b9b167552c1832b566faa328ed68e9af315da42ab25beebe54e102"}
Oct 30 22:50:39.794852 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:50:39.794816    5510 scope.go:122] "RemoveContainer" containerID="d7e2ccbbb52bd53c00b5409705cfc10e0f1934b469a11a335608ef6168e10013"
Oct 30 22:50:39.803482 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:50:39.803466    5510 scope.go:122] "RemoveContainer" containerID="b1e4702ea8b25b90b10aa8ee9331855d05c3639329595cb9c429d2bd2ab280f6"
Oct 30 22:50:39.811248 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:50:39.811230    5510 scope.go:122] "RemoveContainer" containerID="d7e2ccbbb52bd53c00b5409705cfc10e0f1934b469a11a335608ef6168e10013"
Oct 30 22:50:39.811513 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:50:39.811492    5510 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d7e2ccbbb52bd53c00b5409705cfc10e0f1934b469a11a335608ef6168e10013\": container with ID starting with d7e2ccbbb52bd53c00b5409705cfc10e0f1934b469a11a335608ef6168e10013 not found: ID does not exist" containerID="d7e2ccbbb52bd53c00b5409705cfc10e0f1934b469a11a335608ef6168e10013"
Oct 30 22:50:39.811554 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:50:39.811523    5510 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d7e2ccbbb52bd53c00b5409705cfc10e0f1934b469a11a335608ef6168e10013"} err="failed to get container status \"d7e2ccbbb52bd53c00b5409705cfc10e0f1934b469a11a335608ef6168e10013\": rpc error: code = NotFound desc = could not find container \"d7e2ccbbb52bd53c00b5409705cfc10e0f1934b469a11a335608ef6168e10013\": container with ID starting with d7e2ccbbb52bd53c00b5409705cfc10e0f1934b469a11a335608ef6168e10013 not found: ID does not exist"
Oct 30 22:50:39.811554 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:50:39.811541    5510 scope.go:122] "RemoveContainer" containerID="b1e4702ea8b25b90b10aa8ee9331855d05c3639329595cb9c429d2bd2ab280f6"
Oct 30 22:50:39.811805 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:50:39.811784    5510 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b1e4702ea8b25b90b10aa8ee9331855d05c3639329595cb9c429d2bd2ab280f6\": container with ID starting with b1e4702ea8b25b90b10aa8ee9331855d05c3639329595cb9c429d2bd2ab280f6 not found: ID does not exist" containerID="b1e4702ea8b25b90b10aa8ee9331855d05c3639329595cb9c429d2bd2ab280f6"
Oct 30 22:50:39.811845 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:50:39.811812    5510 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b1e4702ea8b25b90b10aa8ee9331855d05c3639329595cb9c429d2bd2ab280f6"} err="failed to get container status \"b1e4702ea8b25b90b10aa8ee9331855d05c3639329595cb9c429d2bd2ab280f6\": rpc error: code = NotFound desc = could not find container \"b1e4702ea8b25b90b10aa8ee9331855d05c3639329595cb9c429d2bd2ab280f6\": container with ID starting with b1e4702ea8b25b90b10aa8ee9331855d05c3639329595cb9c429d2bd2ab280f6 not found: ID does not exist"
Oct 30 22:50:39.951032 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:50:39.950996    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-single-main-88e685-openshift-default-94f47957-dn4cw"]
Oct 30 22:50:39.997543 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:50:39.997511    5510 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-single-main-88e685-openshift-default-94f47957-dn4cw"]
Oct 30 22:50:41.551698 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:50:41.551641    5510 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="31c93ac4-77c9-4231-845d-b13607c94688" path="/var/lib/kubelet/pods/31c93ac4-77c9-4231-845d-b13607c94688/volumes"
Oct 30 22:50:45.230909 ip-10-0-136-210 kubenswrapper[5510]: W1030 22:50:45.230874    5510 logging.go:55] [core] [Channel #6783 SubChannel #6785]grpc: addrConn.createTransport failed to connect to {Addr: "10.133.0.155:50052", ServerName: "10.133.0.155:50052", BalancerAttributes: {"<%!p(pickfirstleaf.managedByPickfirstKeyType={})>": "<%!p(bool=true)>" }}. Err: connection error: desc = "transport: Error while dialing: dial tcp 10.133.0.155:50052: connect: connection refused"
Oct 30 22:50:45.272145 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:50:45.272120    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-single-main-88e685-77594c5967zjvx"
Oct 30 22:50:45.368337 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:50:45.368312    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-88e685-59954dqthn"
Oct 30 22:50:45.410957 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:50:45.410929    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/920155b2-a008-4621-8019-de154b8a1fea-kube-api-access-2p2sg\" (UniqueName: \"kubernetes.io/projected/920155b2-a008-4621-8019-de154b8a1fea-kube-api-access-2p2sg\") pod \"920155b2-a008-4621-8019-de154b8a1fea\" (UID: \"920155b2-a008-4621-8019-de154b8a1fea\") "
Oct 30 22:50:45.411149 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:50:45.411104    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/920155b2-a008-4621-8019-de154b8a1fea-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/920155b2-a008-4621-8019-de154b8a1fea-plugins-config-volume\") pod \"920155b2-a008-4621-8019-de154b8a1fea\" (UID: \"920155b2-a008-4621-8019-de154b8a1fea\") "
Oct 30 22:50:45.411529 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:50:45.411499    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/920155b2-a008-4621-8019-de154b8a1fea-plugins-config-volume" pod "920155b2-a008-4621-8019-de154b8a1fea" (UID: "920155b2-a008-4621-8019-de154b8a1fea"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:50:45.413019 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:50:45.413000    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/920155b2-a008-4621-8019-de154b8a1fea-kube-api-access-2p2sg" pod "920155b2-a008-4621-8019-de154b8a1fea" (UID: "920155b2-a008-4621-8019-de154b8a1fea"). InnerVolumeSpecName "kube-api-access-2p2sg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:50:45.512166 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:50:45.512121    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/acebd8f6-36f7-4152-9154-0b8b186d2a81-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/acebd8f6-36f7-4152-9154-0b8b186d2a81-plugins-config-volume\") pod \"acebd8f6-36f7-4152-9154-0b8b186d2a81\" (UID: \"acebd8f6-36f7-4152-9154-0b8b186d2a81\") "
Oct 30 22:50:45.512334 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:50:45.512275    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/acebd8f6-36f7-4152-9154-0b8b186d2a81-kube-api-access-fbvcq\" (UniqueName: \"kubernetes.io/projected/acebd8f6-36f7-4152-9154-0b8b186d2a81-kube-api-access-fbvcq\") pod \"acebd8f6-36f7-4152-9154-0b8b186d2a81\" (UID: \"acebd8f6-36f7-4152-9154-0b8b186d2a81\") "
Oct 30 22:50:45.512565 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:50:45.512532    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/acebd8f6-36f7-4152-9154-0b8b186d2a81-plugins-config-volume" pod "acebd8f6-36f7-4152-9154-0b8b186d2a81" (UID: "acebd8f6-36f7-4152-9154-0b8b186d2a81"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:50:45.512709 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:50:45.512690    5510 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/920155b2-a008-4621-8019-de154b8a1fea-plugins-config-volume\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:50:45.512709 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:50:45.512705    5510 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/acebd8f6-36f7-4152-9154-0b8b186d2a81-plugins-config-volume\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:50:45.512804 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:50:45.512714    5510 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2p2sg\" (UniqueName: \"kubernetes.io/projected/920155b2-a008-4621-8019-de154b8a1fea-kube-api-access-2p2sg\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:50:45.514326 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:50:45.514309    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/acebd8f6-36f7-4152-9154-0b8b186d2a81-kube-api-access-fbvcq" pod "acebd8f6-36f7-4152-9154-0b8b186d2a81" (UID: "acebd8f6-36f7-4152-9154-0b8b186d2a81"). InnerVolumeSpecName "kube-api-access-fbvcq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:50:45.614207 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:50:45.614166    5510 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fbvcq\" (UniqueName: \"kubernetes.io/projected/acebd8f6-36f7-4152-9154-0b8b186d2a81-kube-api-access-fbvcq\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:50:45.825298 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:50:45.825212    5510 generic.go:356] "Generic (PLEG): container finished" podID="acebd8f6-36f7-4152-9154-0b8b186d2a81" containerID="d617393d0278be031079d40dbf5f79c7b35d653e5174e75ca4bde672e2c921fb" exitCode=0
Oct 30 22:50:45.825298 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:50:45.825255    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-88e685-59954dqthn"
Oct 30 22:50:45.825501 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:50:45.825309    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-88e685-59954dqthn" event={"ID":"acebd8f6-36f7-4152-9154-0b8b186d2a81","Type":"ContainerDied","Data":"d617393d0278be031079d40dbf5f79c7b35d653e5174e75ca4bde672e2c921fb"}
Oct 30 22:50:45.825501 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:50:45.825366    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-88e685-59954dqthn" event={"ID":"acebd8f6-36f7-4152-9154-0b8b186d2a81","Type":"ContainerDied","Data":"f5868bb368a69f2ff875c8f9d9901ef8842bf289f77df84994021f487a1fdcbb"}
Oct 30 22:50:45.825501 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:50:45.825390    5510 scope.go:122] "RemoveContainer" containerID="d617393d0278be031079d40dbf5f79c7b35d653e5174e75ca4bde672e2c921fb"
Oct 30 22:50:45.826688 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:50:45.826640    5510 generic.go:356] "Generic (PLEG): container finished" podID="920155b2-a008-4621-8019-de154b8a1fea" containerID="d5e58440d61fe8b4c597310353441f773842c2cc873c8789c76a2999a75d142e" exitCode=0
Oct 30 22:50:45.826787 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:50:45.826704    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-single-main-88e685-77594c5967zjvx" event={"ID":"920155b2-a008-4621-8019-de154b8a1fea","Type":"ContainerDied","Data":"d5e58440d61fe8b4c597310353441f773842c2cc873c8789c76a2999a75d142e"}
Oct 30 22:50:45.826787 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:50:45.826743    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-single-main-88e685-77594c5967zjvx" event={"ID":"920155b2-a008-4621-8019-de154b8a1fea","Type":"ContainerDied","Data":"598ed122f0e4e7562d22b2872d64f5512dfad6310f70de7aec6f737df0eec124"}
Oct 30 22:50:45.826787 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:50:45.826702    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-single-main-88e685-77594c5967zjvx"
Oct 30 22:50:45.834056 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:50:45.834042    5510 scope.go:122] "RemoveContainer" containerID="d617393d0278be031079d40dbf5f79c7b35d653e5174e75ca4bde672e2c921fb"
Oct 30 22:50:45.834301 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:50:45.834284    5510 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d617393d0278be031079d40dbf5f79c7b35d653e5174e75ca4bde672e2c921fb\": container with ID starting with d617393d0278be031079d40dbf5f79c7b35d653e5174e75ca4bde672e2c921fb not found: ID does not exist" containerID="d617393d0278be031079d40dbf5f79c7b35d653e5174e75ca4bde672e2c921fb"
Oct 30 22:50:45.834346 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:50:45.834308    5510 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d617393d0278be031079d40dbf5f79c7b35d653e5174e75ca4bde672e2c921fb"} err="failed to get container status \"d617393d0278be031079d40dbf5f79c7b35d653e5174e75ca4bde672e2c921fb\": rpc error: code = NotFound desc = could not find container \"d617393d0278be031079d40dbf5f79c7b35d653e5174e75ca4bde672e2c921fb\": container with ID starting with d617393d0278be031079d40dbf5f79c7b35d653e5174e75ca4bde672e2c921fb not found: ID does not exist"
Oct 30 22:50:45.834346 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:50:45.834323    5510 scope.go:122] "RemoveContainer" containerID="d5e58440d61fe8b4c597310353441f773842c2cc873c8789c76a2999a75d142e"
Oct 30 22:50:45.844539 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:50:45.844520    5510 scope.go:122] "RemoveContainer" containerID="d5e58440d61fe8b4c597310353441f773842c2cc873c8789c76a2999a75d142e"
Oct 30 22:50:45.844826 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:50:45.844806    5510 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d5e58440d61fe8b4c597310353441f773842c2cc873c8789c76a2999a75d142e\": container with ID starting with d5e58440d61fe8b4c597310353441f773842c2cc873c8789c76a2999a75d142e not found: ID does not exist" containerID="d5e58440d61fe8b4c597310353441f773842c2cc873c8789c76a2999a75d142e"
Oct 30 22:50:45.844872 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:50:45.844833    5510 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d5e58440d61fe8b4c597310353441f773842c2cc873c8789c76a2999a75d142e"} err="failed to get container status \"d5e58440d61fe8b4c597310353441f773842c2cc873c8789c76a2999a75d142e\": rpc error: code = NotFound desc = could not find container \"d5e58440d61fe8b4c597310353441f773842c2cc873c8789c76a2999a75d142e\": container with ID starting with d5e58440d61fe8b4c597310353441f773842c2cc873c8789c76a2999a75d142e not found: ID does not exist"
Oct 30 22:50:45.986009 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:50:45.985976    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-88e685-77594c5967zjvx"]
Oct 30 22:50:46.031980 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:50:46.031946    5510 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-88e685-77594c5967zjvx"]
Oct 30 22:50:46.092382 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:50:46.092301    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-88e685-59954dqthn"]
Oct 30 22:50:46.096698 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:50:46.096665    5510 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-88e685-59954dqthn"]
Oct 30 22:50:46.230966 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:50:46.230920    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-88e685-59954dqthn" podUID="acebd8f6-36f7-4152-9154-0b8b186d2a81" containerName="payload-pre-processing" probeResult="failure" output="timeout: failed to connect service \"10.133.0.155:50052\" within 1s: context deadline exceeded"
Oct 30 22:50:46.231389 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:50:46.230927    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-single-main-88e685-77594c5967zjvx" podUID="920155b2-a008-4621-8019-de154b8a1fea" containerName="payload-processing" probeResult="failure" output="timeout: failed to connect service \"10.133.0.156:50052\" within 1s: context deadline exceeded"
Oct 30 22:50:46.231389 ip-10-0-136-210 kubenswrapper[5510]: W1030 22:50:46.231023    5510 logging.go:55] [core] [Channel #6784 SubChannel #6786]grpc: addrConn.createTransport failed to connect to {Addr: "10.133.0.156:50052", ServerName: "10.133.0.156:50052", BalancerAttributes: {"<%!p(pickfirstleaf.managedByPickfirstKeyType={})>": "<%!p(bool=true)>" }}. Err: connection error: desc = "transport: Error while dialing: dial tcp 10.133.0.156:50052: operation was canceled"
Oct 30 22:50:47.551428 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:50:47.551392    5510 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="920155b2-a008-4621-8019-de154b8a1fea" path="/var/lib/kubelet/pods/920155b2-a008-4621-8019-de154b8a1fea/volumes"
Oct 30 22:50:47.551821 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:50:47.551787    5510 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="acebd8f6-36f7-4152-9154-0b8b186d2a81" path="/var/lib/kubelet/pods/acebd8f6-36f7-4152-9154-0b8b186d2a81/volumes"
Oct 30 22:50:50.702681 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:50:50.702619    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:50:51.625679 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:50:51.624877    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:50:57.330369 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:50:57.330326    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:51:08.418809 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:51:08.418773    5510 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6gjbf" containerName="extractor"
Oct 30 22:51:16.252486 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:51:16.252440    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:51:38.419612 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:51:38.419573    5510 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6gjbf" containerName="extractor"
Oct 30 22:51:39.273494 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:51:39.273451    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:51:39.506420 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:51:39.506379    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:52:08.419859 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:52:08.419777    5510 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6gjbf" containerName="extractor"
Oct 30 22:52:13.821467 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:52:13.821423    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:52:14.112976 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:52:14.112896    5510 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-678d87766f-p74zh"]
Oct 30 22:52:14.113342 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:52:14.113327    5510 cpu_manager.go:414] "containerMap: removing container" podUID="31c93ac4-77c9-4231-845d-b13607c94688" containerName="istio-proxy"
Oct 30 22:52:14.113391 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:52:14.113344    5510 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="31c93ac4-77c9-4231-845d-b13607c94688" containerName="istio-proxy"
Oct 30 22:52:14.113391 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:52:14.113359    5510 cpu_manager.go:414] "containerMap: removing container" podUID="491c333b-c990-4332-930c-a1fe707644ca" containerName="main"
Oct 30 22:52:14.113391 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:52:14.113364    5510 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="491c333b-c990-4332-930c-a1fe707644ca" containerName="main"
Oct 30 22:52:14.113391 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:52:14.113380    5510 cpu_manager.go:414] "containerMap: removing container" podUID="fc669e80-db80-4479-93d5-79c6651b405d" containerName="maas-api"
Oct 30 22:52:14.113391 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:52:14.113386    5510 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fc669e80-db80-4479-93d5-79c6651b405d" containerName="maas-api"
Oct 30 22:52:14.113559 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:52:14.113400    5510 cpu_manager.go:414] "containerMap: removing container" podUID="acebd8f6-36f7-4152-9154-0b8b186d2a81" containerName="payload-pre-processing"
Oct 30 22:52:14.113559 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:52:14.113405    5510 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="acebd8f6-36f7-4152-9154-0b8b186d2a81" containerName="payload-pre-processing"
Oct 30 22:52:14.113559 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:52:14.113415    5510 cpu_manager.go:414] "containerMap: removing container" podUID="31c93ac4-77c9-4231-845d-b13607c94688" containerName="istio-proxy"
Oct 30 22:52:14.113559 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:52:14.113420    5510 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="31c93ac4-77c9-4231-845d-b13607c94688" containerName="istio-proxy"
Oct 30 22:52:14.113559 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:52:14.113430    5510 cpu_manager.go:414] "containerMap: removing container" podUID="920155b2-a008-4621-8019-de154b8a1fea" containerName="payload-processing"
Oct 30 22:52:14.113559 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:52:14.113435    5510 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="920155b2-a008-4621-8019-de154b8a1fea" containerName="payload-processing"
Oct 30 22:52:14.113559 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:52:14.113492    5510 memory_manager.go:365] "RemoveStaleState removing state" podUID="acebd8f6-36f7-4152-9154-0b8b186d2a81" containerName="payload-pre-processing"
Oct 30 22:52:14.113559 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:52:14.113502    5510 memory_manager.go:365] "RemoveStaleState removing state" podUID="31c93ac4-77c9-4231-845d-b13607c94688" containerName="istio-proxy"
Oct 30 22:52:14.113559 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:52:14.113508    5510 memory_manager.go:365] "RemoveStaleState removing state" podUID="920155b2-a008-4621-8019-de154b8a1fea" containerName="payload-processing"
Oct 30 22:52:14.113559 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:52:14.113516    5510 memory_manager.go:365] "RemoveStaleState removing state" podUID="31c93ac4-77c9-4231-845d-b13607c94688" containerName="istio-proxy"
Oct 30 22:52:14.113559 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:52:14.113523    5510 memory_manager.go:365] "RemoveStaleState removing state" podUID="491c333b-c990-4332-930c-a1fe707644ca" containerName="main"
Oct 30 22:52:14.113559 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:52:14.113529    5510 memory_manager.go:365] "RemoveStaleState removing state" podUID="fc669e80-db80-4479-93d5-79c6651b405d" containerName="maas-api"
Oct 30 22:52:14.116580 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:52:14.116566    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-p74zh"
Oct 30 22:52:14.164962 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:52:14.164924    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-678d87766f-p74zh"]
Oct 30 22:52:14.209604 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:52:14.209565    5510 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Oct 30 22:52:14.209604 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:52:14.209565    5510 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Oct 30 22:52:14.209814 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:52:14.209723    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-84lq7\""
Oct 30 22:52:14.221107 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:52:14.221080    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x92h2\" (UniqueName: \"kubernetes.io/projected/e36f91b0-e733-4cf5-91da-f805f35f56ba-kube-api-access-x92h2\") pod \"kuadrant-operator-controller-manager-678d87766f-p74zh\" (UID: \"e36f91b0-e733-4cf5-91da-f805f35f56ba\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-p74zh"
Oct 30 22:52:14.221172 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:52:14.221117    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e36f91b0-e733-4cf5-91da-f805f35f56ba-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-678d87766f-p74zh\" (UID: \"e36f91b0-e733-4cf5-91da-f805f35f56ba\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-p74zh"
Oct 30 22:52:14.322679 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:52:14.322631    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x92h2\" (UniqueName: \"kubernetes.io/projected/e36f91b0-e733-4cf5-91da-f805f35f56ba-kube-api-access-x92h2\") pod \"kuadrant-operator-controller-manager-678d87766f-p74zh\" (UID: \"e36f91b0-e733-4cf5-91da-f805f35f56ba\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-p74zh"
Oct 30 22:52:14.322867 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:52:14.322688    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e36f91b0-e733-4cf5-91da-f805f35f56ba-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-678d87766f-p74zh\" (UID: \"e36f91b0-e733-4cf5-91da-f805f35f56ba\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-p74zh"
Oct 30 22:52:14.323074 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:52:14.323055    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e36f91b0-e733-4cf5-91da-f805f35f56ba-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-678d87766f-p74zh\" (UID: \"e36f91b0-e733-4cf5-91da-f805f35f56ba\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-p74zh"
Oct 30 22:52:14.381800 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:52:14.381723    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x92h2\" (UniqueName: \"kubernetes.io/projected/e36f91b0-e733-4cf5-91da-f805f35f56ba-kube-api-access-x92h2\") pod \"kuadrant-operator-controller-manager-678d87766f-p74zh\" (UID: \"e36f91b0-e733-4cf5-91da-f805f35f56ba\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-p74zh"
Oct 30 22:52:14.429738 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:52:14.429708    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-p74zh"
Oct 30 22:52:14.891901 ip-10-0-136-210 kubenswrapper[5510]: W1030 22:52:14.891873    5510 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode36f91b0_e733_4cf5_91da_f805f35f56ba.slice/crio-830b62967cc22abe349973bf7790cc2d5a8555ce083c365d8da432795e322120 WatchSource:0}: Error finding container 830b62967cc22abe349973bf7790cc2d5a8555ce083c365d8da432795e322120: Status 404 returned error can't find the container with id 830b62967cc22abe349973bf7790cc2d5a8555ce083c365d8da432795e322120
Oct 30 22:52:14.941986 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:52:14.938274    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-678d87766f-p74zh"]
Oct 30 22:52:15.276512 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:52:15.276482    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-p74zh" event={"ID":"e36f91b0-e733-4cf5-91da-f805f35f56ba","Type":"ContainerStarted","Data":"830b62967cc22abe349973bf7790cc2d5a8555ce083c365d8da432795e322120"}
Oct 30 22:52:20.306028 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:52:20.305990    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-p74zh" event={"ID":"e36f91b0-e733-4cf5-91da-f805f35f56ba","Type":"ContainerStarted","Data":"7c14847e055e0860f12b49abed0da28ecb0fe8a273ff4a02a7a5c8b81d70a7e8"}
Oct 30 22:52:20.306559 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:52:20.306133    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-p74zh"
Oct 30 22:52:20.522347 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:52:20.522294    5510 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-p74zh" podStartSLOduration=1.5078730139999998 podStartE2EDuration="6.52228098s" totalImagesPullingTime="5.014407966s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:52:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:52:20.474810873 +0000 UTC m=+3721.035620909" watchObservedRunningTime="2026-10-30 22:52:20.52228098 +0000 UTC m=+3721.083090988"
Oct 30 22:52:31.313791 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:52:31.313759    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-p74zh"
Oct 30 22:52:38.419768 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:52:38.419727    5510 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6gjbf" containerName="extractor"
Oct 30 22:52:39.575278 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:52:39.575240    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:53:06.271144 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:53:06.271086    5510 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-85544d4cd7-hfrbk container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.149:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 22:53:06.274322 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:53:06.271943    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-hfrbk" podUID="474fc2c2-03ae-420b-a831-c8e98a3e5ed8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.149:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:53:06.405282 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:53:06.404804    5510 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-85544d4cd7-llmh8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.150:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 22:53:06.405282 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:53:06.404882    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-llmh8" podUID="1123817a-b255-47e2-9188-91650a8b4378" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.150:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:53:08.419162 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:53:08.419122    5510 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6gjbf" containerName="extractor"
Oct 30 22:53:27.480839 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:53:27.480794    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:53:27.954724 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:53:27.954686    5510 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-single-main-4334bf-openshift-default-76c7b68c89xpql6"]
Oct 30 22:53:27.960240 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:53:27.960212    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-single-main-4334bf-openshift-default-76c7b68c89xpql6"
Oct 30 22:53:28.056885 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:53:28.056847    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-single-main-4334bf-openshift-default-dockercfg-mmzlm\""
Oct 30 22:53:28.059875 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:53:28.059839    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-single-main-4334bf-openshift-default-76c7b68c89xpql6"]
Oct 30 22:53:28.073392 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:53:28.073349    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aa30756c-e70d-4152-94b4-ad27b87061ca-istio-data\") pod \"e2e-worker-single-main-4334bf-openshift-default-76c7b68c89xpql6\" (UID: \"aa30756c-e70d-4152-94b4-ad27b87061ca\") " pod="openshift-ingress/e2e-worker-single-main-4334bf-openshift-default-76c7b68c89xpql6"
Oct 30 22:53:28.073551 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:53:28.073409    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa30756c-e70d-4152-94b4-ad27b87061ca-credential-socket\") pod \"e2e-worker-single-main-4334bf-openshift-default-76c7b68c89xpql6\" (UID: \"aa30756c-e70d-4152-94b4-ad27b87061ca\") " pod="openshift-ingress/e2e-worker-single-main-4334bf-openshift-default-76c7b68c89xpql6"
Oct 30 22:53:28.073551 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:53:28.073469    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aa30756c-e70d-4152-94b4-ad27b87061ca-istio-envoy\") pod \"e2e-worker-single-main-4334bf-openshift-default-76c7b68c89xpql6\" (UID: \"aa30756c-e70d-4152-94b4-ad27b87061ca\") " pod="openshift-ingress/e2e-worker-single-main-4334bf-openshift-default-76c7b68c89xpql6"
Oct 30 22:53:28.073551 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:53:28.073503    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aa30756c-e70d-4152-94b4-ad27b87061ca-istiod-ca-cert\") pod \"e2e-worker-single-main-4334bf-openshift-default-76c7b68c89xpql6\" (UID: \"aa30756c-e70d-4152-94b4-ad27b87061ca\") " pod="openshift-ingress/e2e-worker-single-main-4334bf-openshift-default-76c7b68c89xpql6"
Oct 30 22:53:28.073551 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:53:28.073522    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mg92q\" (UniqueName: \"kubernetes.io/projected/aa30756c-e70d-4152-94b4-ad27b87061ca-kube-api-access-mg92q\") pod \"e2e-worker-single-main-4334bf-openshift-default-76c7b68c89xpql6\" (UID: \"aa30756c-e70d-4152-94b4-ad27b87061ca\") " pod="openshift-ingress/e2e-worker-single-main-4334bf-openshift-default-76c7b68c89xpql6"
Oct 30 22:53:28.073883 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:53:28.073857    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa30756c-e70d-4152-94b4-ad27b87061ca-workload-socket\") pod \"e2e-worker-single-main-4334bf-openshift-default-76c7b68c89xpql6\" (UID: \"aa30756c-e70d-4152-94b4-ad27b87061ca\") " pod="openshift-ingress/e2e-worker-single-main-4334bf-openshift-default-76c7b68c89xpql6"
Oct 30 22:53:28.073970 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:53:28.073898    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aa30756c-e70d-4152-94b4-ad27b87061ca-istio-token\") pod \"e2e-worker-single-main-4334bf-openshift-default-76c7b68c89xpql6\" (UID: \"aa30756c-e70d-4152-94b4-ad27b87061ca\") " pod="openshift-ingress/e2e-worker-single-main-4334bf-openshift-default-76c7b68c89xpql6"
Oct 30 22:53:28.073970 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:53:28.073958    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aa30756c-e70d-4152-94b4-ad27b87061ca-workload-certs\") pod \"e2e-worker-single-main-4334bf-openshift-default-76c7b68c89xpql6\" (UID: \"aa30756c-e70d-4152-94b4-ad27b87061ca\") " pod="openshift-ingress/e2e-worker-single-main-4334bf-openshift-default-76c7b68c89xpql6"
Oct 30 22:53:28.074087 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:53:28.073992    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aa30756c-e70d-4152-94b4-ad27b87061ca-istio-podinfo\") pod \"e2e-worker-single-main-4334bf-openshift-default-76c7b68c89xpql6\" (UID: \"aa30756c-e70d-4152-94b4-ad27b87061ca\") " pod="openshift-ingress/e2e-worker-single-main-4334bf-openshift-default-76c7b68c89xpql6"
Oct 30 22:53:28.160752 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:53:28.160724    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:53:28.174908 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:53:28.174879    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa30756c-e70d-4152-94b4-ad27b87061ca-workload-socket\") pod \"e2e-worker-single-main-4334bf-openshift-default-76c7b68c89xpql6\" (UID: \"aa30756c-e70d-4152-94b4-ad27b87061ca\") " pod="openshift-ingress/e2e-worker-single-main-4334bf-openshift-default-76c7b68c89xpql6"
Oct 30 22:53:28.175045 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:53:28.174914    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aa30756c-e70d-4152-94b4-ad27b87061ca-istio-token\") pod \"e2e-worker-single-main-4334bf-openshift-default-76c7b68c89xpql6\" (UID: \"aa30756c-e70d-4152-94b4-ad27b87061ca\") " pod="openshift-ingress/e2e-worker-single-main-4334bf-openshift-default-76c7b68c89xpql6"
Oct 30 22:53:28.175045 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:53:28.174949    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aa30756c-e70d-4152-94b4-ad27b87061ca-workload-certs\") pod \"e2e-worker-single-main-4334bf-openshift-default-76c7b68c89xpql6\" (UID: \"aa30756c-e70d-4152-94b4-ad27b87061ca\") " pod="openshift-ingress/e2e-worker-single-main-4334bf-openshift-default-76c7b68c89xpql6"
Oct 30 22:53:28.175045 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:53:28.174979    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aa30756c-e70d-4152-94b4-ad27b87061ca-istio-podinfo\") pod \"e2e-worker-single-main-4334bf-openshift-default-76c7b68c89xpql6\" (UID: \"aa30756c-e70d-4152-94b4-ad27b87061ca\") " pod="openshift-ingress/e2e-worker-single-main-4334bf-openshift-default-76c7b68c89xpql6"
Oct 30 22:53:28.175231 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:53:28.175044    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aa30756c-e70d-4152-94b4-ad27b87061ca-istio-data\") pod \"e2e-worker-single-main-4334bf-openshift-default-76c7b68c89xpql6\" (UID: \"aa30756c-e70d-4152-94b4-ad27b87061ca\") " pod="openshift-ingress/e2e-worker-single-main-4334bf-openshift-default-76c7b68c89xpql6"
Oct 30 22:53:28.175231 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:53:28.175196    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa30756c-e70d-4152-94b4-ad27b87061ca-credential-socket\") pod \"e2e-worker-single-main-4334bf-openshift-default-76c7b68c89xpql6\" (UID: \"aa30756c-e70d-4152-94b4-ad27b87061ca\") " pod="openshift-ingress/e2e-worker-single-main-4334bf-openshift-default-76c7b68c89xpql6"
Oct 30 22:53:28.175350 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:53:28.175238    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aa30756c-e70d-4152-94b4-ad27b87061ca-istio-envoy\") pod \"e2e-worker-single-main-4334bf-openshift-default-76c7b68c89xpql6\" (UID: \"aa30756c-e70d-4152-94b4-ad27b87061ca\") " pod="openshift-ingress/e2e-worker-single-main-4334bf-openshift-default-76c7b68c89xpql6"
Oct 30 22:53:28.175350 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:53:28.175267    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aa30756c-e70d-4152-94b4-ad27b87061ca-istiod-ca-cert\") pod \"e2e-worker-single-main-4334bf-openshift-default-76c7b68c89xpql6\" (UID: \"aa30756c-e70d-4152-94b4-ad27b87061ca\") " pod="openshift-ingress/e2e-worker-single-main-4334bf-openshift-default-76c7b68c89xpql6"
Oct 30 22:53:28.175350 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:53:28.175331    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa30756c-e70d-4152-94b4-ad27b87061ca-workload-socket\") pod \"e2e-worker-single-main-4334bf-openshift-default-76c7b68c89xpql6\" (UID: \"aa30756c-e70d-4152-94b4-ad27b87061ca\") " pod="openshift-ingress/e2e-worker-single-main-4334bf-openshift-default-76c7b68c89xpql6"
Oct 30 22:53:28.175727 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:53:28.175346    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mg92q\" (UniqueName: \"kubernetes.io/projected/aa30756c-e70d-4152-94b4-ad27b87061ca-kube-api-access-mg92q\") pod \"e2e-worker-single-main-4334bf-openshift-default-76c7b68c89xpql6\" (UID: \"aa30756c-e70d-4152-94b4-ad27b87061ca\") " pod="openshift-ingress/e2e-worker-single-main-4334bf-openshift-default-76c7b68c89xpql6"
Oct 30 22:53:28.175727 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:53:28.175355    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aa30756c-e70d-4152-94b4-ad27b87061ca-workload-certs\") pod \"e2e-worker-single-main-4334bf-openshift-default-76c7b68c89xpql6\" (UID: \"aa30756c-e70d-4152-94b4-ad27b87061ca\") " pod="openshift-ingress/e2e-worker-single-main-4334bf-openshift-default-76c7b68c89xpql6"
Oct 30 22:53:28.175727 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:53:28.175429    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aa30756c-e70d-4152-94b4-ad27b87061ca-istio-data\") pod \"e2e-worker-single-main-4334bf-openshift-default-76c7b68c89xpql6\" (UID: \"aa30756c-e70d-4152-94b4-ad27b87061ca\") " pod="openshift-ingress/e2e-worker-single-main-4334bf-openshift-default-76c7b68c89xpql6"
Oct 30 22:53:28.175888 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:53:28.175788    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa30756c-e70d-4152-94b4-ad27b87061ca-credential-socket\") pod \"e2e-worker-single-main-4334bf-openshift-default-76c7b68c89xpql6\" (UID: \"aa30756c-e70d-4152-94b4-ad27b87061ca\") " pod="openshift-ingress/e2e-worker-single-main-4334bf-openshift-default-76c7b68c89xpql6"
Oct 30 22:53:28.175995 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:53:28.175979    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aa30756c-e70d-4152-94b4-ad27b87061ca-istiod-ca-cert\") pod \"e2e-worker-single-main-4334bf-openshift-default-76c7b68c89xpql6\" (UID: \"aa30756c-e70d-4152-94b4-ad27b87061ca\") " pod="openshift-ingress/e2e-worker-single-main-4334bf-openshift-default-76c7b68c89xpql6"
Oct 30 22:53:28.177941 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:53:28.177922    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aa30756c-e70d-4152-94b4-ad27b87061ca-istio-envoy\") pod \"e2e-worker-single-main-4334bf-openshift-default-76c7b68c89xpql6\" (UID: \"aa30756c-e70d-4152-94b4-ad27b87061ca\") " pod="openshift-ingress/e2e-worker-single-main-4334bf-openshift-default-76c7b68c89xpql6"
Oct 30 22:53:28.178259 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:53:28.178236    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aa30756c-e70d-4152-94b4-ad27b87061ca-istio-podinfo\") pod \"e2e-worker-single-main-4334bf-openshift-default-76c7b68c89xpql6\" (UID: \"aa30756c-e70d-4152-94b4-ad27b87061ca\") " pod="openshift-ingress/e2e-worker-single-main-4334bf-openshift-default-76c7b68c89xpql6"
Oct 30 22:53:28.235249 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:53:28.235137    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mg92q\" (UniqueName: \"kubernetes.io/projected/aa30756c-e70d-4152-94b4-ad27b87061ca-kube-api-access-mg92q\") pod \"e2e-worker-single-main-4334bf-openshift-default-76c7b68c89xpql6\" (UID: \"aa30756c-e70d-4152-94b4-ad27b87061ca\") " pod="openshift-ingress/e2e-worker-single-main-4334bf-openshift-default-76c7b68c89xpql6"
Oct 30 22:53:28.235526 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:53:28.235503    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aa30756c-e70d-4152-94b4-ad27b87061ca-istio-token\") pod \"e2e-worker-single-main-4334bf-openshift-default-76c7b68c89xpql6\" (UID: \"aa30756c-e70d-4152-94b4-ad27b87061ca\") " pod="openshift-ingress/e2e-worker-single-main-4334bf-openshift-default-76c7b68c89xpql6"
Oct 30 22:53:28.280650 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:53:28.280607    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-single-main-4334bf-openshift-default-76c7b68c89xpql6"
Oct 30 22:53:28.363353 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:53:28.363288    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:53:28.708991 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:53:28.708966    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-single-main-4334bf-openshift-default-76c7b68c89xpql6"]
Oct 30 22:53:28.711446 ip-10-0-136-210 kubenswrapper[5510]: W1030 22:53:28.711417    5510 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa30756c_e70d_4152_94b4_ad27b87061ca.slice/crio-59f6d210a5dd5f5e552e5772bc7b7145576c730e56061b20daf1f77b389de5c2 WatchSource:0}: Error finding container 59f6d210a5dd5f5e552e5772bc7b7145576c730e56061b20daf1f77b389de5c2: Status 404 returned error can't find the container with id 59f6d210a5dd5f5e552e5772bc7b7145576c730e56061b20daf1f77b389de5c2
Oct 30 22:53:28.713787 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:53:28.713752    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:53:28.713904 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:53:28.713831    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:53:28.713904 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:53:28.713881    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:53:28.731902 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:53:28.731867    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-4334bf-openshift-default-76c7b68c89xpql6" event={"ID":"aa30756c-e70d-4152-94b4-ad27b87061ca","Type":"ContainerStarted","Data":"59f6d210a5dd5f5e552e5772bc7b7145576c730e56061b20daf1f77b389de5c2"}
Oct 30 22:53:29.745931 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:53:29.745892    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-4334bf-openshift-default-76c7b68c89xpql6" event={"ID":"aa30756c-e70d-4152-94b4-ad27b87061ca","Type":"ContainerStarted","Data":"960f32fe1bb8ed1dd73a85d1c48807e6c2f4681fa55139af55d26029877db1eb"}
Oct 30 22:53:29.906942 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:53:29.906885    5510 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-single-main-4334bf-openshift-default-76c7b68c89xpql6" podStartSLOduration=2.906868599 podStartE2EDuration="2.906868599s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:53:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:53:29.906216042 +0000 UTC m=+3790.467026058" watchObservedRunningTime="2026-10-30 22:53:29.906868599 +0000 UTC m=+3790.467678675"
Oct 30 22:53:30.281417 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:53:30.281378    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-single-main-4334bf-openshift-default-76c7b68c89xpql6"
Oct 30 22:53:30.286878 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:53:30.286849    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-single-main-4334bf-openshift-default-76c7b68c89xpql6"
Oct 30 22:53:30.751201 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:53:30.751116    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-single-main-4334bf-openshift-default-76c7b68c89xpql6"
Oct 30 22:53:30.752092 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:53:30.752073    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-single-main-4334bf-openshift-default-76c7b68c89xpql6"
Oct 30 22:53:32.994700 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:53:32.994640    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:53:33.269316 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:53:33.269279    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:53:36.341836 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:53:36.341800    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:53:36.577857 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:53:36.577815    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:53:36.830376 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:53:36.830346    5510 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-4334bf-7d64b6cdcslfpj"]
Oct 30 22:53:36.835896 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:53:36.835874    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-single-main-4334bf-7d64b6cdcslfpj"
Oct 30 22:53:36.933755 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:53:36.933713    5510 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-single-main-4334bf\""
Oct 30 22:53:36.933755 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:53:36.933725    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-single-m-668d8b8b-dockercfg-sls9l\""
Oct 30 22:53:36.935514 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:53:36.935487    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-4334bf-7d64b6cdcslfpj"]
Oct 30 22:53:36.967831 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:53:36.967797    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rlp85\" (UniqueName: \"kubernetes.io/projected/8681e23d-a36a-4e71-8113-e364fc1f284e-kube-api-access-rlp85\") pod \"payload-processing-e2e-worker-single-main-4334bf-7d64b6cdcslfpj\" (UID: \"8681e23d-a36a-4e71-8113-e364fc1f284e\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-4334bf-7d64b6cdcslfpj"
Oct 30 22:53:36.967984 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:53:36.967863    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8681e23d-a36a-4e71-8113-e364fc1f284e-plugins-config-volume\") pod \"payload-processing-e2e-worker-single-main-4334bf-7d64b6cdcslfpj\" (UID: \"8681e23d-a36a-4e71-8113-e364fc1f284e\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-4334bf-7d64b6cdcslfpj"
Oct 30 22:53:37.026564 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:53:37.026527    5510 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-4334bf-7869clrjkp"]
Oct 30 22:53:37.030162 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:53:37.030140    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-4334bf-7869clrjkp"
Oct 30 22:53:37.069363 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:53:37.069336    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8681e23d-a36a-4e71-8113-e364fc1f284e-plugins-config-volume\") pod \"payload-processing-e2e-worker-single-main-4334bf-7d64b6cdcslfpj\" (UID: \"8681e23d-a36a-4e71-8113-e364fc1f284e\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-4334bf-7d64b6cdcslfpj"
Oct 30 22:53:37.069486 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:53:37.069406    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rlp85\" (UniqueName: \"kubernetes.io/projected/8681e23d-a36a-4e71-8113-e364fc1f284e-kube-api-access-rlp85\") pod \"payload-processing-e2e-worker-single-main-4334bf-7d64b6cdcslfpj\" (UID: \"8681e23d-a36a-4e71-8113-e364fc1f284e\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-4334bf-7d64b6cdcslfpj"
Oct 30 22:53:37.070006 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:53:37.069985    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8681e23d-a36a-4e71-8113-e364fc1f284e-plugins-config-volume\") pod \"payload-processing-e2e-worker-single-main-4334bf-7d64b6cdcslfpj\" (UID: \"8681e23d-a36a-4e71-8113-e364fc1f284e\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-4334bf-7d64b6cdcslfpj"
Oct 30 22:53:37.114140 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:53:37.114052    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-4334bf-7869clrjkp"]
Oct 30 22:53:37.130791 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:53:37.130754    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rlp85\" (UniqueName: \"kubernetes.io/projected/8681e23d-a36a-4e71-8113-e364fc1f284e-kube-api-access-rlp85\") pod \"payload-processing-e2e-worker-single-main-4334bf-7d64b6cdcslfpj\" (UID: \"8681e23d-a36a-4e71-8113-e364fc1f284e\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-4334bf-7d64b6cdcslfpj"
Oct 30 22:53:37.156463 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:53:37.156428    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-single-main-4334bf-7d64b6cdcslfpj"
Oct 30 22:53:37.170256 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:53:37.170211    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b1ecb832-79a2-4265-bb1d-ea52e0f2a699-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-single-main-4334bf-7869clrjkp\" (UID: \"b1ecb832-79a2-4265-bb1d-ea52e0f2a699\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-4334bf-7869clrjkp"
Oct 30 22:53:37.170436 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:53:37.170409    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-29tt8\" (UniqueName: \"kubernetes.io/projected/b1ecb832-79a2-4265-bb1d-ea52e0f2a699-kube-api-access-29tt8\") pod \"payload-pre-processing-e2e-worker-single-main-4334bf-7869clrjkp\" (UID: \"b1ecb832-79a2-4265-bb1d-ea52e0f2a699\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-4334bf-7869clrjkp"
Oct 30 22:53:37.271319 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:53:37.271273    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b1ecb832-79a2-4265-bb1d-ea52e0f2a699-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-single-main-4334bf-7869clrjkp\" (UID: \"b1ecb832-79a2-4265-bb1d-ea52e0f2a699\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-4334bf-7869clrjkp"
Oct 30 22:53:37.271500 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:53:37.271410    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-29tt8\" (UniqueName: \"kubernetes.io/projected/b1ecb832-79a2-4265-bb1d-ea52e0f2a699-kube-api-access-29tt8\") pod \"payload-pre-processing-e2e-worker-single-main-4334bf-7869clrjkp\" (UID: \"b1ecb832-79a2-4265-bb1d-ea52e0f2a699\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-4334bf-7869clrjkp"
Oct 30 22:53:37.272267 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:53:37.272239    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b1ecb832-79a2-4265-bb1d-ea52e0f2a699-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-single-main-4334bf-7869clrjkp\" (UID: \"b1ecb832-79a2-4265-bb1d-ea52e0f2a699\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-4334bf-7869clrjkp"
Oct 30 22:53:37.329871 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:53:37.329833    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-29tt8\" (UniqueName: \"kubernetes.io/projected/b1ecb832-79a2-4265-bb1d-ea52e0f2a699-kube-api-access-29tt8\") pod \"payload-pre-processing-e2e-worker-single-main-4334bf-7869clrjkp\" (UID: \"b1ecb832-79a2-4265-bb1d-ea52e0f2a699\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-4334bf-7869clrjkp"
Oct 30 22:53:37.342736 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:53:37.342709    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-4334bf-7869clrjkp"
Oct 30 22:53:37.527809 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:53:37.527777    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-4334bf-7d64b6cdcslfpj"]
Oct 30 22:53:37.530889 ip-10-0-136-210 kubenswrapper[5510]: W1030 22:53:37.530857    5510 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8681e23d_a36a_4e71_8113_e364fc1f284e.slice/crio-43cc8f012ede96a659dcbfbe3b713f2e3027988d9ea5ad3d2012dce5b980ac2b WatchSource:0}: Error finding container 43cc8f012ede96a659dcbfbe3b713f2e3027988d9ea5ad3d2012dce5b980ac2b: Status 404 returned error can't find the container with id 43cc8f012ede96a659dcbfbe3b713f2e3027988d9ea5ad3d2012dce5b980ac2b
Oct 30 22:53:37.712742 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:53:37.712716    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-4334bf-7869clrjkp"]
Oct 30 22:53:37.715404 ip-10-0-136-210 kubenswrapper[5510]: W1030 22:53:37.715377    5510 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1ecb832_79a2_4265_bb1d_ea52e0f2a699.slice/crio-1e704b32b1d8904c1e6fac667b6ed590c06adde376ad22776b805b7628efd940 WatchSource:0}: Error finding container 1e704b32b1d8904c1e6fac667b6ed590c06adde376ad22776b805b7628efd940: Status 404 returned error can't find the container with id 1e704b32b1d8904c1e6fac667b6ed590c06adde376ad22776b805b7628efd940
Oct 30 22:53:37.787724 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:53:37.787696    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-4334bf-7869clrjkp" event={"ID":"b1ecb832-79a2-4265-bb1d-ea52e0f2a699","Type":"ContainerStarted","Data":"89f186cf01bb4514314b19602b8b9cf9b6488575c2c18c1e60583dc743a4257c"}
Oct 30 22:53:37.787868 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:53:37.787749    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-4334bf-7869clrjkp" event={"ID":"b1ecb832-79a2-4265-bb1d-ea52e0f2a699","Type":"ContainerStarted","Data":"1e704b32b1d8904c1e6fac667b6ed590c06adde376ad22776b805b7628efd940"}
Oct 30 22:53:37.787868 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:53:37.787813    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-4334bf-7869clrjkp"
Oct 30 22:53:37.789255 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:53:37.789234    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-single-main-4334bf-7d64b6cdcslfpj" event={"ID":"8681e23d-a36a-4e71-8113-e364fc1f284e","Type":"ContainerStarted","Data":"d384b5daf89b448dd8e90496396c6e065e277a78612cd4fd7f1c20aba1603b79"}
Oct 30 22:53:37.789360 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:53:37.789268    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-single-main-4334bf-7d64b6cdcslfpj" event={"ID":"8681e23d-a36a-4e71-8113-e364fc1f284e","Type":"ContainerStarted","Data":"43cc8f012ede96a659dcbfbe3b713f2e3027988d9ea5ad3d2012dce5b980ac2b"}
Oct 30 22:53:37.789360 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:53:37.789343    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-single-main-4334bf-7d64b6cdcslfpj"
Oct 30 22:53:37.899819 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:53:37.899759    5510 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-4334bf-7869clrjkp" podStartSLOduration=1.899745434 podStartE2EDuration="1.899745434s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:53:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:53:37.898812455 +0000 UTC m=+3798.459622480" watchObservedRunningTime="2026-10-30 22:53:37.899745434 +0000 UTC m=+3798.460555438"
Oct 30 22:53:38.011084 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:53:38.010965    5510 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-single-main-4334bf-7d64b6cdcslfpj" podStartSLOduration=2.010949742 podStartE2EDuration="2.010949742s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:53:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:53:38.008338617 +0000 UTC m=+3798.569148621" watchObservedRunningTime="2026-10-30 22:53:38.010949742 +0000 UTC m=+3798.571759747"
Oct 30 22:53:38.419562 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:53:38.419516    5510 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6gjbf" containerName="extractor"
Oct 30 22:53:39.551829 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:53:39.551795    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:53:39.748256 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:53:39.748212    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:53:39.932286 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:53:39.932201    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:53:44.615000 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:53:44.614951    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:53:48.796934 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:53:48.796902    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-single-main-4334bf-7d64b6cdcslfpj"
Oct 30 22:53:48.797379 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:53:48.796979    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-4334bf-7869clrjkp"
Oct 30 22:53:49.542725 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:53:49.542644    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:53:54.447387 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:53:54.447349    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:53:56.912995 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:53:56.912956    5510 generic.go:356] "Generic (PLEG): container finished" podID="aa30756c-e70d-4152-94b4-ad27b87061ca" containerID="960f32fe1bb8ed1dd73a85d1c48807e6c2f4681fa55139af55d26029877db1eb" exitCode=0
Oct 30 22:53:56.913456 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:53:56.913027    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-4334bf-openshift-default-76c7b68c89xpql6" event={"ID":"aa30756c-e70d-4152-94b4-ad27b87061ca","Type":"ContainerDied","Data":"960f32fe1bb8ed1dd73a85d1c48807e6c2f4681fa55139af55d26029877db1eb"}
Oct 30 22:53:56.913456 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:53:56.913422    5510 scope.go:122] "RemoveContainer" containerID="960f32fe1bb8ed1dd73a85d1c48807e6c2f4681fa55139af55d26029877db1eb"
Oct 30 22:53:56.914698 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:53:56.914648    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:53:56.914773 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:53:56.914739    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:53:56.914773 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:53:56.914769    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:53:57.672489 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:53:57.672437    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:53:57.919749 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:53:57.919710    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-4334bf-openshift-default-76c7b68c89xpql6" event={"ID":"aa30756c-e70d-4152-94b4-ad27b87061ca","Type":"ContainerStarted","Data":"e9ac1d98851f92f029aeee3d0a7970654d2889ba7bf052fe9a319036ff93fa06"}
Oct 30 22:53:58.280984 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:53:58.280944    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-single-main-4334bf-openshift-default-76c7b68c89xpql6"
Oct 30 22:53:58.282270 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:53:58.282240    5510 patch_prober.go:29] interesting pod/e2e-worker-single-main-4334bf-openshift-default-76c7b68c89xpql6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.160:15021/healthz/ready\": dial tcp 10.133.0.160:15021: connect: connection refused" start-of-body=
Oct 30 22:53:58.282421 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:53:58.282290    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-4334bf-openshift-default-76c7b68c89xpql6" podUID="aa30756c-e70d-4152-94b4-ad27b87061ca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.160:15021/healthz/ready\": dial tcp 10.133.0.160:15021: connect: connection refused"
Oct 30 22:53:58.307719 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:53:58.307683    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:53:58.502552 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:53:58.502515    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:53:59.281737 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:53:59.281696    5510 patch_prober.go:29] interesting pod/e2e-worker-single-main-4334bf-openshift-default-76c7b68c89xpql6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.160:15021/healthz/ready\": dial tcp 10.133.0.160:15021: connect: connection refused" start-of-body=
Oct 30 22:53:59.282154 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:53:59.281769    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-4334bf-openshift-default-76c7b68c89xpql6" podUID="aa30756c-e70d-4152-94b4-ad27b87061ca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.160:15021/healthz/ready\": dial tcp 10.133.0.160:15021: connect: connection refused"
Oct 30 22:53:59.772028 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:53:59.771747    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:54:00.281365 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:54:00.281331    5510 patch_prober.go:29] interesting pod/e2e-worker-single-main-4334bf-openshift-default-76c7b68c89xpql6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.160:15021/healthz/ready\": dial tcp 10.133.0.160:15021: connect: connection refused" start-of-body=
Oct 30 22:54:00.281544 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:54:00.281382    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-4334bf-openshift-default-76c7b68c89xpql6" podUID="aa30756c-e70d-4152-94b4-ad27b87061ca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.160:15021/healthz/ready\": dial tcp 10.133.0.160:15021: connect: connection refused"
Oct 30 22:54:00.764867 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:54:00.764766    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-single-main-4334bf-openshift-default-76c7b68c89xpql6"
Oct 30 22:54:01.281395 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:54:01.281359    5510 patch_prober.go:29] interesting pod/e2e-worker-single-main-4334bf-openshift-default-76c7b68c89xpql6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.160:15021/healthz/ready\": dial tcp 10.133.0.160:15021: connect: connection refused" start-of-body=
Oct 30 22:54:01.281594 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:54:01.281413    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-4334bf-openshift-default-76c7b68c89xpql6" podUID="aa30756c-e70d-4152-94b4-ad27b87061ca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.160:15021/healthz/ready\": dial tcp 10.133.0.160:15021: connect: connection refused"
Oct 30 22:54:01.299766 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:54:01.299727    5510 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-single-main-4334bf/simulator-single-main-4334bf-kserve-6674c67d4d-rl82b"]
Oct 30 22:54:01.303182 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:54:01.303160    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-single-main-4334bf/simulator-single-main-4334bf-kserve-6674c67d4d-rl82b"
Oct 30 22:54:01.380358 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:54:01.380325    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-single-main-4334bf\"/\"simulator-single-main-4334bf-kserve-self-signed-certs\""
Oct 30 22:54:01.380555 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:54:01.380368    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-single-main-4334bf\"/\"default-dockercfg-rwhq5\""
Oct 30 22:54:01.380555 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:54:01.380482    5510 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-single-main-4334bf\"/\"openshift-service-ca.crt\""
Oct 30 22:54:01.402009 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:54:01.401974    5510 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-single-main-4334bf\"/\"kube-root-ca.crt\""
Oct 30 22:54:01.403905 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:54:01.403878    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-single-main-4334bf/simulator-single-main-4334bf-kserve-6674c67d4d-rl82b"]
Oct 30 22:54:01.473681 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:54:01.466420    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:54:01.476554 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:54:01.476435    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d069c582-5b79-4a80-93d3-2157f96d17ac-home\") pod \"simulator-single-main-4334bf-kserve-6674c67d4d-rl82b\" (UID: \"d069c582-5b79-4a80-93d3-2157f96d17ac\") " pod="e2e-models-e2e-worker-single-main-4334bf/simulator-single-main-4334bf-kserve-6674c67d4d-rl82b"
Oct 30 22:54:01.476766 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:54:01.476613    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d069c582-5b79-4a80-93d3-2157f96d17ac-dshm\") pod \"simulator-single-main-4334bf-kserve-6674c67d4d-rl82b\" (UID: \"d069c582-5b79-4a80-93d3-2157f96d17ac\") " pod="e2e-models-e2e-worker-single-main-4334bf/simulator-single-main-4334bf-kserve-6674c67d4d-rl82b"
Oct 30 22:54:01.476766 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:54:01.476698    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d069c582-5b79-4a80-93d3-2157f96d17ac-tls-certs\") pod \"simulator-single-main-4334bf-kserve-6674c67d4d-rl82b\" (UID: \"d069c582-5b79-4a80-93d3-2157f96d17ac\") " pod="e2e-models-e2e-worker-single-main-4334bf/simulator-single-main-4334bf-kserve-6674c67d4d-rl82b"
Oct 30 22:54:01.476909 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:54:01.476769    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d069c582-5b79-4a80-93d3-2157f96d17ac-model-cache\") pod \"simulator-single-main-4334bf-kserve-6674c67d4d-rl82b\" (UID: \"d069c582-5b79-4a80-93d3-2157f96d17ac\") " pod="e2e-models-e2e-worker-single-main-4334bf/simulator-single-main-4334bf-kserve-6674c67d4d-rl82b"
Oct 30 22:54:01.476909 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:54:01.476832    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8w9wh\" (UniqueName: \"kubernetes.io/projected/d069c582-5b79-4a80-93d3-2157f96d17ac-kube-api-access-8w9wh\") pod \"simulator-single-main-4334bf-kserve-6674c67d4d-rl82b\" (UID: \"d069c582-5b79-4a80-93d3-2157f96d17ac\") " pod="e2e-models-e2e-worker-single-main-4334bf/simulator-single-main-4334bf-kserve-6674c67d4d-rl82b"
Oct 30 22:54:01.476909 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:54:01.476876    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d069c582-5b79-4a80-93d3-2157f96d17ac-tmp-dir\") pod \"simulator-single-main-4334bf-kserve-6674c67d4d-rl82b\" (UID: \"d069c582-5b79-4a80-93d3-2157f96d17ac\") " pod="e2e-models-e2e-worker-single-main-4334bf/simulator-single-main-4334bf-kserve-6674c67d4d-rl82b"
Oct 30 22:54:01.577773 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:54:01.577689    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d069c582-5b79-4a80-93d3-2157f96d17ac-model-cache\") pod \"simulator-single-main-4334bf-kserve-6674c67d4d-rl82b\" (UID: \"d069c582-5b79-4a80-93d3-2157f96d17ac\") " pod="e2e-models-e2e-worker-single-main-4334bf/simulator-single-main-4334bf-kserve-6674c67d4d-rl82b"
Oct 30 22:54:01.577773 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:54:01.577752    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8w9wh\" (UniqueName: \"kubernetes.io/projected/d069c582-5b79-4a80-93d3-2157f96d17ac-kube-api-access-8w9wh\") pod \"simulator-single-main-4334bf-kserve-6674c67d4d-rl82b\" (UID: \"d069c582-5b79-4a80-93d3-2157f96d17ac\") " pod="e2e-models-e2e-worker-single-main-4334bf/simulator-single-main-4334bf-kserve-6674c67d4d-rl82b"
Oct 30 22:54:01.578013 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:54:01.577791    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d069c582-5b79-4a80-93d3-2157f96d17ac-tmp-dir\") pod \"simulator-single-main-4334bf-kserve-6674c67d4d-rl82b\" (UID: \"d069c582-5b79-4a80-93d3-2157f96d17ac\") " pod="e2e-models-e2e-worker-single-main-4334bf/simulator-single-main-4334bf-kserve-6674c67d4d-rl82b"
Oct 30 22:54:01.578013 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:54:01.577871    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d069c582-5b79-4a80-93d3-2157f96d17ac-home\") pod \"simulator-single-main-4334bf-kserve-6674c67d4d-rl82b\" (UID: \"d069c582-5b79-4a80-93d3-2157f96d17ac\") " pod="e2e-models-e2e-worker-single-main-4334bf/simulator-single-main-4334bf-kserve-6674c67d4d-rl82b"
Oct 30 22:54:01.578013 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:54:01.577928    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d069c582-5b79-4a80-93d3-2157f96d17ac-dshm\") pod \"simulator-single-main-4334bf-kserve-6674c67d4d-rl82b\" (UID: \"d069c582-5b79-4a80-93d3-2157f96d17ac\") " pod="e2e-models-e2e-worker-single-main-4334bf/simulator-single-main-4334bf-kserve-6674c67d4d-rl82b"
Oct 30 22:54:01.578013 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:54:01.577958    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d069c582-5b79-4a80-93d3-2157f96d17ac-tls-certs\") pod \"simulator-single-main-4334bf-kserve-6674c67d4d-rl82b\" (UID: \"d069c582-5b79-4a80-93d3-2157f96d17ac\") " pod="e2e-models-e2e-worker-single-main-4334bf/simulator-single-main-4334bf-kserve-6674c67d4d-rl82b"
Oct 30 22:54:01.578218 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:54:01.578198    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d069c582-5b79-4a80-93d3-2157f96d17ac-tmp-dir\") pod \"simulator-single-main-4334bf-kserve-6674c67d4d-rl82b\" (UID: \"d069c582-5b79-4a80-93d3-2157f96d17ac\") " pod="e2e-models-e2e-worker-single-main-4334bf/simulator-single-main-4334bf-kserve-6674c67d4d-rl82b"
Oct 30 22:54:01.578424 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:54:01.578405    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d069c582-5b79-4a80-93d3-2157f96d17ac-home\") pod \"simulator-single-main-4334bf-kserve-6674c67d4d-rl82b\" (UID: \"d069c582-5b79-4a80-93d3-2157f96d17ac\") " pod="e2e-models-e2e-worker-single-main-4334bf/simulator-single-main-4334bf-kserve-6674c67d4d-rl82b"
Oct 30 22:54:01.578511 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:54:01.578494    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d069c582-5b79-4a80-93d3-2157f96d17ac-model-cache\") pod \"simulator-single-main-4334bf-kserve-6674c67d4d-rl82b\" (UID: \"d069c582-5b79-4a80-93d3-2157f96d17ac\") " pod="e2e-models-e2e-worker-single-main-4334bf/simulator-single-main-4334bf-kserve-6674c67d4d-rl82b"
Oct 30 22:54:01.580330 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:54:01.580298    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d069c582-5b79-4a80-93d3-2157f96d17ac-dshm\") pod \"simulator-single-main-4334bf-kserve-6674c67d4d-rl82b\" (UID: \"d069c582-5b79-4a80-93d3-2157f96d17ac\") " pod="e2e-models-e2e-worker-single-main-4334bf/simulator-single-main-4334bf-kserve-6674c67d4d-rl82b"
Oct 30 22:54:01.580456 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:54:01.580439    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d069c582-5b79-4a80-93d3-2157f96d17ac-tls-certs\") pod \"simulator-single-main-4334bf-kserve-6674c67d4d-rl82b\" (UID: \"d069c582-5b79-4a80-93d3-2157f96d17ac\") " pod="e2e-models-e2e-worker-single-main-4334bf/simulator-single-main-4334bf-kserve-6674c67d4d-rl82b"
Oct 30 22:54:01.637519 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:54:01.637483    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8w9wh\" (UniqueName: \"kubernetes.io/projected/d069c582-5b79-4a80-93d3-2157f96d17ac-kube-api-access-8w9wh\") pod \"simulator-single-main-4334bf-kserve-6674c67d4d-rl82b\" (UID: \"d069c582-5b79-4a80-93d3-2157f96d17ac\") " pod="e2e-models-e2e-worker-single-main-4334bf/simulator-single-main-4334bf-kserve-6674c67d4d-rl82b"
Oct 30 22:54:01.915811 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:54:01.915730    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-single-main-4334bf/simulator-single-main-4334bf-kserve-6674c67d4d-rl82b"
Oct 30 22:54:02.281579 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:54:02.281544    5510 patch_prober.go:29] interesting pod/e2e-worker-single-main-4334bf-openshift-default-76c7b68c89xpql6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.160:15021/healthz/ready\": dial tcp 10.133.0.160:15021: connect: connection refused" start-of-body=
Oct 30 22:54:02.281781 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:54:02.281595    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-4334bf-openshift-default-76c7b68c89xpql6" podUID="aa30756c-e70d-4152-94b4-ad27b87061ca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.160:15021/healthz/ready\": dial tcp 10.133.0.160:15021: connect: connection refused"
Oct 30 22:54:02.310753 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:54:02.310720    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-single-main-4334bf/simulator-single-main-4334bf-kserve-6674c67d4d-rl82b"]
Oct 30 22:54:02.314362 ip-10-0-136-210 kubenswrapper[5510]: W1030 22:54:02.314335    5510 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd069c582_5b79_4a80_93d3_2157f96d17ac.slice/crio-2f781ddf47a60ac83e2d755ad630e0f13d25c1bdd6b55914c3afa1acafcd7ca5 WatchSource:0}: Error finding container 2f781ddf47a60ac83e2d755ad630e0f13d25c1bdd6b55914c3afa1acafcd7ca5: Status 404 returned error can't find the container with id 2f781ddf47a60ac83e2d755ad630e0f13d25c1bdd6b55914c3afa1acafcd7ca5
Oct 30 22:54:02.948539 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:54:02.948489    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-single-main-4334bf/simulator-single-main-4334bf-kserve-6674c67d4d-rl82b" event={"ID":"d069c582-5b79-4a80-93d3-2157f96d17ac","Type":"ContainerStarted","Data":"38550c3c19b13858b480a99f43911edaf537c1025a14a2890eecb2f3a02789a2"}
Oct 30 22:54:02.949069 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:54:02.948549    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-single-main-4334bf/simulator-single-main-4334bf-kserve-6674c67d4d-rl82b" event={"ID":"d069c582-5b79-4a80-93d3-2157f96d17ac","Type":"ContainerStarted","Data":"2f781ddf47a60ac83e2d755ad630e0f13d25c1bdd6b55914c3afa1acafcd7ca5"}
Oct 30 22:54:03.070313 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:54:03.070252    5510 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-single-main-4334bf/simulator-single-main-4334bf-kserve-6674c67d4d-rl82b" podStartSLOduration=2.070238042 podStartE2EDuration="2.070238042s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:54:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:54:03.070152854 +0000 UTC m=+3823.630962860" watchObservedRunningTime="2026-10-30 22:54:03.070238042 +0000 UTC m=+3823.631048046"
Oct 30 22:54:03.285160 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:54:03.285127    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-single-main-4334bf-openshift-default-76c7b68c89xpql6"
Oct 30 22:54:03.286080 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:54:03.286058    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-single-main-4334bf-openshift-default-76c7b68c89xpql6"
Oct 30 22:54:03.916701 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:54:03.916631    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-single-main-4334bf/simulator-single-main-4334bf-kserve-6674c67d4d-rl82b"
Oct 30 22:54:04.726446 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:54:04.726401    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:54:08.419690 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:54:08.419640    5510 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6gjbf" containerName="extractor"
Oct 30 22:54:09.875043 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:54:09.875006    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:54:11.916223 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:54:11.916169    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-single-main-4334bf/simulator-single-main-4334bf-kserve-6674c67d4d-rl82b"
Oct 30 22:54:11.928563 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:54:11.928537    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-single-main-4334bf/simulator-single-main-4334bf-kserve-6674c67d4d-rl82b"
Oct 30 22:54:12.008838 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:54:12.008804    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-single-main-4334bf/simulator-single-main-4334bf-kserve-6674c67d4d-rl82b"
Oct 30 22:54:14.935511 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:54:14.935475    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:54:16.984303 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:54:16.984267    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:54:19.803592 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:54:19.803552    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:54:24.720556 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:54:24.720516    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:54:29.637396 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:54:29.637351    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:54:32.397339 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:54:32.397293    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:54:32.598305 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:54:32.598260    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:54:34.574077 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:54:34.574037    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:54:38.418720 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:54:38.418647    5510 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6gjbf" containerName="extractor"
Oct 30 22:54:39.517494 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:54:39.517456    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:54:40.251860 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:54:40.251821    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:54:41.431067 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:54:41.431027    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:54:44.435565 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:54:44.435530    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:54:49.440953 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:54:49.440914    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:54:54.329783 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:54:54.329742    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:54:57.715091 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:54:57.715055    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:54:59.483376 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:54:59.483336    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:55:02.596251 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:55:02.596160    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-single-main-4334bf/simulator-single-main-4334bf-kserve-6674c67d4d-rl82b"]
Oct 30 22:55:02.732587 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:55:02.732542    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:55:04.614811 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:55:04.614769    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:55:08.419903 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:55:08.419864    5510 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6gjbf" containerName="extractor"
Oct 30 22:55:08.582911 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:55:08.582874    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-4334bf-7d64b6cdcslfpj"]
Oct 30 22:55:08.686006 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:55:08.685921    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-4334bf-7869clrjkp"]
Oct 30 22:55:08.946202 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:55:08.946113    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:55:09.143401 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:55:09.143360    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:55:12.880085 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:55:12.880043    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:55:13.159896 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:55:13.159800    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:55:13.583531 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:55:13.583480    5510 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-single-main-4334bf-7d64b6cdcslfpj" podUID="8681e23d-a36a-4e71-8113-e364fc1f284e" containerName="payload-processing" containerID="cri-o://d384b5daf89b448dd8e90496396c6e065e277a78612cd4fd7f1c20aba1603b79" gracePeriod=25
Oct 30 22:55:13.686214 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:55:13.686153    5510 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-4334bf-7869clrjkp" podUID="b1ecb832-79a2-4265-bb1d-ea52e0f2a699" containerName="payload-pre-processing" containerID="cri-o://89f186cf01bb4514314b19602b8b9cf9b6488575c2c18c1e60583dc743a4257c" gracePeriod=25
Oct 30 22:55:13.934890 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:55:13.934808    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:55:17.616636 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:55:17.616577    5510 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-single-main-4334bf/simulator-single-main-4334bf-kserve-6674c67d4d-rl82b" podUID="d069c582-5b79-4a80-93d3-2157f96d17ac" containerName="main" containerID="cri-o://38550c3c19b13858b480a99f43911edaf537c1025a14a2890eecb2f3a02789a2" gracePeriod=45
Oct 30 22:55:17.869421 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:55:17.869359    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-single-main-4334bf/simulator-single-main-4334bf-kserve-6674c67d4d-rl82b"
Oct 30 22:55:17.963013 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:55:17.962975    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d069c582-5b79-4a80-93d3-2157f96d17ac-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d069c582-5b79-4a80-93d3-2157f96d17ac-model-cache\") pod \"d069c582-5b79-4a80-93d3-2157f96d17ac\" (UID: \"d069c582-5b79-4a80-93d3-2157f96d17ac\") "
Oct 30 22:55:17.963190 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:55:17.963094    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d069c582-5b79-4a80-93d3-2157f96d17ac-kube-api-access-8w9wh\" (UniqueName: \"kubernetes.io/projected/d069c582-5b79-4a80-93d3-2157f96d17ac-kube-api-access-8w9wh\") pod \"d069c582-5b79-4a80-93d3-2157f96d17ac\" (UID: \"d069c582-5b79-4a80-93d3-2157f96d17ac\") "
Oct 30 22:55:17.963190 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:55:17.963142    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d069c582-5b79-4a80-93d3-2157f96d17ac-home\" (UniqueName: \"kubernetes.io/empty-dir/d069c582-5b79-4a80-93d3-2157f96d17ac-home\") pod \"d069c582-5b79-4a80-93d3-2157f96d17ac\" (UID: \"d069c582-5b79-4a80-93d3-2157f96d17ac\") "
Oct 30 22:55:17.963190 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:55:17.963179    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d069c582-5b79-4a80-93d3-2157f96d17ac-dshm\" (UniqueName: \"kubernetes.io/empty-dir/d069c582-5b79-4a80-93d3-2157f96d17ac-dshm\") pod \"d069c582-5b79-4a80-93d3-2157f96d17ac\" (UID: \"d069c582-5b79-4a80-93d3-2157f96d17ac\") "
Oct 30 22:55:17.963383 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:55:17.963225    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d069c582-5b79-4a80-93d3-2157f96d17ac-tls-certs\" (UniqueName: \"kubernetes.io/secret/d069c582-5b79-4a80-93d3-2157f96d17ac-tls-certs\") pod \"d069c582-5b79-4a80-93d3-2157f96d17ac\" (UID: \"d069c582-5b79-4a80-93d3-2157f96d17ac\") "
Oct 30 22:55:17.963383 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:55:17.963238    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d069c582-5b79-4a80-93d3-2157f96d17ac-model-cache" pod "d069c582-5b79-4a80-93d3-2157f96d17ac" (UID: "d069c582-5b79-4a80-93d3-2157f96d17ac"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:55:17.963383 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:55:17.963292    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d069c582-5b79-4a80-93d3-2157f96d17ac-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d069c582-5b79-4a80-93d3-2157f96d17ac-tmp-dir\") pod \"d069c582-5b79-4a80-93d3-2157f96d17ac\" (UID: \"d069c582-5b79-4a80-93d3-2157f96d17ac\") "
Oct 30 22:55:17.963557 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:55:17.963442    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d069c582-5b79-4a80-93d3-2157f96d17ac-home" pod "d069c582-5b79-4a80-93d3-2157f96d17ac" (UID: "d069c582-5b79-4a80-93d3-2157f96d17ac"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:55:17.963741 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:55:17.963720    5510 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d069c582-5b79-4a80-93d3-2157f96d17ac-model-cache\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:55:17.963741 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:55:17.963744    5510 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d069c582-5b79-4a80-93d3-2157f96d17ac-home\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:55:17.963867 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:55:17.963749    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d069c582-5b79-4a80-93d3-2157f96d17ac-tmp-dir" pod "d069c582-5b79-4a80-93d3-2157f96d17ac" (UID: "d069c582-5b79-4a80-93d3-2157f96d17ac"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:55:17.965611 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:55:17.965586    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d069c582-5b79-4a80-93d3-2157f96d17ac-dshm" pod "d069c582-5b79-4a80-93d3-2157f96d17ac" (UID: "d069c582-5b79-4a80-93d3-2157f96d17ac"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:55:17.965739 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:55:17.965620    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d069c582-5b79-4a80-93d3-2157f96d17ac-kube-api-access-8w9wh" pod "d069c582-5b79-4a80-93d3-2157f96d17ac" (UID: "d069c582-5b79-4a80-93d3-2157f96d17ac"). InnerVolumeSpecName "kube-api-access-8w9wh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:55:17.965933 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:55:17.965906    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d069c582-5b79-4a80-93d3-2157f96d17ac-tls-certs" pod "d069c582-5b79-4a80-93d3-2157f96d17ac" (UID: "d069c582-5b79-4a80-93d3-2157f96d17ac"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:55:18.064510 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:55:18.064471    5510 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8w9wh\" (UniqueName: \"kubernetes.io/projected/d069c582-5b79-4a80-93d3-2157f96d17ac-kube-api-access-8w9wh\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:55:18.064510 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:55:18.064501    5510 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d069c582-5b79-4a80-93d3-2157f96d17ac-dshm\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:55:18.064510 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:55:18.064510    5510 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d069c582-5b79-4a80-93d3-2157f96d17ac-tls-certs\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:55:18.064819 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:55:18.064543    5510 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d069c582-5b79-4a80-93d3-2157f96d17ac-tmp-dir\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:55:18.337854 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:55:18.337824    5510 generic.go:356] "Generic (PLEG): container finished" podID="d069c582-5b79-4a80-93d3-2157f96d17ac" containerID="38550c3c19b13858b480a99f43911edaf537c1025a14a2890eecb2f3a02789a2" exitCode=0
Oct 30 22:55:18.338024 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:55:18.337869    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-single-main-4334bf/simulator-single-main-4334bf-kserve-6674c67d4d-rl82b"
Oct 30 22:55:18.338024 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:55:18.337908    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-single-main-4334bf/simulator-single-main-4334bf-kserve-6674c67d4d-rl82b" event={"ID":"d069c582-5b79-4a80-93d3-2157f96d17ac","Type":"ContainerDied","Data":"38550c3c19b13858b480a99f43911edaf537c1025a14a2890eecb2f3a02789a2"}
Oct 30 22:55:18.338024 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:55:18.337961    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-single-main-4334bf/simulator-single-main-4334bf-kserve-6674c67d4d-rl82b" event={"ID":"d069c582-5b79-4a80-93d3-2157f96d17ac","Type":"ContainerDied","Data":"2f781ddf47a60ac83e2d755ad630e0f13d25c1bdd6b55914c3afa1acafcd7ca5"}
Oct 30 22:55:18.338024 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:55:18.337978    5510 scope.go:122] "RemoveContainer" containerID="38550c3c19b13858b480a99f43911edaf537c1025a14a2890eecb2f3a02789a2"
Oct 30 22:55:18.347408 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:55:18.347391    5510 scope.go:122] "RemoveContainer" containerID="38550c3c19b13858b480a99f43911edaf537c1025a14a2890eecb2f3a02789a2"
Oct 30 22:55:18.347710 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:55:18.347694    5510 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"38550c3c19b13858b480a99f43911edaf537c1025a14a2890eecb2f3a02789a2\": container with ID starting with 38550c3c19b13858b480a99f43911edaf537c1025a14a2890eecb2f3a02789a2 not found: ID does not exist" containerID="38550c3c19b13858b480a99f43911edaf537c1025a14a2890eecb2f3a02789a2"
Oct 30 22:55:18.347766 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:55:18.347718    5510 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"38550c3c19b13858b480a99f43911edaf537c1025a14a2890eecb2f3a02789a2"} err="failed to get container status \"38550c3c19b13858b480a99f43911edaf537c1025a14a2890eecb2f3a02789a2\": rpc error: code = NotFound desc = could not find container \"38550c3c19b13858b480a99f43911edaf537c1025a14a2890eecb2f3a02789a2\": container with ID starting with 38550c3c19b13858b480a99f43911edaf537c1025a14a2890eecb2f3a02789a2 not found: ID does not exist"
Oct 30 22:55:18.492451 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:55:18.492411    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-single-main-4334bf/simulator-single-main-4334bf-kserve-6674c67d4d-rl82b"]
Oct 30 22:55:18.539103 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:55:18.539060    5510 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-single-main-4334bf/simulator-single-main-4334bf-kserve-6674c67d4d-rl82b"]
Oct 30 22:55:18.795768 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:55:18.795725    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-4334bf-7869clrjkp" podUID="b1ecb832-79a2-4265-bb1d-ea52e0f2a699" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:55:18.795768 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:55:18.795752    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-single-main-4334bf-7d64b6cdcslfpj" podUID="8681e23d-a36a-4e71-8113-e364fc1f284e" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:55:19.551847 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:55:19.551763    5510 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d069c582-5b79-4a80-93d3-2157f96d17ac" path="/var/lib/kubelet/pods/d069c582-5b79-4a80-93d3-2157f96d17ac/volumes"
Oct 30 22:55:19.875965 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:55:19.875866    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:55:20.108447 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:55:20.108405    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:55:20.176420 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:55:20.176333    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-single-main-4334bf-openshift-default-76c7b68c89xpql6"]
Oct 30 22:55:20.176628 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:55:20.176548    5510 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-single-main-4334bf-openshift-default-76c7b68c89xpql6" podUID="aa30756c-e70d-4152-94b4-ad27b87061ca" containerName="istio-proxy" containerID="cri-o://e9ac1d98851f92f029aeee3d0a7970654d2889ba7bf052fe9a319036ff93fa06" gracePeriod=30
Oct 30 22:55:20.303144 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:55:20.303109    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:55:25.377540 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:55:25.377506    5510 generic.go:356] "Generic (PLEG): container finished" podID="aa30756c-e70d-4152-94b4-ad27b87061ca" containerID="e9ac1d98851f92f029aeee3d0a7970654d2889ba7bf052fe9a319036ff93fa06" exitCode=0
Oct 30 22:55:25.378022 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:55:25.377554    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-4334bf-openshift-default-76c7b68c89xpql6" event={"ID":"aa30756c-e70d-4152-94b4-ad27b87061ca","Type":"ContainerDied","Data":"e9ac1d98851f92f029aeee3d0a7970654d2889ba7bf052fe9a319036ff93fa06"}
Oct 30 22:55:25.378022 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:55:25.377604    5510 scope.go:122] "RemoveContainer" containerID="960f32fe1bb8ed1dd73a85d1c48807e6c2f4681fa55139af55d26029877db1eb"
Oct 30 22:55:25.436041 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:55:25.436011    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-single-main-4334bf-openshift-default-76c7b68c89xpql6"
Oct 30 22:55:25.537249 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:55:25.537222    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/aa30756c-e70d-4152-94b4-ad27b87061ca-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa30756c-e70d-4152-94b4-ad27b87061ca-credential-socket\") pod \"aa30756c-e70d-4152-94b4-ad27b87061ca\" (UID: \"aa30756c-e70d-4152-94b4-ad27b87061ca\") "
Oct 30 22:55:25.537427 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:55:25.537272    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/aa30756c-e70d-4152-94b4-ad27b87061ca-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aa30756c-e70d-4152-94b4-ad27b87061ca-istiod-ca-cert\") pod \"aa30756c-e70d-4152-94b4-ad27b87061ca\" (UID: \"aa30756c-e70d-4152-94b4-ad27b87061ca\") "
Oct 30 22:55:25.537427 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:55:25.537315    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/aa30756c-e70d-4152-94b4-ad27b87061ca-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aa30756c-e70d-4152-94b4-ad27b87061ca-istio-podinfo\") pod \"aa30756c-e70d-4152-94b4-ad27b87061ca\" (UID: \"aa30756c-e70d-4152-94b4-ad27b87061ca\") "
Oct 30 22:55:25.537427 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:55:25.537341    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/aa30756c-e70d-4152-94b4-ad27b87061ca-istio-token\" (UniqueName: \"kubernetes.io/projected/aa30756c-e70d-4152-94b4-ad27b87061ca-istio-token\") pod \"aa30756c-e70d-4152-94b4-ad27b87061ca\" (UID: \"aa30756c-e70d-4152-94b4-ad27b87061ca\") "
Oct 30 22:55:25.537427 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:55:25.537386    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/aa30756c-e70d-4152-94b4-ad27b87061ca-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aa30756c-e70d-4152-94b4-ad27b87061ca-workload-certs\") pod \"aa30756c-e70d-4152-94b4-ad27b87061ca\" (UID: \"aa30756c-e70d-4152-94b4-ad27b87061ca\") "
Oct 30 22:55:25.537427 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:55:25.537420    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/aa30756c-e70d-4152-94b4-ad27b87061ca-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa30756c-e70d-4152-94b4-ad27b87061ca-workload-socket\") pod \"aa30756c-e70d-4152-94b4-ad27b87061ca\" (UID: \"aa30756c-e70d-4152-94b4-ad27b87061ca\") "
Oct 30 22:55:25.537746 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:55:25.537497    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/aa30756c-e70d-4152-94b4-ad27b87061ca-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aa30756c-e70d-4152-94b4-ad27b87061ca-istio-envoy\") pod \"aa30756c-e70d-4152-94b4-ad27b87061ca\" (UID: \"aa30756c-e70d-4152-94b4-ad27b87061ca\") "
Oct 30 22:55:25.537746 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:55:25.537528    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/aa30756c-e70d-4152-94b4-ad27b87061ca-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aa30756c-e70d-4152-94b4-ad27b87061ca-istio-data\") pod \"aa30756c-e70d-4152-94b4-ad27b87061ca\" (UID: \"aa30756c-e70d-4152-94b4-ad27b87061ca\") "
Oct 30 22:55:25.537746 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:55:25.537550    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa30756c-e70d-4152-94b4-ad27b87061ca-credential-socket" pod "aa30756c-e70d-4152-94b4-ad27b87061ca" (UID: "aa30756c-e70d-4152-94b4-ad27b87061ca"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:55:25.537746 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:55:25.537557    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/aa30756c-e70d-4152-94b4-ad27b87061ca-kube-api-access-mg92q\" (UniqueName: \"kubernetes.io/projected/aa30756c-e70d-4152-94b4-ad27b87061ca-kube-api-access-mg92q\") pod \"aa30756c-e70d-4152-94b4-ad27b87061ca\" (UID: \"aa30756c-e70d-4152-94b4-ad27b87061ca\") "
Oct 30 22:55:25.537975 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:55:25.537800    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa30756c-e70d-4152-94b4-ad27b87061ca-workload-certs" pod "aa30756c-e70d-4152-94b4-ad27b87061ca" (UID: "aa30756c-e70d-4152-94b4-ad27b87061ca"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:55:25.537975 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:55:25.537817    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aa30756c-e70d-4152-94b4-ad27b87061ca-istiod-ca-cert" pod "aa30756c-e70d-4152-94b4-ad27b87061ca" (UID: "aa30756c-e70d-4152-94b4-ad27b87061ca"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:55:25.538095 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:55:25.537987    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa30756c-e70d-4152-94b4-ad27b87061ca-istio-data" pod "aa30756c-e70d-4152-94b4-ad27b87061ca" (UID: "aa30756c-e70d-4152-94b4-ad27b87061ca"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:55:25.538095 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:55:25.538068    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa30756c-e70d-4152-94b4-ad27b87061ca-workload-socket" pod "aa30756c-e70d-4152-94b4-ad27b87061ca" (UID: "aa30756c-e70d-4152-94b4-ad27b87061ca"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:55:25.538204 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:55:25.538126    5510 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aa30756c-e70d-4152-94b4-ad27b87061ca-workload-certs\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:55:25.538204 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:55:25.538139    5510 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aa30756c-e70d-4152-94b4-ad27b87061ca-istio-data\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:55:25.538204 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:55:25.538154    5510 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa30756c-e70d-4152-94b4-ad27b87061ca-credential-socket\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:55:25.538204 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:55:25.538167    5510 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aa30756c-e70d-4152-94b4-ad27b87061ca-istiod-ca-cert\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:55:25.539648 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:55:25.539621    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/aa30756c-e70d-4152-94b4-ad27b87061ca-istio-podinfo" pod "aa30756c-e70d-4152-94b4-ad27b87061ca" (UID: "aa30756c-e70d-4152-94b4-ad27b87061ca"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:55:25.539775 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:55:25.539685    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aa30756c-e70d-4152-94b4-ad27b87061ca-istio-token" pod "aa30756c-e70d-4152-94b4-ad27b87061ca" (UID: "aa30756c-e70d-4152-94b4-ad27b87061ca"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:55:25.540023 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:55:25.540008    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa30756c-e70d-4152-94b4-ad27b87061ca-istio-envoy" pod "aa30756c-e70d-4152-94b4-ad27b87061ca" (UID: "aa30756c-e70d-4152-94b4-ad27b87061ca"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:55:25.540076 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:55:25.540055    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aa30756c-e70d-4152-94b4-ad27b87061ca-kube-api-access-mg92q" pod "aa30756c-e70d-4152-94b4-ad27b87061ca" (UID: "aa30756c-e70d-4152-94b4-ad27b87061ca"). InnerVolumeSpecName "kube-api-access-mg92q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:55:25.639619 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:55:25.639584    5510 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aa30756c-e70d-4152-94b4-ad27b87061ca-istio-podinfo\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:55:25.639619 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:55:25.639614    5510 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aa30756c-e70d-4152-94b4-ad27b87061ca-istio-token\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:55:25.639619 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:55:25.639623    5510 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa30756c-e70d-4152-94b4-ad27b87061ca-workload-socket\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:55:25.639862 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:55:25.639632    5510 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aa30756c-e70d-4152-94b4-ad27b87061ca-istio-envoy\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:55:25.639862 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:55:25.639641    5510 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mg92q\" (UniqueName: \"kubernetes.io/projected/aa30756c-e70d-4152-94b4-ad27b87061ca-kube-api-access-mg92q\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:55:26.384109 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:55:26.384080    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-single-main-4334bf-openshift-default-76c7b68c89xpql6"
Oct 30 22:55:26.384109 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:55:26.384093    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-4334bf-openshift-default-76c7b68c89xpql6" event={"ID":"aa30756c-e70d-4152-94b4-ad27b87061ca","Type":"ContainerDied","Data":"59f6d210a5dd5f5e552e5772bc7b7145576c730e56061b20daf1f77b389de5c2"}
Oct 30 22:55:26.384590 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:55:26.384148    5510 scope.go:122] "RemoveContainer" containerID="e9ac1d98851f92f029aeee3d0a7970654d2889ba7bf052fe9a319036ff93fa06"
Oct 30 22:55:26.541323 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:55:26.541281    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-single-main-4334bf-openshift-default-76c7b68c89xpql6"]
Oct 30 22:55:26.587677 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:55:26.587634    5510 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-single-main-4334bf-openshift-default-76c7b68c89xpql6"]
Oct 30 22:55:27.552746 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:55:27.552711    5510 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="aa30756c-e70d-4152-94b4-ad27b87061ca" path="/var/lib/kubelet/pods/aa30756c-e70d-4152-94b4-ad27b87061ca/volumes"
Oct 30 22:55:28.166118 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:55:28.166074    5510 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-713f1911-kserve-5f6bf5c57c-cz75f"]
Oct 30 22:55:28.166539 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:55:28.166526    5510 cpu_manager.go:414] "containerMap: removing container" podUID="d069c582-5b79-4a80-93d3-2157f96d17ac" containerName="main"
Oct 30 22:55:28.166591 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:55:28.166540    5510 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d069c582-5b79-4a80-93d3-2157f96d17ac" containerName="main"
Oct 30 22:55:28.166591 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:55:28.166553    5510 cpu_manager.go:414] "containerMap: removing container" podUID="aa30756c-e70d-4152-94b4-ad27b87061ca" containerName="istio-proxy"
Oct 30 22:55:28.166591 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:55:28.166558    5510 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="aa30756c-e70d-4152-94b4-ad27b87061ca" containerName="istio-proxy"
Oct 30 22:55:28.166591 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:55:28.166573    5510 cpu_manager.go:414] "containerMap: removing container" podUID="aa30756c-e70d-4152-94b4-ad27b87061ca" containerName="istio-proxy"
Oct 30 22:55:28.166591 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:55:28.166578    5510 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="aa30756c-e70d-4152-94b4-ad27b87061ca" containerName="istio-proxy"
Oct 30 22:55:28.166794 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:55:28.166635    5510 memory_manager.go:365] "RemoveStaleState removing state" podUID="aa30756c-e70d-4152-94b4-ad27b87061ca" containerName="istio-proxy"
Oct 30 22:55:28.166794 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:55:28.166649    5510 memory_manager.go:365] "RemoveStaleState removing state" podUID="d069c582-5b79-4a80-93d3-2157f96d17ac" containerName="main"
Oct 30 22:55:28.170964 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:55:28.170942    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-713f1911-kserve-5f6bf5c57c-cz75f"
Oct 30 22:55:28.262193 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:55:28.262157    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dd1877bb-c1ff-46ab-99ca-33047d8fd7e8-tmp-dir\") pod \"e2e-same-id-a-713f1911-kserve-5f6bf5c57c-cz75f\" (UID: \"dd1877bb-c1ff-46ab-99ca-33047d8fd7e8\") " pod="llm/e2e-same-id-a-713f1911-kserve-5f6bf5c57c-cz75f"
Oct 30 22:55:28.262403 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:55:28.262223    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rp9hr\" (UniqueName: \"kubernetes.io/projected/dd1877bb-c1ff-46ab-99ca-33047d8fd7e8-kube-api-access-rp9hr\") pod \"e2e-same-id-a-713f1911-kserve-5f6bf5c57c-cz75f\" (UID: \"dd1877bb-c1ff-46ab-99ca-33047d8fd7e8\") " pod="llm/e2e-same-id-a-713f1911-kserve-5f6bf5c57c-cz75f"
Oct 30 22:55:28.262403 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:55:28.262294    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dd1877bb-c1ff-46ab-99ca-33047d8fd7e8-dshm\") pod \"e2e-same-id-a-713f1911-kserve-5f6bf5c57c-cz75f\" (UID: \"dd1877bb-c1ff-46ab-99ca-33047d8fd7e8\") " pod="llm/e2e-same-id-a-713f1911-kserve-5f6bf5c57c-cz75f"
Oct 30 22:55:28.262403 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:55:28.262342    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dd1877bb-c1ff-46ab-99ca-33047d8fd7e8-tls-certs\") pod \"e2e-same-id-a-713f1911-kserve-5f6bf5c57c-cz75f\" (UID: \"dd1877bb-c1ff-46ab-99ca-33047d8fd7e8\") " pod="llm/e2e-same-id-a-713f1911-kserve-5f6bf5c57c-cz75f"
Oct 30 22:55:28.262403 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:55:28.262373    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dd1877bb-c1ff-46ab-99ca-33047d8fd7e8-model-cache\") pod \"e2e-same-id-a-713f1911-kserve-5f6bf5c57c-cz75f\" (UID: \"dd1877bb-c1ff-46ab-99ca-33047d8fd7e8\") " pod="llm/e2e-same-id-a-713f1911-kserve-5f6bf5c57c-cz75f"
Oct 30 22:55:28.262591 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:55:28.262447    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dd1877bb-c1ff-46ab-99ca-33047d8fd7e8-home\") pod \"e2e-same-id-a-713f1911-kserve-5f6bf5c57c-cz75f\" (UID: \"dd1877bb-c1ff-46ab-99ca-33047d8fd7e8\") " pod="llm/e2e-same-id-a-713f1911-kserve-5f6bf5c57c-cz75f"
Oct 30 22:55:28.267042 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:55:28.267018    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-713f1911-kserve-self-signed-certs\""
Oct 30 22:55:28.268712 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:55:28.268692    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-713f1911-kserve-5f6bf5c57c-cz75f"]
Oct 30 22:55:28.357062 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:55:28.357030    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:55:28.363446 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:55:28.363421    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dd1877bb-c1ff-46ab-99ca-33047d8fd7e8-model-cache\") pod \"e2e-same-id-a-713f1911-kserve-5f6bf5c57c-cz75f\" (UID: \"dd1877bb-c1ff-46ab-99ca-33047d8fd7e8\") " pod="llm/e2e-same-id-a-713f1911-kserve-5f6bf5c57c-cz75f"
Oct 30 22:55:28.363583 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:55:28.363515    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dd1877bb-c1ff-46ab-99ca-33047d8fd7e8-home\") pod \"e2e-same-id-a-713f1911-kserve-5f6bf5c57c-cz75f\" (UID: \"dd1877bb-c1ff-46ab-99ca-33047d8fd7e8\") " pod="llm/e2e-same-id-a-713f1911-kserve-5f6bf5c57c-cz75f"
Oct 30 22:55:28.363679 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:55:28.363599    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dd1877bb-c1ff-46ab-99ca-33047d8fd7e8-tmp-dir\") pod \"e2e-same-id-a-713f1911-kserve-5f6bf5c57c-cz75f\" (UID: \"dd1877bb-c1ff-46ab-99ca-33047d8fd7e8\") " pod="llm/e2e-same-id-a-713f1911-kserve-5f6bf5c57c-cz75f"
Oct 30 22:55:28.363679 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:55:28.363652    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rp9hr\" (UniqueName: \"kubernetes.io/projected/dd1877bb-c1ff-46ab-99ca-33047d8fd7e8-kube-api-access-rp9hr\") pod \"e2e-same-id-a-713f1911-kserve-5f6bf5c57c-cz75f\" (UID: \"dd1877bb-c1ff-46ab-99ca-33047d8fd7e8\") " pod="llm/e2e-same-id-a-713f1911-kserve-5f6bf5c57c-cz75f"
Oct 30 22:55:28.363796 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:55:28.363708    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dd1877bb-c1ff-46ab-99ca-33047d8fd7e8-dshm\") pod \"e2e-same-id-a-713f1911-kserve-5f6bf5c57c-cz75f\" (UID: \"dd1877bb-c1ff-46ab-99ca-33047d8fd7e8\") " pod="llm/e2e-same-id-a-713f1911-kserve-5f6bf5c57c-cz75f"
Oct 30 22:55:28.363796 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:55:28.363749    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dd1877bb-c1ff-46ab-99ca-33047d8fd7e8-tls-certs\") pod \"e2e-same-id-a-713f1911-kserve-5f6bf5c57c-cz75f\" (UID: \"dd1877bb-c1ff-46ab-99ca-33047d8fd7e8\") " pod="llm/e2e-same-id-a-713f1911-kserve-5f6bf5c57c-cz75f"
Oct 30 22:55:28.363930 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:55:28.363908    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dd1877bb-c1ff-46ab-99ca-33047d8fd7e8-model-cache\") pod \"e2e-same-id-a-713f1911-kserve-5f6bf5c57c-cz75f\" (UID: \"dd1877bb-c1ff-46ab-99ca-33047d8fd7e8\") " pod="llm/e2e-same-id-a-713f1911-kserve-5f6bf5c57c-cz75f"
Oct 30 22:55:28.364005 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:55:28.363985    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dd1877bb-c1ff-46ab-99ca-33047d8fd7e8-home\") pod \"e2e-same-id-a-713f1911-kserve-5f6bf5c57c-cz75f\" (UID: \"dd1877bb-c1ff-46ab-99ca-33047d8fd7e8\") " pod="llm/e2e-same-id-a-713f1911-kserve-5f6bf5c57c-cz75f"
Oct 30 22:55:28.364063 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:55:28.364033    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dd1877bb-c1ff-46ab-99ca-33047d8fd7e8-tmp-dir\") pod \"e2e-same-id-a-713f1911-kserve-5f6bf5c57c-cz75f\" (UID: \"dd1877bb-c1ff-46ab-99ca-33047d8fd7e8\") " pod="llm/e2e-same-id-a-713f1911-kserve-5f6bf5c57c-cz75f"
Oct 30 22:55:28.366038 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:55:28.366015    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dd1877bb-c1ff-46ab-99ca-33047d8fd7e8-dshm\") pod \"e2e-same-id-a-713f1911-kserve-5f6bf5c57c-cz75f\" (UID: \"dd1877bb-c1ff-46ab-99ca-33047d8fd7e8\") " pod="llm/e2e-same-id-a-713f1911-kserve-5f6bf5c57c-cz75f"
Oct 30 22:55:28.366252 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:55:28.366233    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dd1877bb-c1ff-46ab-99ca-33047d8fd7e8-tls-certs\") pod \"e2e-same-id-a-713f1911-kserve-5f6bf5c57c-cz75f\" (UID: \"dd1877bb-c1ff-46ab-99ca-33047d8fd7e8\") " pod="llm/e2e-same-id-a-713f1911-kserve-5f6bf5c57c-cz75f"
Oct 30 22:55:28.422219 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:55:28.422148    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rp9hr\" (UniqueName: \"kubernetes.io/projected/dd1877bb-c1ff-46ab-99ca-33047d8fd7e8-kube-api-access-rp9hr\") pod \"e2e-same-id-a-713f1911-kserve-5f6bf5c57c-cz75f\" (UID: \"dd1877bb-c1ff-46ab-99ca-33047d8fd7e8\") " pod="llm/e2e-same-id-a-713f1911-kserve-5f6bf5c57c-cz75f"
Oct 30 22:55:28.482591 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:55:28.482543    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-713f1911-kserve-5f6bf5c57c-cz75f"
Oct 30 22:55:28.795597 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:55:28.795560    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-4334bf-7869clrjkp" podUID="b1ecb832-79a2-4265-bb1d-ea52e0f2a699" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:55:28.796002 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:55:28.795556    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-single-main-4334bf-7d64b6cdcslfpj" podUID="8681e23d-a36a-4e71-8113-e364fc1f284e" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:55:28.883955 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:55:28.883917    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-713f1911-kserve-5f6bf5c57c-cz75f"]
Oct 30 22:55:28.891256 ip-10-0-136-210 kubenswrapper[5510]: W1030 22:55:28.891227    5510 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddd1877bb_c1ff_46ab_99ca_33047d8fd7e8.slice/crio-f0ee424b646970aa531cce2e32da21814ec915b181bee95562e8b98dcf5803f2 WatchSource:0}: Error finding container f0ee424b646970aa531cce2e32da21814ec915b181bee95562e8b98dcf5803f2: Status 404 returned error can't find the container with id f0ee424b646970aa531cce2e32da21814ec915b181bee95562e8b98dcf5803f2
Oct 30 22:55:29.401342 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:55:29.401309    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-713f1911-kserve-5f6bf5c57c-cz75f" event={"ID":"dd1877bb-c1ff-46ab-99ca-33047d8fd7e8","Type":"ContainerStarted","Data":"9995d7e3d8fc511717dee37b0ab064e555db7037518c9e57cbbac0df5d908246"}
Oct 30 22:55:29.401342 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:55:29.401347    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-713f1911-kserve-5f6bf5c57c-cz75f" event={"ID":"dd1877bb-c1ff-46ab-99ca-33047d8fd7e8","Type":"ContainerStarted","Data":"f0ee424b646970aa531cce2e32da21814ec915b181bee95562e8b98dcf5803f2"}
Oct 30 22:55:29.483691 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:55:29.483643    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-713f1911-kserve-5f6bf5c57c-cz75f"
Oct 30 22:55:29.515469 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:55:29.515414    5510 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-same-id-a-713f1911-kserve-5f6bf5c57c-cz75f" podStartSLOduration=1.515400313 podStartE2EDuration="1.515400313s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:55:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:55:29.513608898 +0000 UTC m=+3910.074418902" watchObservedRunningTime="2026-10-30 22:55:29.515400313 +0000 UTC m=+3910.076210317"
Oct 30 22:55:33.929136 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:55:33.927798    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-single-main-4334bf-7d64b6cdcslfpj"
Oct 30 22:55:33.999691 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:55:33.999592    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-4334bf-7869clrjkp"
Oct 30 22:55:34.026180 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:55:34.026143    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8681e23d-a36a-4e71-8113-e364fc1f284e-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8681e23d-a36a-4e71-8113-e364fc1f284e-plugins-config-volume\") pod \"8681e23d-a36a-4e71-8113-e364fc1f284e\" (UID: \"8681e23d-a36a-4e71-8113-e364fc1f284e\") "
Oct 30 22:55:34.026333 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:55:34.026260    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b1ecb832-79a2-4265-bb1d-ea52e0f2a699-kube-api-access-29tt8\" (UniqueName: \"kubernetes.io/projected/b1ecb832-79a2-4265-bb1d-ea52e0f2a699-kube-api-access-29tt8\") pod \"b1ecb832-79a2-4265-bb1d-ea52e0f2a699\" (UID: \"b1ecb832-79a2-4265-bb1d-ea52e0f2a699\") "
Oct 30 22:55:34.026404 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:55:34.026344    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b1ecb832-79a2-4265-bb1d-ea52e0f2a699-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b1ecb832-79a2-4265-bb1d-ea52e0f2a699-plugins-config-volume\") pod \"b1ecb832-79a2-4265-bb1d-ea52e0f2a699\" (UID: \"b1ecb832-79a2-4265-bb1d-ea52e0f2a699\") "
Oct 30 22:55:34.026404 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:55:34.026391    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8681e23d-a36a-4e71-8113-e364fc1f284e-kube-api-access-rlp85\" (UniqueName: \"kubernetes.io/projected/8681e23d-a36a-4e71-8113-e364fc1f284e-kube-api-access-rlp85\") pod \"8681e23d-a36a-4e71-8113-e364fc1f284e\" (UID: \"8681e23d-a36a-4e71-8113-e364fc1f284e\") "
Oct 30 22:55:34.026594 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:55:34.026563    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8681e23d-a36a-4e71-8113-e364fc1f284e-plugins-config-volume" pod "8681e23d-a36a-4e71-8113-e364fc1f284e" (UID: "8681e23d-a36a-4e71-8113-e364fc1f284e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:55:34.026805 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:55:34.026783    5510 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8681e23d-a36a-4e71-8113-e364fc1f284e-plugins-config-volume\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:55:34.027333 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:55:34.027295    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b1ecb832-79a2-4265-bb1d-ea52e0f2a699-plugins-config-volume" pod "b1ecb832-79a2-4265-bb1d-ea52e0f2a699" (UID: "b1ecb832-79a2-4265-bb1d-ea52e0f2a699"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:55:34.029231 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:55:34.029200    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8681e23d-a36a-4e71-8113-e364fc1f284e-kube-api-access-rlp85" pod "8681e23d-a36a-4e71-8113-e364fc1f284e" (UID: "8681e23d-a36a-4e71-8113-e364fc1f284e"). InnerVolumeSpecName "kube-api-access-rlp85". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:55:34.029395 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:55:34.029370    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b1ecb832-79a2-4265-bb1d-ea52e0f2a699-kube-api-access-29tt8" pod "b1ecb832-79a2-4265-bb1d-ea52e0f2a699" (UID: "b1ecb832-79a2-4265-bb1d-ea52e0f2a699"). InnerVolumeSpecName "kube-api-access-29tt8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:55:34.128278 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:55:34.128237    5510 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-29tt8\" (UniqueName: \"kubernetes.io/projected/b1ecb832-79a2-4265-bb1d-ea52e0f2a699-kube-api-access-29tt8\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:55:34.128278 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:55:34.128280    5510 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b1ecb832-79a2-4265-bb1d-ea52e0f2a699-plugins-config-volume\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:55:34.128550 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:55:34.128297    5510 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rlp85\" (UniqueName: \"kubernetes.io/projected/8681e23d-a36a-4e71-8113-e364fc1f284e-kube-api-access-rlp85\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:55:34.466714 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:55:34.462720    5510 generic.go:356] "Generic (PLEG): container finished" podID="8681e23d-a36a-4e71-8113-e364fc1f284e" containerID="d384b5daf89b448dd8e90496396c6e065e277a78612cd4fd7f1c20aba1603b79" exitCode=0
Oct 30 22:55:34.466714 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:55:34.462850    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-single-main-4334bf-7d64b6cdcslfpj" event={"ID":"8681e23d-a36a-4e71-8113-e364fc1f284e","Type":"ContainerDied","Data":"d384b5daf89b448dd8e90496396c6e065e277a78612cd4fd7f1c20aba1603b79"}
Oct 30 22:55:34.466714 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:55:34.462904    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-single-main-4334bf-7d64b6cdcslfpj" event={"ID":"8681e23d-a36a-4e71-8113-e364fc1f284e","Type":"ContainerDied","Data":"43cc8f012ede96a659dcbfbe3b713f2e3027988d9ea5ad3d2012dce5b980ac2b"}
Oct 30 22:55:34.466714 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:55:34.462933    5510 scope.go:122] "RemoveContainer" containerID="d384b5daf89b448dd8e90496396c6e065e277a78612cd4fd7f1c20aba1603b79"
Oct 30 22:55:34.466714 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:55:34.463080    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-single-main-4334bf-7d64b6cdcslfpj"
Oct 30 22:55:34.487830 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:55:34.487763    5510 generic.go:356] "Generic (PLEG): container finished" podID="b1ecb832-79a2-4265-bb1d-ea52e0f2a699" containerID="89f186cf01bb4514314b19602b8b9cf9b6488575c2c18c1e60583dc743a4257c" exitCode=0
Oct 30 22:55:34.488139 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:55:34.488107    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-4334bf-7869clrjkp" event={"ID":"b1ecb832-79a2-4265-bb1d-ea52e0f2a699","Type":"ContainerDied","Data":"89f186cf01bb4514314b19602b8b9cf9b6488575c2c18c1e60583dc743a4257c"}
Oct 30 22:55:34.488374 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:55:34.488314    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-4334bf-7869clrjkp" event={"ID":"b1ecb832-79a2-4265-bb1d-ea52e0f2a699","Type":"ContainerDied","Data":"1e704b32b1d8904c1e6fac667b6ed590c06adde376ad22776b805b7628efd940"}
Oct 30 22:55:34.488604 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:55:34.488547    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-4334bf-7869clrjkp"
Oct 30 22:55:34.557787 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:55:34.555810    5510 scope.go:122] "RemoveContainer" containerID="d384b5daf89b448dd8e90496396c6e065e277a78612cd4fd7f1c20aba1603b79"
Oct 30 22:55:34.557787 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:55:34.556269    5510 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d384b5daf89b448dd8e90496396c6e065e277a78612cd4fd7f1c20aba1603b79\": container with ID starting with d384b5daf89b448dd8e90496396c6e065e277a78612cd4fd7f1c20aba1603b79 not found: ID does not exist" containerID="d384b5daf89b448dd8e90496396c6e065e277a78612cd4fd7f1c20aba1603b79"
Oct 30 22:55:34.557787 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:55:34.556306    5510 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d384b5daf89b448dd8e90496396c6e065e277a78612cd4fd7f1c20aba1603b79"} err="failed to get container status \"d384b5daf89b448dd8e90496396c6e065e277a78612cd4fd7f1c20aba1603b79\": rpc error: code = NotFound desc = could not find container \"d384b5daf89b448dd8e90496396c6e065e277a78612cd4fd7f1c20aba1603b79\": container with ID starting with d384b5daf89b448dd8e90496396c6e065e277a78612cd4fd7f1c20aba1603b79 not found: ID does not exist"
Oct 30 22:55:34.557787 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:55:34.556335    5510 scope.go:122] "RemoveContainer" containerID="89f186cf01bb4514314b19602b8b9cf9b6488575c2c18c1e60583dc743a4257c"
Oct 30 22:55:34.624483 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:55:34.623974    5510 scope.go:122] "RemoveContainer" containerID="89f186cf01bb4514314b19602b8b9cf9b6488575c2c18c1e60583dc743a4257c"
Oct 30 22:55:34.624483 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:55:34.624368    5510 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"89f186cf01bb4514314b19602b8b9cf9b6488575c2c18c1e60583dc743a4257c\": container with ID starting with 89f186cf01bb4514314b19602b8b9cf9b6488575c2c18c1e60583dc743a4257c not found: ID does not exist" containerID="89f186cf01bb4514314b19602b8b9cf9b6488575c2c18c1e60583dc743a4257c"
Oct 30 22:55:34.624483 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:55:34.624408    5510 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"89f186cf01bb4514314b19602b8b9cf9b6488575c2c18c1e60583dc743a4257c"} err="failed to get container status \"89f186cf01bb4514314b19602b8b9cf9b6488575c2c18c1e60583dc743a4257c\": rpc error: code = NotFound desc = could not find container \"89f186cf01bb4514314b19602b8b9cf9b6488575c2c18c1e60583dc743a4257c\": container with ID starting with 89f186cf01bb4514314b19602b8b9cf9b6488575c2c18c1e60583dc743a4257c not found: ID does not exist"
Oct 30 22:55:34.702374 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:55:34.702335    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-4334bf-7d64b6cdcslfpj"]
Oct 30 22:55:34.704372 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:55:34.704343    5510 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-4334bf-7d64b6cdcslfpj"]
Oct 30 22:55:34.804695 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:55:34.804634    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-4334bf-7869clrjkp"]
Oct 30 22:55:34.817840 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:55:34.817812    5510 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-4334bf-7869clrjkp"]
Oct 30 22:55:35.575681 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:55:35.575189    5510 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8681e23d-a36a-4e71-8113-e364fc1f284e" path="/var/lib/kubelet/pods/8681e23d-a36a-4e71-8113-e364fc1f284e/volumes"
Oct 30 22:55:35.592686 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:55:35.591782    5510 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b1ecb832-79a2-4265-bb1d-ea52e0f2a699" path="/var/lib/kubelet/pods/b1ecb832-79a2-4265-bb1d-ea52e0f2a699/volumes"
Oct 30 22:55:36.255350 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:55:36.254813    5510 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-85544d4cd7-hfrbk container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.149:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 22:55:36.255350 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:55:36.254899    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-85544d4cd7-hfrbk" podUID="474fc2c2-03ae-420b-a831-c8e98a3e5ed8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.149:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:55:36.351181 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:55:36.351136    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:55:36.540199 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:55:36.540160    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:55:38.419825 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:55:38.419790    5510 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6gjbf" containerName="extractor"
Oct 30 22:55:38.483334 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:55:38.483301    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-713f1911-kserve-5f6bf5c57c-cz75f"
Oct 30 22:55:38.495861 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:55:38.495831    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-713f1911-kserve-5f6bf5c57c-cz75f"
Oct 30 22:55:38.538562 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:55:38.538530    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-713f1911-kserve-5f6bf5c57c-cz75f"
Oct 30 22:55:40.070670 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:55:40.070617    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:55:40.520035 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:55:40.520000    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:55:40.712375 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:55:40.712339    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:55:55.220096 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:55:55.220060    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:56:08.419091 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:56:08.419050    5510 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6gjbf" containerName="extractor"
Oct 30 22:56:10.063683 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:56:10.063634    5510 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-713f1911-kserve-b89899d97-rfjpc"]
Oct 30 22:56:10.064092 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:56:10.064067    5510 cpu_manager.go:414] "containerMap: removing container" podUID="8681e23d-a36a-4e71-8113-e364fc1f284e" containerName="payload-processing"
Oct 30 22:56:10.064092 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:56:10.064078    5510 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8681e23d-a36a-4e71-8113-e364fc1f284e" containerName="payload-processing"
Oct 30 22:56:10.064092 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:56:10.064090    5510 cpu_manager.go:414] "containerMap: removing container" podUID="b1ecb832-79a2-4265-bb1d-ea52e0f2a699" containerName="payload-pre-processing"
Oct 30 22:56:10.064203 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:56:10.064096    5510 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b1ecb832-79a2-4265-bb1d-ea52e0f2a699" containerName="payload-pre-processing"
Oct 30 22:56:10.064203 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:56:10.064179    5510 memory_manager.go:365] "RemoveStaleState removing state" podUID="8681e23d-a36a-4e71-8113-e364fc1f284e" containerName="payload-processing"
Oct 30 22:56:10.064203 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:56:10.064187    5510 memory_manager.go:365] "RemoveStaleState removing state" podUID="b1ecb832-79a2-4265-bb1d-ea52e0f2a699" containerName="payload-pre-processing"
Oct 30 22:56:10.064203 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:56:10.064198    5510 memory_manager.go:365] "RemoveStaleState removing state" podUID="aa30756c-e70d-4152-94b4-ad27b87061ca" containerName="istio-proxy"
Oct 30 22:56:10.067076 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:56:10.067048    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-713f1911-kserve-b89899d97-rfjpc"
Oct 30 22:56:10.159571 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:56:10.159533    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/63d6dfdb-38b8-46f2-8d67-e28787857fbf-home\") pod \"e2e-same-id-b-713f1911-kserve-b89899d97-rfjpc\" (UID: \"63d6dfdb-38b8-46f2-8d67-e28787857fbf\") " pod="llm/e2e-same-id-b-713f1911-kserve-b89899d97-rfjpc"
Oct 30 22:56:10.159777 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:56:10.159627    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/63d6dfdb-38b8-46f2-8d67-e28787857fbf-dshm\") pod \"e2e-same-id-b-713f1911-kserve-b89899d97-rfjpc\" (UID: \"63d6dfdb-38b8-46f2-8d67-e28787857fbf\") " pod="llm/e2e-same-id-b-713f1911-kserve-b89899d97-rfjpc"
Oct 30 22:56:10.159777 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:56:10.159709    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/63d6dfdb-38b8-46f2-8d67-e28787857fbf-tls-certs\") pod \"e2e-same-id-b-713f1911-kserve-b89899d97-rfjpc\" (UID: \"63d6dfdb-38b8-46f2-8d67-e28787857fbf\") " pod="llm/e2e-same-id-b-713f1911-kserve-b89899d97-rfjpc"
Oct 30 22:56:10.159777 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:56:10.159743    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/63d6dfdb-38b8-46f2-8d67-e28787857fbf-tmp-dir\") pod \"e2e-same-id-b-713f1911-kserve-b89899d97-rfjpc\" (UID: \"63d6dfdb-38b8-46f2-8d67-e28787857fbf\") " pod="llm/e2e-same-id-b-713f1911-kserve-b89899d97-rfjpc"
Oct 30 22:56:10.159914 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:56:10.159900    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/63d6dfdb-38b8-46f2-8d67-e28787857fbf-model-cache\") pod \"e2e-same-id-b-713f1911-kserve-b89899d97-rfjpc\" (UID: \"63d6dfdb-38b8-46f2-8d67-e28787857fbf\") " pod="llm/e2e-same-id-b-713f1911-kserve-b89899d97-rfjpc"
Oct 30 22:56:10.159952 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:56:10.159940    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fj8c5\" (UniqueName: \"kubernetes.io/projected/63d6dfdb-38b8-46f2-8d67-e28787857fbf-kube-api-access-fj8c5\") pod \"e2e-same-id-b-713f1911-kserve-b89899d97-rfjpc\" (UID: \"63d6dfdb-38b8-46f2-8d67-e28787857fbf\") " pod="llm/e2e-same-id-b-713f1911-kserve-b89899d97-rfjpc"
Oct 30 22:56:10.163737 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:56:10.163717    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-713f1911-kserve-self-signed-certs\""
Oct 30 22:56:10.165338 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:56:10.165320    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-713f1911-kserve-b89899d97-rfjpc"]
Oct 30 22:56:10.204410 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:56:10.204384    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:56:10.260509 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:56:10.260471    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/63d6dfdb-38b8-46f2-8d67-e28787857fbf-model-cache\") pod \"e2e-same-id-b-713f1911-kserve-b89899d97-rfjpc\" (UID: \"63d6dfdb-38b8-46f2-8d67-e28787857fbf\") " pod="llm/e2e-same-id-b-713f1911-kserve-b89899d97-rfjpc"
Oct 30 22:56:10.260720 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:56:10.260526    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fj8c5\" (UniqueName: \"kubernetes.io/projected/63d6dfdb-38b8-46f2-8d67-e28787857fbf-kube-api-access-fj8c5\") pod \"e2e-same-id-b-713f1911-kserve-b89899d97-rfjpc\" (UID: \"63d6dfdb-38b8-46f2-8d67-e28787857fbf\") " pod="llm/e2e-same-id-b-713f1911-kserve-b89899d97-rfjpc"
Oct 30 22:56:10.260720 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:56:10.260556    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/63d6dfdb-38b8-46f2-8d67-e28787857fbf-home\") pod \"e2e-same-id-b-713f1911-kserve-b89899d97-rfjpc\" (UID: \"63d6dfdb-38b8-46f2-8d67-e28787857fbf\") " pod="llm/e2e-same-id-b-713f1911-kserve-b89899d97-rfjpc"
Oct 30 22:56:10.260720 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:56:10.260578    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/63d6dfdb-38b8-46f2-8d67-e28787857fbf-dshm\") pod \"e2e-same-id-b-713f1911-kserve-b89899d97-rfjpc\" (UID: \"63d6dfdb-38b8-46f2-8d67-e28787857fbf\") " pod="llm/e2e-same-id-b-713f1911-kserve-b89899d97-rfjpc"
Oct 30 22:56:10.260720 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:56:10.260602    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/63d6dfdb-38b8-46f2-8d67-e28787857fbf-tls-certs\") pod \"e2e-same-id-b-713f1911-kserve-b89899d97-rfjpc\" (UID: \"63d6dfdb-38b8-46f2-8d67-e28787857fbf\") " pod="llm/e2e-same-id-b-713f1911-kserve-b89899d97-rfjpc"
Oct 30 22:56:10.260720 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:56:10.260627    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/63d6dfdb-38b8-46f2-8d67-e28787857fbf-tmp-dir\") pod \"e2e-same-id-b-713f1911-kserve-b89899d97-rfjpc\" (UID: \"63d6dfdb-38b8-46f2-8d67-e28787857fbf\") " pod="llm/e2e-same-id-b-713f1911-kserve-b89899d97-rfjpc"
Oct 30 22:56:10.260989 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:56:10.260973    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/63d6dfdb-38b8-46f2-8d67-e28787857fbf-model-cache\") pod \"e2e-same-id-b-713f1911-kserve-b89899d97-rfjpc\" (UID: \"63d6dfdb-38b8-46f2-8d67-e28787857fbf\") " pod="llm/e2e-same-id-b-713f1911-kserve-b89899d97-rfjpc"
Oct 30 22:56:10.261058 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:56:10.261028    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/63d6dfdb-38b8-46f2-8d67-e28787857fbf-home\") pod \"e2e-same-id-b-713f1911-kserve-b89899d97-rfjpc\" (UID: \"63d6dfdb-38b8-46f2-8d67-e28787857fbf\") " pod="llm/e2e-same-id-b-713f1911-kserve-b89899d97-rfjpc"
Oct 30 22:56:10.261149 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:56:10.261126    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/63d6dfdb-38b8-46f2-8d67-e28787857fbf-tmp-dir\") pod \"e2e-same-id-b-713f1911-kserve-b89899d97-rfjpc\" (UID: \"63d6dfdb-38b8-46f2-8d67-e28787857fbf\") " pod="llm/e2e-same-id-b-713f1911-kserve-b89899d97-rfjpc"
Oct 30 22:56:10.262876 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:56:10.262854    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/63d6dfdb-38b8-46f2-8d67-e28787857fbf-dshm\") pod \"e2e-same-id-b-713f1911-kserve-b89899d97-rfjpc\" (UID: \"63d6dfdb-38b8-46f2-8d67-e28787857fbf\") " pod="llm/e2e-same-id-b-713f1911-kserve-b89899d97-rfjpc"
Oct 30 22:56:10.263248 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:56:10.263229    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/63d6dfdb-38b8-46f2-8d67-e28787857fbf-tls-certs\") pod \"e2e-same-id-b-713f1911-kserve-b89899d97-rfjpc\" (UID: \"63d6dfdb-38b8-46f2-8d67-e28787857fbf\") " pod="llm/e2e-same-id-b-713f1911-kserve-b89899d97-rfjpc"
Oct 30 22:56:10.320995 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:56:10.320501    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fj8c5\" (UniqueName: \"kubernetes.io/projected/63d6dfdb-38b8-46f2-8d67-e28787857fbf-kube-api-access-fj8c5\") pod \"e2e-same-id-b-713f1911-kserve-b89899d97-rfjpc\" (UID: \"63d6dfdb-38b8-46f2-8d67-e28787857fbf\") " pod="llm/e2e-same-id-b-713f1911-kserve-b89899d97-rfjpc"
Oct 30 22:56:10.378636 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:56:10.378599    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-713f1911-kserve-b89899d97-rfjpc"
Oct 30 22:56:10.759011 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:56:10.758924    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-713f1911-kserve-b89899d97-rfjpc"]
Oct 30 22:56:10.766418 ip-10-0-136-210 kubenswrapper[5510]: W1030 22:56:10.766392    5510 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod63d6dfdb_38b8_46f2_8d67_e28787857fbf.slice/crio-6786877d88fccd105073f94c6d9497822dddd41920d91a60e3252c7c005208c3 WatchSource:0}: Error finding container 6786877d88fccd105073f94c6d9497822dddd41920d91a60e3252c7c005208c3: Status 404 returned error can't find the container with id 6786877d88fccd105073f94c6d9497822dddd41920d91a60e3252c7c005208c3
Oct 30 22:56:11.697265 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:56:11.697229    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-713f1911-kserve-b89899d97-rfjpc" event={"ID":"63d6dfdb-38b8-46f2-8d67-e28787857fbf","Type":"ContainerStarted","Data":"18ce25271dba69f2e8782e823d22bc8e97f050dfbb26e76e911da547e5837960"}
Oct 30 22:56:11.697752 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:56:11.697274    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-713f1911-kserve-b89899d97-rfjpc" event={"ID":"63d6dfdb-38b8-46f2-8d67-e28787857fbf","Type":"ContainerStarted","Data":"6786877d88fccd105073f94c6d9497822dddd41920d91a60e3252c7c005208c3"}
Oct 30 22:56:11.809175 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:56:11.809118    5510 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-same-id-b-713f1911-kserve-b89899d97-rfjpc" podStartSLOduration=1.809103221 podStartE2EDuration="1.809103221s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:56:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:56:11.80802079 +0000 UTC m=+3952.368830823" watchObservedRunningTime="2026-10-30 22:56:11.809103221 +0000 UTC m=+3952.369913225"
Oct 30 22:56:12.379581 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:56:12.379543    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-713f1911-kserve-b89899d97-rfjpc"
Oct 30 22:56:20.378897 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:56:20.378853    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-713f1911-kserve-b89899d97-rfjpc"
Oct 30 22:56:20.391359 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:56:20.391332    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-713f1911-kserve-b89899d97-rfjpc"
Oct 30 22:56:20.789237 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:56:20.789207    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-713f1911-kserve-b89899d97-rfjpc"
Oct 30 22:56:38.419327 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:56:38.419289    5510 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6gjbf" containerName="extractor"
Oct 30 22:56:40.968751 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:56:40.968702    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:56:56.435178 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:56:56.435139    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-713f1911-kserve-5f6bf5c57c-cz75f"]
Oct 30 22:56:56.531198 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:56:56.531147    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:57:08.419832 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:57:08.419788    5510 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6gjbf" containerName="extractor"
Oct 30 22:57:08.729811 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:57:08.729724    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-713f1911-kserve-b89899d97-rfjpc"]
Oct 30 22:57:08.816584 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:57:08.816543    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:57:11.456024 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:57:11.455986    5510 kuberuntime_container.go:898] "Killing container with a grace period" pod="llm/e2e-same-id-a-713f1911-kserve-5f6bf5c57c-cz75f" podUID="dd1877bb-c1ff-46ab-99ca-33047d8fd7e8" containerName="main" containerID="cri-o://9995d7e3d8fc511717dee37b0ab064e555db7037518c9e57cbbac0df5d908246" gracePeriod=45
Oct 30 22:57:11.528419 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:57:11.528379    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/e2e-same-id-a-713f1911-kserve-5f6bf5c57c-cz75f" podUID="dd1877bb-c1ff-46ab-99ca-33047d8fd7e8" containerName="main" probeResult="failure" output="Get \"https://10.133.0.164:8000/ready\": dial tcp 10.133.0.164:8000: connect: connection refused"
Oct 30 22:57:11.702891 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:57:11.702861    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-713f1911-kserve-5f6bf5c57c-cz75f"
Oct 30 22:57:11.866115 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:57:11.866082    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/dd1877bb-c1ff-46ab-99ca-33047d8fd7e8-kube-api-access-rp9hr\" (UniqueName: \"kubernetes.io/projected/dd1877bb-c1ff-46ab-99ca-33047d8fd7e8-kube-api-access-rp9hr\") pod \"dd1877bb-c1ff-46ab-99ca-33047d8fd7e8\" (UID: \"dd1877bb-c1ff-46ab-99ca-33047d8fd7e8\") "
Oct 30 22:57:11.866326 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:57:11.866151    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dd1877bb-c1ff-46ab-99ca-33047d8fd7e8-home\" (UniqueName: \"kubernetes.io/empty-dir/dd1877bb-c1ff-46ab-99ca-33047d8fd7e8-home\") pod \"dd1877bb-c1ff-46ab-99ca-33047d8fd7e8\" (UID: \"dd1877bb-c1ff-46ab-99ca-33047d8fd7e8\") "
Oct 30 22:57:11.866326 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:57:11.866182    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/dd1877bb-c1ff-46ab-99ca-33047d8fd7e8-tls-certs\" (UniqueName: \"kubernetes.io/secret/dd1877bb-c1ff-46ab-99ca-33047d8fd7e8-tls-certs\") pod \"dd1877bb-c1ff-46ab-99ca-33047d8fd7e8\" (UID: \"dd1877bb-c1ff-46ab-99ca-33047d8fd7e8\") "
Oct 30 22:57:11.866326 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:57:11.866213    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dd1877bb-c1ff-46ab-99ca-33047d8fd7e8-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dd1877bb-c1ff-46ab-99ca-33047d8fd7e8-tmp-dir\") pod \"dd1877bb-c1ff-46ab-99ca-33047d8fd7e8\" (UID: \"dd1877bb-c1ff-46ab-99ca-33047d8fd7e8\") "
Oct 30 22:57:11.866326 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:57:11.866313    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dd1877bb-c1ff-46ab-99ca-33047d8fd7e8-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dd1877bb-c1ff-46ab-99ca-33047d8fd7e8-model-cache\") pod \"dd1877bb-c1ff-46ab-99ca-33047d8fd7e8\" (UID: \"dd1877bb-c1ff-46ab-99ca-33047d8fd7e8\") "
Oct 30 22:57:11.866562 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:57:11.866346    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dd1877bb-c1ff-46ab-99ca-33047d8fd7e8-dshm\" (UniqueName: \"kubernetes.io/empty-dir/dd1877bb-c1ff-46ab-99ca-33047d8fd7e8-dshm\") pod \"dd1877bb-c1ff-46ab-99ca-33047d8fd7e8\" (UID: \"dd1877bb-c1ff-46ab-99ca-33047d8fd7e8\") "
Oct 30 22:57:11.866562 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:57:11.866445    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd1877bb-c1ff-46ab-99ca-33047d8fd7e8-home" pod "dd1877bb-c1ff-46ab-99ca-33047d8fd7e8" (UID: "dd1877bb-c1ff-46ab-99ca-33047d8fd7e8"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:57:11.866562 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:57:11.866514    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd1877bb-c1ff-46ab-99ca-33047d8fd7e8-tmp-dir" pod "dd1877bb-c1ff-46ab-99ca-33047d8fd7e8" (UID: "dd1877bb-c1ff-46ab-99ca-33047d8fd7e8"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:57:11.866736 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:57:11.866606    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd1877bb-c1ff-46ab-99ca-33047d8fd7e8-model-cache" pod "dd1877bb-c1ff-46ab-99ca-33047d8fd7e8" (UID: "dd1877bb-c1ff-46ab-99ca-33047d8fd7e8"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:57:11.866808 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:57:11.866754    5510 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dd1877bb-c1ff-46ab-99ca-33047d8fd7e8-model-cache\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:57:11.866808 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:57:11.866769    5510 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dd1877bb-c1ff-46ab-99ca-33047d8fd7e8-home\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:57:11.866808 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:57:11.866778    5510 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dd1877bb-c1ff-46ab-99ca-33047d8fd7e8-tmp-dir\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:57:11.868634 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:57:11.868605    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd1877bb-c1ff-46ab-99ca-33047d8fd7e8-dshm" pod "dd1877bb-c1ff-46ab-99ca-33047d8fd7e8" (UID: "dd1877bb-c1ff-46ab-99ca-33047d8fd7e8"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:57:11.868634 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:57:11.868621    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dd1877bb-c1ff-46ab-99ca-33047d8fd7e8-tls-certs" pod "dd1877bb-c1ff-46ab-99ca-33047d8fd7e8" (UID: "dd1877bb-c1ff-46ab-99ca-33047d8fd7e8"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:57:11.868817 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:57:11.868612    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dd1877bb-c1ff-46ab-99ca-33047d8fd7e8-kube-api-access-rp9hr" pod "dd1877bb-c1ff-46ab-99ca-33047d8fd7e8" (UID: "dd1877bb-c1ff-46ab-99ca-33047d8fd7e8"). InnerVolumeSpecName "kube-api-access-rp9hr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:57:11.968188 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:57:11.968148    5510 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dd1877bb-c1ff-46ab-99ca-33047d8fd7e8-dshm\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:57:11.968188 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:57:11.968183    5510 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rp9hr\" (UniqueName: \"kubernetes.io/projected/dd1877bb-c1ff-46ab-99ca-33047d8fd7e8-kube-api-access-rp9hr\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:57:11.968188 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:57:11.968193    5510 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dd1877bb-c1ff-46ab-99ca-33047d8fd7e8-tls-certs\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:57:12.079459 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:57:12.079422    5510 generic.go:356] "Generic (PLEG): container finished" podID="dd1877bb-c1ff-46ab-99ca-33047d8fd7e8" containerID="9995d7e3d8fc511717dee37b0ab064e555db7037518c9e57cbbac0df5d908246" exitCode=0
Oct 30 22:57:12.079699 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:57:12.079469    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-713f1911-kserve-5f6bf5c57c-cz75f"
Oct 30 22:57:12.079699 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:57:12.079508    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-713f1911-kserve-5f6bf5c57c-cz75f" event={"ID":"dd1877bb-c1ff-46ab-99ca-33047d8fd7e8","Type":"ContainerDied","Data":"9995d7e3d8fc511717dee37b0ab064e555db7037518c9e57cbbac0df5d908246"}
Oct 30 22:57:12.079699 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:57:12.079545    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-713f1911-kserve-5f6bf5c57c-cz75f" event={"ID":"dd1877bb-c1ff-46ab-99ca-33047d8fd7e8","Type":"ContainerDied","Data":"f0ee424b646970aa531cce2e32da21814ec915b181bee95562e8b98dcf5803f2"}
Oct 30 22:57:12.079699 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:57:12.079567    5510 scope.go:122] "RemoveContainer" containerID="9995d7e3d8fc511717dee37b0ab064e555db7037518c9e57cbbac0df5d908246"
Oct 30 22:57:12.089880 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:57:12.089852    5510 scope.go:122] "RemoveContainer" containerID="9995d7e3d8fc511717dee37b0ab064e555db7037518c9e57cbbac0df5d908246"
Oct 30 22:57:12.090164 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:57:12.090142    5510 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9995d7e3d8fc511717dee37b0ab064e555db7037518c9e57cbbac0df5d908246\": container with ID starting with 9995d7e3d8fc511717dee37b0ab064e555db7037518c9e57cbbac0df5d908246 not found: ID does not exist" containerID="9995d7e3d8fc511717dee37b0ab064e555db7037518c9e57cbbac0df5d908246"
Oct 30 22:57:12.090219 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:57:12.090176    5510 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9995d7e3d8fc511717dee37b0ab064e555db7037518c9e57cbbac0df5d908246"} err="failed to get container status \"9995d7e3d8fc511717dee37b0ab064e555db7037518c9e57cbbac0df5d908246\": rpc error: code = NotFound desc = could not find container \"9995d7e3d8fc511717dee37b0ab064e555db7037518c9e57cbbac0df5d908246\": container with ID starting with 9995d7e3d8fc511717dee37b0ab064e555db7037518c9e57cbbac0df5d908246 not found: ID does not exist"
Oct 30 22:57:12.233251 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:57:12.233160    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-713f1911-kserve-5f6bf5c57c-cz75f"]
Oct 30 22:57:12.280110 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:57:12.280080    5510 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-713f1911-kserve-5f6bf5c57c-cz75f"]
Oct 30 22:57:13.552726 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:57:13.552694    5510 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="dd1877bb-c1ff-46ab-99ca-33047d8fd7e8" path="/var/lib/kubelet/pods/dd1877bb-c1ff-46ab-99ca-33047d8fd7e8/volumes"
Oct 30 22:57:23.749779 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:57:23.749732    5510 kuberuntime_container.go:898] "Killing container with a grace period" pod="llm/e2e-same-id-b-713f1911-kserve-b89899d97-rfjpc" podUID="63d6dfdb-38b8-46f2-8d67-e28787857fbf" containerName="main" containerID="cri-o://18ce25271dba69f2e8782e823d22bc8e97f050dfbb26e76e911da547e5837960" gracePeriod=45
Oct 30 22:57:23.778998 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:57:23.778966    5510 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/e2e-same-id-b-713f1911-kserve-b89899d97-rfjpc" podUID="63d6dfdb-38b8-46f2-8d67-e28787857fbf" containerName="main" probeResult="failure" output="Get \"https://10.133.0.165:8000/ready\": dial tcp 10.133.0.165:8000: connect: connection refused"
Oct 30 22:57:24.001559 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:57:24.001532    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-713f1911-kserve-b89899d97-rfjpc"
Oct 30 22:57:24.126748 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:57:24.126648    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/63d6dfdb-38b8-46f2-8d67-e28787857fbf-kube-api-access-fj8c5\" (UniqueName: \"kubernetes.io/projected/63d6dfdb-38b8-46f2-8d67-e28787857fbf-kube-api-access-fj8c5\") pod \"63d6dfdb-38b8-46f2-8d67-e28787857fbf\" (UID: \"63d6dfdb-38b8-46f2-8d67-e28787857fbf\") "
Oct 30 22:57:24.126929 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:57:24.126752    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/63d6dfdb-38b8-46f2-8d67-e28787857fbf-dshm\" (UniqueName: \"kubernetes.io/empty-dir/63d6dfdb-38b8-46f2-8d67-e28787857fbf-dshm\") pod \"63d6dfdb-38b8-46f2-8d67-e28787857fbf\" (UID: \"63d6dfdb-38b8-46f2-8d67-e28787857fbf\") "
Oct 30 22:57:24.126929 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:57:24.126787    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/63d6dfdb-38b8-46f2-8d67-e28787857fbf-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/63d6dfdb-38b8-46f2-8d67-e28787857fbf-model-cache\") pod \"63d6dfdb-38b8-46f2-8d67-e28787857fbf\" (UID: \"63d6dfdb-38b8-46f2-8d67-e28787857fbf\") "
Oct 30 22:57:24.126929 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:57:24.126806    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/63d6dfdb-38b8-46f2-8d67-e28787857fbf-home\" (UniqueName: \"kubernetes.io/empty-dir/63d6dfdb-38b8-46f2-8d67-e28787857fbf-home\") pod \"63d6dfdb-38b8-46f2-8d67-e28787857fbf\" (UID: \"63d6dfdb-38b8-46f2-8d67-e28787857fbf\") "
Oct 30 22:57:24.126929 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:57:24.126841    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/63d6dfdb-38b8-46f2-8d67-e28787857fbf-tls-certs\" (UniqueName: \"kubernetes.io/secret/63d6dfdb-38b8-46f2-8d67-e28787857fbf-tls-certs\") pod \"63d6dfdb-38b8-46f2-8d67-e28787857fbf\" (UID: \"63d6dfdb-38b8-46f2-8d67-e28787857fbf\") "
Oct 30 22:57:24.126929 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:57:24.126877    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/63d6dfdb-38b8-46f2-8d67-e28787857fbf-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/63d6dfdb-38b8-46f2-8d67-e28787857fbf-tmp-dir\") pod \"63d6dfdb-38b8-46f2-8d67-e28787857fbf\" (UID: \"63d6dfdb-38b8-46f2-8d67-e28787857fbf\") "
Oct 30 22:57:24.127222 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:57:24.127075    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/63d6dfdb-38b8-46f2-8d67-e28787857fbf-model-cache" pod "63d6dfdb-38b8-46f2-8d67-e28787857fbf" (UID: "63d6dfdb-38b8-46f2-8d67-e28787857fbf"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:57:24.127222 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:57:24.127098    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/63d6dfdb-38b8-46f2-8d67-e28787857fbf-home" pod "63d6dfdb-38b8-46f2-8d67-e28787857fbf" (UID: "63d6dfdb-38b8-46f2-8d67-e28787857fbf"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:57:24.127348 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:57:24.127282    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/63d6dfdb-38b8-46f2-8d67-e28787857fbf-tmp-dir" pod "63d6dfdb-38b8-46f2-8d67-e28787857fbf" (UID: "63d6dfdb-38b8-46f2-8d67-e28787857fbf"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:57:24.127400 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:57:24.127348    5510 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/63d6dfdb-38b8-46f2-8d67-e28787857fbf-model-cache\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:57:24.127400 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:57:24.127369    5510 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/63d6dfdb-38b8-46f2-8d67-e28787857fbf-home\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:57:24.128997 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:57:24.128967    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/63d6dfdb-38b8-46f2-8d67-e28787857fbf-tls-certs" pod "63d6dfdb-38b8-46f2-8d67-e28787857fbf" (UID: "63d6dfdb-38b8-46f2-8d67-e28787857fbf"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:57:24.129285 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:57:24.129258    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/63d6dfdb-38b8-46f2-8d67-e28787857fbf-dshm" pod "63d6dfdb-38b8-46f2-8d67-e28787857fbf" (UID: "63d6dfdb-38b8-46f2-8d67-e28787857fbf"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:57:24.129285 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:57:24.129275    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/63d6dfdb-38b8-46f2-8d67-e28787857fbf-kube-api-access-fj8c5" pod "63d6dfdb-38b8-46f2-8d67-e28787857fbf" (UID: "63d6dfdb-38b8-46f2-8d67-e28787857fbf"). InnerVolumeSpecName "kube-api-access-fj8c5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:57:24.164134 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:57:24.164108    5510 generic.go:356] "Generic (PLEG): container finished" podID="63d6dfdb-38b8-46f2-8d67-e28787857fbf" containerID="18ce25271dba69f2e8782e823d22bc8e97f050dfbb26e76e911da547e5837960" exitCode=0
Oct 30 22:57:24.164257 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:57:24.164151    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-713f1911-kserve-b89899d97-rfjpc"
Oct 30 22:57:24.164257 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:57:24.164188    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-713f1911-kserve-b89899d97-rfjpc" event={"ID":"63d6dfdb-38b8-46f2-8d67-e28787857fbf","Type":"ContainerDied","Data":"18ce25271dba69f2e8782e823d22bc8e97f050dfbb26e76e911da547e5837960"}
Oct 30 22:57:24.164257 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:57:24.164233    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-713f1911-kserve-b89899d97-rfjpc" event={"ID":"63d6dfdb-38b8-46f2-8d67-e28787857fbf","Type":"ContainerDied","Data":"6786877d88fccd105073f94c6d9497822dddd41920d91a60e3252c7c005208c3"}
Oct 30 22:57:24.164257 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:57:24.164256    5510 scope.go:122] "RemoveContainer" containerID="18ce25271dba69f2e8782e823d22bc8e97f050dfbb26e76e911da547e5837960"
Oct 30 22:57:24.173861 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:57:24.173844    5510 scope.go:122] "RemoveContainer" containerID="18ce25271dba69f2e8782e823d22bc8e97f050dfbb26e76e911da547e5837960"
Oct 30 22:57:24.174151 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:57:24.174129    5510 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"18ce25271dba69f2e8782e823d22bc8e97f050dfbb26e76e911da547e5837960\": container with ID starting with 18ce25271dba69f2e8782e823d22bc8e97f050dfbb26e76e911da547e5837960 not found: ID does not exist" containerID="18ce25271dba69f2e8782e823d22bc8e97f050dfbb26e76e911da547e5837960"
Oct 30 22:57:24.174255 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:57:24.174155    5510 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"18ce25271dba69f2e8782e823d22bc8e97f050dfbb26e76e911da547e5837960"} err="failed to get container status \"18ce25271dba69f2e8782e823d22bc8e97f050dfbb26e76e911da547e5837960\": rpc error: code = NotFound desc = could not find container \"18ce25271dba69f2e8782e823d22bc8e97f050dfbb26e76e911da547e5837960\": container with ID starting with 18ce25271dba69f2e8782e823d22bc8e97f050dfbb26e76e911da547e5837960 not found: ID does not exist"
Oct 30 22:57:24.228161 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:57:24.228124    5510 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/63d6dfdb-38b8-46f2-8d67-e28787857fbf-tmp-dir\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:57:24.228161 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:57:24.228155    5510 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fj8c5\" (UniqueName: \"kubernetes.io/projected/63d6dfdb-38b8-46f2-8d67-e28787857fbf-kube-api-access-fj8c5\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:57:24.228373 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:57:24.228170    5510 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/63d6dfdb-38b8-46f2-8d67-e28787857fbf-dshm\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:57:24.228373 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:57:24.228183    5510 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/63d6dfdb-38b8-46f2-8d67-e28787857fbf-tls-certs\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:57:24.319054 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:57:24.319014    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-713f1911-kserve-b89899d97-rfjpc"]
Oct 30 22:57:24.323950 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:57:24.323915    5510 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-713f1911-kserve-b89899d97-rfjpc"]
Oct 30 22:57:25.552714 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:57:25.552675    5510 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="63d6dfdb-38b8-46f2-8d67-e28787857fbf" path="/var/lib/kubelet/pods/63d6dfdb-38b8-46f2-8d67-e28787857fbf/volumes"
Oct 30 22:57:37.002940 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:57:37.002896    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:57:38.420206 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:57:38.420169    5510 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6gjbf" containerName="extractor"
Oct 30 22:57:41.244714 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:57:41.244678    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:57:41.439123 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:57:41.439079    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:57:58.940025 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:57:58.939989    5510 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-57df996b79-zq2fb"]
Oct 30 22:57:58.940517 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:57:58.940447    5510 cpu_manager.go:414] "containerMap: removing container" podUID="dd1877bb-c1ff-46ab-99ca-33047d8fd7e8" containerName="main"
Oct 30 22:57:58.940517 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:57:58.940462    5510 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="dd1877bb-c1ff-46ab-99ca-33047d8fd7e8" containerName="main"
Oct 30 22:57:58.940517 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:57:58.940497    5510 cpu_manager.go:414] "containerMap: removing container" podUID="63d6dfdb-38b8-46f2-8d67-e28787857fbf" containerName="main"
Oct 30 22:57:58.940517 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:57:58.940503    5510 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="63d6dfdb-38b8-46f2-8d67-e28787857fbf" containerName="main"
Oct 30 22:57:58.940688 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:57:58.940573    5510 memory_manager.go:365] "RemoveStaleState removing state" podUID="dd1877bb-c1ff-46ab-99ca-33047d8fd7e8" containerName="main"
Oct 30 22:57:58.940688 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:57:58.940589    5510 memory_manager.go:365] "RemoveStaleState removing state" podUID="63d6dfdb-38b8-46f2-8d67-e28787857fbf" containerName="main"
Oct 30 22:57:58.943477 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:57:58.943450    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-57df996b79-zq2fb"
Oct 30 22:57:59.063435 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:57:59.063386    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-57df996b79-zq2fb"]
Oct 30 22:57:59.068648 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:57:59.068603    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8twmw\" (UniqueName: \"kubernetes.io/projected/8f990085-4005-4a21-b7b4-91ce25c74761-kube-api-access-8twmw\") pod \"maas-api-57df996b79-zq2fb\" (UID: \"8f990085-4005-4a21-b7b4-91ce25c74761\") " pod="odh-ai-gateway-infra/maas-api-57df996b79-zq2fb"
Oct 30 22:57:59.069020 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:57:59.068981    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8f990085-4005-4a21-b7b4-91ce25c74761-maas-api-tls\") pod \"maas-api-57df996b79-zq2fb\" (UID: \"8f990085-4005-4a21-b7b4-91ce25c74761\") " pod="odh-ai-gateway-infra/maas-api-57df996b79-zq2fb"
Oct 30 22:57:59.069189 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:57:59.069172    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8f990085-4005-4a21-b7b4-91ce25c74761-openshift-service-ca\") pod \"maas-api-57df996b79-zq2fb\" (UID: \"8f990085-4005-4a21-b7b4-91ce25c74761\") " pod="odh-ai-gateway-infra/maas-api-57df996b79-zq2fb"
Oct 30 22:57:59.069340 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:57:59.069327    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8f990085-4005-4a21-b7b4-91ce25c74761-metrics-certs\") pod \"maas-api-57df996b79-zq2fb\" (UID: \"8f990085-4005-4a21-b7b4-91ce25c74761\") " pod="odh-ai-gateway-infra/maas-api-57df996b79-zq2fb"
Oct 30 22:57:59.170845 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:57:59.170806    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8twmw\" (UniqueName: \"kubernetes.io/projected/8f990085-4005-4a21-b7b4-91ce25c74761-kube-api-access-8twmw\") pod \"maas-api-57df996b79-zq2fb\" (UID: \"8f990085-4005-4a21-b7b4-91ce25c74761\") " pod="odh-ai-gateway-infra/maas-api-57df996b79-zq2fb"
Oct 30 22:57:59.171045 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:57:59.170853    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8f990085-4005-4a21-b7b4-91ce25c74761-maas-api-tls\") pod \"maas-api-57df996b79-zq2fb\" (UID: \"8f990085-4005-4a21-b7b4-91ce25c74761\") " pod="odh-ai-gateway-infra/maas-api-57df996b79-zq2fb"
Oct 30 22:57:59.171045 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:57:59.170969    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8f990085-4005-4a21-b7b4-91ce25c74761-openshift-service-ca\") pod \"maas-api-57df996b79-zq2fb\" (UID: \"8f990085-4005-4a21-b7b4-91ce25c74761\") " pod="odh-ai-gateway-infra/maas-api-57df996b79-zq2fb"
Oct 30 22:57:59.171045 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:57:59.171012    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8f990085-4005-4a21-b7b4-91ce25c74761-metrics-certs\") pod \"maas-api-57df996b79-zq2fb\" (UID: \"8f990085-4005-4a21-b7b4-91ce25c74761\") " pod="odh-ai-gateway-infra/maas-api-57df996b79-zq2fb"
Oct 30 22:57:59.171625 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:57:59.171600    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8f990085-4005-4a21-b7b4-91ce25c74761-openshift-service-ca\") pod \"maas-api-57df996b79-zq2fb\" (UID: \"8f990085-4005-4a21-b7b4-91ce25c74761\") " pod="odh-ai-gateway-infra/maas-api-57df996b79-zq2fb"
Oct 30 22:57:59.173365 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:57:59.173345    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8f990085-4005-4a21-b7b4-91ce25c74761-metrics-certs\") pod \"maas-api-57df996b79-zq2fb\" (UID: \"8f990085-4005-4a21-b7b4-91ce25c74761\") " pod="odh-ai-gateway-infra/maas-api-57df996b79-zq2fb"
Oct 30 22:57:59.173511 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:57:59.173489    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8f990085-4005-4a21-b7b4-91ce25c74761-maas-api-tls\") pod \"maas-api-57df996b79-zq2fb\" (UID: \"8f990085-4005-4a21-b7b4-91ce25c74761\") " pod="odh-ai-gateway-infra/maas-api-57df996b79-zq2fb"
Oct 30 22:57:59.229869 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:57:59.229799    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8twmw\" (UniqueName: \"kubernetes.io/projected/8f990085-4005-4a21-b7b4-91ce25c74761-kube-api-access-8twmw\") pod \"maas-api-57df996b79-zq2fb\" (UID: \"8f990085-4005-4a21-b7b4-91ce25c74761\") " pod="odh-ai-gateway-infra/maas-api-57df996b79-zq2fb"
Oct 30 22:57:59.256650 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:57:59.256624    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-57df996b79-zq2fb"
Oct 30 22:57:59.632472 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:57:59.632438    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-57df996b79-zq2fb"]
Oct 30 22:57:59.639872 ip-10-0-136-210 kubenswrapper[5510]: W1030 22:57:59.639841    5510 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f990085_4005_4a21_b7b4_91ce25c74761.slice/crio-84663b8e5c9e3f08bfa45a35bd74496d1924110bcd81924d307738616dd27931 WatchSource:0}: Error finding container 84663b8e5c9e3f08bfa45a35bd74496d1924110bcd81924d307738616dd27931: Status 404 returned error can't find the container with id 84663b8e5c9e3f08bfa45a35bd74496d1924110bcd81924d307738616dd27931
Oct 30 22:57:59.641200 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:57:59.641177    5510 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 22:58:00.396330 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:00.396295    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-57df996b79-zq2fb" event={"ID":"8f990085-4005-4a21-b7b4-91ce25c74761","Type":"ContainerStarted","Data":"84663b8e5c9e3f08bfa45a35bd74496d1924110bcd81924d307738616dd27931"}
Oct 30 22:58:01.403995 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:01.403960    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-57df996b79-zq2fb" event={"ID":"8f990085-4005-4a21-b7b4-91ce25c74761","Type":"ContainerStarted","Data":"d1603aef7ebd655606df96a498c6efca52857a09eb725a777e31e2f8fb3d54be"}
Oct 30 22:58:01.404486 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:01.404067    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-57df996b79-zq2fb"
Oct 30 22:58:01.518074 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:01.518016    5510 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-57df996b79-zq2fb" podStartSLOduration=2.674668398 podStartE2EDuration="3.518001478s" totalImagesPullingTime="843.33308ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:57:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:58:01.516437394 +0000 UTC m=+4062.077247532" watchObservedRunningTime="2026-10-30 22:58:01.518001478 +0000 UTC m=+4062.078811483"
Oct 30 22:58:05.488612 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:05.488575    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:58:05.947418 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:05.947379    5510 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-f23d4bb3-openshift-default-865c59c95c-fzcbj"]
Oct 30 22:58:05.951285 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:05.951264    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-f23d4bb3-openshift-default-865c59c95c-fzcbj"
Oct 30 22:58:06.012606 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:06.012576    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-f23d4bb3-openshift-default-dockercfg-24mpf\""
Oct 30 22:58:06.039174 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:06.039140    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2f4c97c6-6598-4189-b7ca-0244b2799bf5-istio-envoy\") pod \"e2e-ait-f23d4bb3-openshift-default-865c59c95c-fzcbj\" (UID: \"2f4c97c6-6598-4189-b7ca-0244b2799bf5\") " pod="openshift-ingress/e2e-ait-f23d4bb3-openshift-default-865c59c95c-fzcbj"
Oct 30 22:58:06.039360 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:06.039178    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2f4c97c6-6598-4189-b7ca-0244b2799bf5-workload-certs\") pod \"e2e-ait-f23d4bb3-openshift-default-865c59c95c-fzcbj\" (UID: \"2f4c97c6-6598-4189-b7ca-0244b2799bf5\") " pod="openshift-ingress/e2e-ait-f23d4bb3-openshift-default-865c59c95c-fzcbj"
Oct 30 22:58:06.039360 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:06.039211    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c6bn2\" (UniqueName: \"kubernetes.io/projected/2f4c97c6-6598-4189-b7ca-0244b2799bf5-kube-api-access-c6bn2\") pod \"e2e-ait-f23d4bb3-openshift-default-865c59c95c-fzcbj\" (UID: \"2f4c97c6-6598-4189-b7ca-0244b2799bf5\") " pod="openshift-ingress/e2e-ait-f23d4bb3-openshift-default-865c59c95c-fzcbj"
Oct 30 22:58:06.039468 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:06.039363    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f4c97c6-6598-4189-b7ca-0244b2799bf5-credential-socket\") pod \"e2e-ait-f23d4bb3-openshift-default-865c59c95c-fzcbj\" (UID: \"2f4c97c6-6598-4189-b7ca-0244b2799bf5\") " pod="openshift-ingress/e2e-ait-f23d4bb3-openshift-default-865c59c95c-fzcbj"
Oct 30 22:58:06.039468 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:06.039399    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2f4c97c6-6598-4189-b7ca-0244b2799bf5-istio-token\") pod \"e2e-ait-f23d4bb3-openshift-default-865c59c95c-fzcbj\" (UID: \"2f4c97c6-6598-4189-b7ca-0244b2799bf5\") " pod="openshift-ingress/e2e-ait-f23d4bb3-openshift-default-865c59c95c-fzcbj"
Oct 30 22:58:06.039599 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:06.039529    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2f4c97c6-6598-4189-b7ca-0244b2799bf5-istio-data\") pod \"e2e-ait-f23d4bb3-openshift-default-865c59c95c-fzcbj\" (UID: \"2f4c97c6-6598-4189-b7ca-0244b2799bf5\") " pod="openshift-ingress/e2e-ait-f23d4bb3-openshift-default-865c59c95c-fzcbj"
Oct 30 22:58:06.039599 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:06.039581    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2f4c97c6-6598-4189-b7ca-0244b2799bf5-istiod-ca-cert\") pod \"e2e-ait-f23d4bb3-openshift-default-865c59c95c-fzcbj\" (UID: \"2f4c97c6-6598-4189-b7ca-0244b2799bf5\") " pod="openshift-ingress/e2e-ait-f23d4bb3-openshift-default-865c59c95c-fzcbj"
Oct 30 22:58:06.039713 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:06.039621    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f4c97c6-6598-4189-b7ca-0244b2799bf5-workload-socket\") pod \"e2e-ait-f23d4bb3-openshift-default-865c59c95c-fzcbj\" (UID: \"2f4c97c6-6598-4189-b7ca-0244b2799bf5\") " pod="openshift-ingress/e2e-ait-f23d4bb3-openshift-default-865c59c95c-fzcbj"
Oct 30 22:58:06.039713 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:06.039645    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2f4c97c6-6598-4189-b7ca-0244b2799bf5-istio-podinfo\") pod \"e2e-ait-f23d4bb3-openshift-default-865c59c95c-fzcbj\" (UID: \"2f4c97c6-6598-4189-b7ca-0244b2799bf5\") " pod="openshift-ingress/e2e-ait-f23d4bb3-openshift-default-865c59c95c-fzcbj"
Oct 30 22:58:06.056280 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:06.056245    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-f23d4bb3-openshift-default-865c59c95c-fzcbj"]
Oct 30 22:58:06.140257 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:06.140222    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2f4c97c6-6598-4189-b7ca-0244b2799bf5-istio-data\") pod \"e2e-ait-f23d4bb3-openshift-default-865c59c95c-fzcbj\" (UID: \"2f4c97c6-6598-4189-b7ca-0244b2799bf5\") " pod="openshift-ingress/e2e-ait-f23d4bb3-openshift-default-865c59c95c-fzcbj"
Oct 30 22:58:06.140465 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:06.140264    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2f4c97c6-6598-4189-b7ca-0244b2799bf5-istiod-ca-cert\") pod \"e2e-ait-f23d4bb3-openshift-default-865c59c95c-fzcbj\" (UID: \"2f4c97c6-6598-4189-b7ca-0244b2799bf5\") " pod="openshift-ingress/e2e-ait-f23d4bb3-openshift-default-865c59c95c-fzcbj"
Oct 30 22:58:06.140465 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:06.140397    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f4c97c6-6598-4189-b7ca-0244b2799bf5-workload-socket\") pod \"e2e-ait-f23d4bb3-openshift-default-865c59c95c-fzcbj\" (UID: \"2f4c97c6-6598-4189-b7ca-0244b2799bf5\") " pod="openshift-ingress/e2e-ait-f23d4bb3-openshift-default-865c59c95c-fzcbj"
Oct 30 22:58:06.140465 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:06.140447    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2f4c97c6-6598-4189-b7ca-0244b2799bf5-istio-podinfo\") pod \"e2e-ait-f23d4bb3-openshift-default-865c59c95c-fzcbj\" (UID: \"2f4c97c6-6598-4189-b7ca-0244b2799bf5\") " pod="openshift-ingress/e2e-ait-f23d4bb3-openshift-default-865c59c95c-fzcbj"
Oct 30 22:58:06.140638 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:06.140498    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2f4c97c6-6598-4189-b7ca-0244b2799bf5-istio-envoy\") pod \"e2e-ait-f23d4bb3-openshift-default-865c59c95c-fzcbj\" (UID: \"2f4c97c6-6598-4189-b7ca-0244b2799bf5\") " pod="openshift-ingress/e2e-ait-f23d4bb3-openshift-default-865c59c95c-fzcbj"
Oct 30 22:58:06.140638 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:06.140530    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2f4c97c6-6598-4189-b7ca-0244b2799bf5-workload-certs\") pod \"e2e-ait-f23d4bb3-openshift-default-865c59c95c-fzcbj\" (UID: \"2f4c97c6-6598-4189-b7ca-0244b2799bf5\") " pod="openshift-ingress/e2e-ait-f23d4bb3-openshift-default-865c59c95c-fzcbj"
Oct 30 22:58:06.140638 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:06.140605    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c6bn2\" (UniqueName: \"kubernetes.io/projected/2f4c97c6-6598-4189-b7ca-0244b2799bf5-kube-api-access-c6bn2\") pod \"e2e-ait-f23d4bb3-openshift-default-865c59c95c-fzcbj\" (UID: \"2f4c97c6-6598-4189-b7ca-0244b2799bf5\") " pod="openshift-ingress/e2e-ait-f23d4bb3-openshift-default-865c59c95c-fzcbj"
Oct 30 22:58:06.140825 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:06.140642    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2f4c97c6-6598-4189-b7ca-0244b2799bf5-istio-data\") pod \"e2e-ait-f23d4bb3-openshift-default-865c59c95c-fzcbj\" (UID: \"2f4c97c6-6598-4189-b7ca-0244b2799bf5\") " pod="openshift-ingress/e2e-ait-f23d4bb3-openshift-default-865c59c95c-fzcbj"
Oct 30 22:58:06.140825 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:06.140734    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f4c97c6-6598-4189-b7ca-0244b2799bf5-credential-socket\") pod \"e2e-ait-f23d4bb3-openshift-default-865c59c95c-fzcbj\" (UID: \"2f4c97c6-6598-4189-b7ca-0244b2799bf5\") " pod="openshift-ingress/e2e-ait-f23d4bb3-openshift-default-865c59c95c-fzcbj"
Oct 30 22:58:06.140825 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:06.140763    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f4c97c6-6598-4189-b7ca-0244b2799bf5-workload-socket\") pod \"e2e-ait-f23d4bb3-openshift-default-865c59c95c-fzcbj\" (UID: \"2f4c97c6-6598-4189-b7ca-0244b2799bf5\") " pod="openshift-ingress/e2e-ait-f23d4bb3-openshift-default-865c59c95c-fzcbj"
Oct 30 22:58:06.140825 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:06.140772    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2f4c97c6-6598-4189-b7ca-0244b2799bf5-istio-token\") pod \"e2e-ait-f23d4bb3-openshift-default-865c59c95c-fzcbj\" (UID: \"2f4c97c6-6598-4189-b7ca-0244b2799bf5\") " pod="openshift-ingress/e2e-ait-f23d4bb3-openshift-default-865c59c95c-fzcbj"
Oct 30 22:58:06.141034 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:06.140936    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2f4c97c6-6598-4189-b7ca-0244b2799bf5-workload-certs\") pod \"e2e-ait-f23d4bb3-openshift-default-865c59c95c-fzcbj\" (UID: \"2f4c97c6-6598-4189-b7ca-0244b2799bf5\") " pod="openshift-ingress/e2e-ait-f23d4bb3-openshift-default-865c59c95c-fzcbj"
Oct 30 22:58:06.141075 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:06.141030    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f4c97c6-6598-4189-b7ca-0244b2799bf5-credential-socket\") pod \"e2e-ait-f23d4bb3-openshift-default-865c59c95c-fzcbj\" (UID: \"2f4c97c6-6598-4189-b7ca-0244b2799bf5\") " pod="openshift-ingress/e2e-ait-f23d4bb3-openshift-default-865c59c95c-fzcbj"
Oct 30 22:58:06.141119 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:06.141080    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2f4c97c6-6598-4189-b7ca-0244b2799bf5-istiod-ca-cert\") pod \"e2e-ait-f23d4bb3-openshift-default-865c59c95c-fzcbj\" (UID: \"2f4c97c6-6598-4189-b7ca-0244b2799bf5\") " pod="openshift-ingress/e2e-ait-f23d4bb3-openshift-default-865c59c95c-fzcbj"
Oct 30 22:58:06.142773 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:06.142743    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2f4c97c6-6598-4189-b7ca-0244b2799bf5-istio-envoy\") pod \"e2e-ait-f23d4bb3-openshift-default-865c59c95c-fzcbj\" (UID: \"2f4c97c6-6598-4189-b7ca-0244b2799bf5\") " pod="openshift-ingress/e2e-ait-f23d4bb3-openshift-default-865c59c95c-fzcbj"
Oct 30 22:58:06.143071 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:06.143052    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2f4c97c6-6598-4189-b7ca-0244b2799bf5-istio-podinfo\") pod \"e2e-ait-f23d4bb3-openshift-default-865c59c95c-fzcbj\" (UID: \"2f4c97c6-6598-4189-b7ca-0244b2799bf5\") " pod="openshift-ingress/e2e-ait-f23d4bb3-openshift-default-865c59c95c-fzcbj"
Oct 30 22:58:06.206597 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:06.206518    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2f4c97c6-6598-4189-b7ca-0244b2799bf5-istio-token\") pod \"e2e-ait-f23d4bb3-openshift-default-865c59c95c-fzcbj\" (UID: \"2f4c97c6-6598-4189-b7ca-0244b2799bf5\") " pod="openshift-ingress/e2e-ait-f23d4bb3-openshift-default-865c59c95c-fzcbj"
Oct 30 22:58:06.234511 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:06.234474    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:58:06.258187 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:06.258150    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c6bn2\" (UniqueName: \"kubernetes.io/projected/2f4c97c6-6598-4189-b7ca-0244b2799bf5-kube-api-access-c6bn2\") pod \"e2e-ait-f23d4bb3-openshift-default-865c59c95c-fzcbj\" (UID: \"2f4c97c6-6598-4189-b7ca-0244b2799bf5\") " pod="openshift-ingress/e2e-ait-f23d4bb3-openshift-default-865c59c95c-fzcbj"
Oct 30 22:58:06.263873 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:06.263849    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-f23d4bb3-openshift-default-865c59c95c-fzcbj"
Oct 30 22:58:06.524000 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:06.523959    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:58:06.701384 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:06.701348    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-f23d4bb3-openshift-default-865c59c95c-fzcbj"]
Oct 30 22:58:06.708761 ip-10-0-136-210 kubenswrapper[5510]: W1030 22:58:06.708732    5510 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f4c97c6_6598_4189_b7ca_0244b2799bf5.slice/crio-b24b5101d8335bc28e94bc15f50ebd7f542ce7d62eaca8358fd10f6f130dc986 WatchSource:0}: Error finding container b24b5101d8335bc28e94bc15f50ebd7f542ce7d62eaca8358fd10f6f130dc986: Status 404 returned error can't find the container with id b24b5101d8335bc28e94bc15f50ebd7f542ce7d62eaca8358fd10f6f130dc986
Oct 30 22:58:06.710945 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:06.710911    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:58:06.711044 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:06.710980    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:58:06.711044 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:06.711011    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:58:07.414741 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:07.414703    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-57df996b79-zq2fb"
Oct 30 22:58:07.435146 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:07.435107    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f23d4bb3-openshift-default-865c59c95c-fzcbj" event={"ID":"2f4c97c6-6598-4189-b7ca-0244b2799bf5","Type":"ContainerStarted","Data":"a2e32a6311b2d9833f56ece0bdbcc31bd2388c1228c7599aab8efc76e8d7a98b"}
Oct 30 22:58:07.435333 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:07.435161    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f23d4bb3-openshift-default-865c59c95c-fzcbj" event={"ID":"2f4c97c6-6598-4189-b7ca-0244b2799bf5","Type":"ContainerStarted","Data":"b24b5101d8335bc28e94bc15f50ebd7f542ce7d62eaca8358fd10f6f130dc986"}
Oct 30 22:58:07.595439 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:07.595402    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-7f4ccfcf79-tlkq8"]
Oct 30 22:58:07.595909 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:07.595589    5510 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-7f4ccfcf79-tlkq8" podUID="91c8233a-1602-49b1-8cd2-51a4c47de80a" containerName="maas-api" containerID="cri-o://a17d77bf9f4e1d003f9c518d0e291e48e6f1d7097e4442d578626ceee3ce0afc" gracePeriod=30
Oct 30 22:58:07.659573 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:07.659522    5510 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-f23d4bb3-openshift-default-865c59c95c-fzcbj" podStartSLOduration=2.659509819 podStartE2EDuration="2.659509819s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:58:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:58:07.658847882 +0000 UTC m=+4068.219657906" watchObservedRunningTime="2026-10-30 22:58:07.659509819 +0000 UTC m=+4068.220319820"
Oct 30 22:58:08.264698 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:08.264635    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-f23d4bb3-openshift-default-865c59c95c-fzcbj"
Oct 30 22:58:08.269463 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:08.269437    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-f23d4bb3-openshift-default-865c59c95c-fzcbj"
Oct 30 22:58:08.420494 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:58:08.420449    5510 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6gjbf" containerName="extractor"
Oct 30 22:58:08.440429 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:08.440397    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-f23d4bb3-openshift-default-865c59c95c-fzcbj"
Oct 30 22:58:08.441268 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:08.441245    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-f23d4bb3-openshift-default-865c59c95c-fzcbj"
Oct 30 22:58:08.856646 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:08.856622    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7f4ccfcf79-tlkq8"
Oct 30 22:58:08.978412 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:08.978329    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/91c8233a-1602-49b1-8cd2-51a4c47de80a-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/91c8233a-1602-49b1-8cd2-51a4c47de80a-maas-api-tls\") pod \"91c8233a-1602-49b1-8cd2-51a4c47de80a\" (UID: \"91c8233a-1602-49b1-8cd2-51a4c47de80a\") "
Oct 30 22:58:08.978566 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:08.978448    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/91c8233a-1602-49b1-8cd2-51a4c47de80a-metrics-certs\" (UniqueName: \"kubernetes.io/secret/91c8233a-1602-49b1-8cd2-51a4c47de80a-metrics-certs\") pod \"91c8233a-1602-49b1-8cd2-51a4c47de80a\" (UID: \"91c8233a-1602-49b1-8cd2-51a4c47de80a\") "
Oct 30 22:58:08.978566 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:08.978524    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/91c8233a-1602-49b1-8cd2-51a4c47de80a-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/91c8233a-1602-49b1-8cd2-51a4c47de80a-openshift-service-ca\") pod \"91c8233a-1602-49b1-8cd2-51a4c47de80a\" (UID: \"91c8233a-1602-49b1-8cd2-51a4c47de80a\") "
Oct 30 22:58:08.978693 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:08.978647    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/91c8233a-1602-49b1-8cd2-51a4c47de80a-kube-api-access-b7qvz\" (UniqueName: \"kubernetes.io/projected/91c8233a-1602-49b1-8cd2-51a4c47de80a-kube-api-access-b7qvz\") pod \"91c8233a-1602-49b1-8cd2-51a4c47de80a\" (UID: \"91c8233a-1602-49b1-8cd2-51a4c47de80a\") "
Oct 30 22:58:08.978905 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:08.978877    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/91c8233a-1602-49b1-8cd2-51a4c47de80a-openshift-service-ca" pod "91c8233a-1602-49b1-8cd2-51a4c47de80a" (UID: "91c8233a-1602-49b1-8cd2-51a4c47de80a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:58:08.979085 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:08.979067    5510 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/91c8233a-1602-49b1-8cd2-51a4c47de80a-openshift-service-ca\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:58:08.980641 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:08.980615    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/91c8233a-1602-49b1-8cd2-51a4c47de80a-metrics-certs" pod "91c8233a-1602-49b1-8cd2-51a4c47de80a" (UID: "91c8233a-1602-49b1-8cd2-51a4c47de80a"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:58:08.981022 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:08.981002    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/91c8233a-1602-49b1-8cd2-51a4c47de80a-maas-api-tls" pod "91c8233a-1602-49b1-8cd2-51a4c47de80a" (UID: "91c8233a-1602-49b1-8cd2-51a4c47de80a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:58:08.981127 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:08.981102    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/91c8233a-1602-49b1-8cd2-51a4c47de80a-kube-api-access-b7qvz" pod "91c8233a-1602-49b1-8cd2-51a4c47de80a" (UID: "91c8233a-1602-49b1-8cd2-51a4c47de80a"). InnerVolumeSpecName "kube-api-access-b7qvz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:58:09.080559 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:09.080512    5510 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/91c8233a-1602-49b1-8cd2-51a4c47de80a-maas-api-tls\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:58:09.080559 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:09.080554    5510 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/91c8233a-1602-49b1-8cd2-51a4c47de80a-metrics-certs\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:58:09.081467 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:09.080570    5510 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b7qvz\" (UniqueName: \"kubernetes.io/projected/91c8233a-1602-49b1-8cd2-51a4c47de80a-kube-api-access-b7qvz\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:58:09.446068 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:09.446028    5510 generic.go:356] "Generic (PLEG): container finished" podID="91c8233a-1602-49b1-8cd2-51a4c47de80a" containerID="a17d77bf9f4e1d003f9c518d0e291e48e6f1d7097e4442d578626ceee3ce0afc" exitCode=0
Oct 30 22:58:09.446287 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:09.446082    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7f4ccfcf79-tlkq8"
Oct 30 22:58:09.446287 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:09.446118    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7f4ccfcf79-tlkq8" event={"ID":"91c8233a-1602-49b1-8cd2-51a4c47de80a","Type":"ContainerDied","Data":"a17d77bf9f4e1d003f9c518d0e291e48e6f1d7097e4442d578626ceee3ce0afc"}
Oct 30 22:58:09.446287 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:09.446166    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7f4ccfcf79-tlkq8" event={"ID":"91c8233a-1602-49b1-8cd2-51a4c47de80a","Type":"ContainerDied","Data":"735306c1e68276d95f0b07a8baf5e6a2fc0147264f95d9295d0f6d335a0a9960"}
Oct 30 22:58:09.446287 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:09.446197    5510 scope.go:122] "RemoveContainer" containerID="a17d77bf9f4e1d003f9c518d0e291e48e6f1d7097e4442d578626ceee3ce0afc"
Oct 30 22:58:09.455749 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:09.455729    5510 scope.go:122] "RemoveContainer" containerID="a17d77bf9f4e1d003f9c518d0e291e48e6f1d7097e4442d578626ceee3ce0afc"
Oct 30 22:58:09.456071 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:58:09.456046    5510 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a17d77bf9f4e1d003f9c518d0e291e48e6f1d7097e4442d578626ceee3ce0afc\": container with ID starting with a17d77bf9f4e1d003f9c518d0e291e48e6f1d7097e4442d578626ceee3ce0afc not found: ID does not exist" containerID="a17d77bf9f4e1d003f9c518d0e291e48e6f1d7097e4442d578626ceee3ce0afc"
Oct 30 22:58:09.456139 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:09.456084    5510 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a17d77bf9f4e1d003f9c518d0e291e48e6f1d7097e4442d578626ceee3ce0afc"} err="failed to get container status \"a17d77bf9f4e1d003f9c518d0e291e48e6f1d7097e4442d578626ceee3ce0afc\": rpc error: code = NotFound desc = could not find container \"a17d77bf9f4e1d003f9c518d0e291e48e6f1d7097e4442d578626ceee3ce0afc\": container with ID starting with a17d77bf9f4e1d003f9c518d0e291e48e6f1d7097e4442d578626ceee3ce0afc not found: ID does not exist"
Oct 30 22:58:09.605944 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:09.605910    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-7f4ccfcf79-tlkq8"]
Oct 30 22:58:09.612753 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:09.612729    5510 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-7f4ccfcf79-tlkq8"]
Oct 30 22:58:09.856230 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:09.856193    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:58:10.084338 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:10.084297    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:58:11.552768 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:11.552740    5510 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="91c8233a-1602-49b1-8cd2-51a4c47de80a" path="/var/lib/kubelet/pods/91c8233a-1602-49b1-8cd2-51a4c47de80a/volumes"
Oct 30 22:58:13.048334 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:13.048282    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:58:13.287358 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:13.287320    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:58:13.435895 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:13.435804    5510 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-f23d4bb3-6999885447-qkfg9"]
Oct 30 22:58:13.436336 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:13.436317    5510 cpu_manager.go:414] "containerMap: removing container" podUID="91c8233a-1602-49b1-8cd2-51a4c47de80a" containerName="maas-api"
Oct 30 22:58:13.436394 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:13.436339    5510 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="91c8233a-1602-49b1-8cd2-51a4c47de80a" containerName="maas-api"
Oct 30 22:58:13.436456 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:13.436444    5510 memory_manager.go:365] "RemoveStaleState removing state" podUID="91c8233a-1602-49b1-8cd2-51a4c47de80a" containerName="maas-api"
Oct 30 22:58:13.440964 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:13.440943    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-f23d4bb3-6999885447-qkfg9"
Oct 30 22:58:13.496709 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:13.496676    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-f23d4bb3-6999885447-qkfg9"]
Oct 30 22:58:13.536908 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:13.536851    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-f23d4bb3-dockercfg-nf4sn\""
Oct 30 22:58:13.537081 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:13.536976    5510 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-f23d4bb3\""
Oct 30 22:58:13.585578 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:13.585542    5510 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-f23d4bb3-78667f84d7-9hl2j"]
Oct 30 22:58:13.589308 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:13.589288    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-f23d4bb3-78667f84d7-9hl2j"
Oct 30 22:58:13.625001 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:13.624969    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-f23d4bb3-78667f84d7-9hl2j"]
Oct 30 22:58:13.629323 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:13.629288    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hplcp\" (UniqueName: \"kubernetes.io/projected/bdb07ae7-7207-4fbe-94b9-5f773d660ac5-kube-api-access-hplcp\") pod \"payload-pre-processing-e2e-ait-f23d4bb3-6999885447-qkfg9\" (UID: \"bdb07ae7-7207-4fbe-94b9-5f773d660ac5\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-f23d4bb3-6999885447-qkfg9"
Oct 30 22:58:13.629510 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:13.629450    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bdb07ae7-7207-4fbe-94b9-5f773d660ac5-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-f23d4bb3-6999885447-qkfg9\" (UID: \"bdb07ae7-7207-4fbe-94b9-5f773d660ac5\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-f23d4bb3-6999885447-qkfg9"
Oct 30 22:58:13.730702 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:13.730602    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d7629350-fdda-4ebb-b98f-618a1d30f88d-plugins-config-volume\") pod \"payload-processing-e2e-ait-f23d4bb3-78667f84d7-9hl2j\" (UID: \"d7629350-fdda-4ebb-b98f-618a1d30f88d\") " pod="openshift-ingress/payload-processing-e2e-ait-f23d4bb3-78667f84d7-9hl2j"
Oct 30 22:58:13.730702 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:13.730646    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hplcp\" (UniqueName: \"kubernetes.io/projected/bdb07ae7-7207-4fbe-94b9-5f773d660ac5-kube-api-access-hplcp\") pod \"payload-pre-processing-e2e-ait-f23d4bb3-6999885447-qkfg9\" (UID: \"bdb07ae7-7207-4fbe-94b9-5f773d660ac5\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-f23d4bb3-6999885447-qkfg9"
Oct 30 22:58:13.730924 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:13.730834    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dzz2s\" (UniqueName: \"kubernetes.io/projected/d7629350-fdda-4ebb-b98f-618a1d30f88d-kube-api-access-dzz2s\") pod \"payload-processing-e2e-ait-f23d4bb3-78667f84d7-9hl2j\" (UID: \"d7629350-fdda-4ebb-b98f-618a1d30f88d\") " pod="openshift-ingress/payload-processing-e2e-ait-f23d4bb3-78667f84d7-9hl2j"
Oct 30 22:58:13.731010 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:13.730994    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bdb07ae7-7207-4fbe-94b9-5f773d660ac5-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-f23d4bb3-6999885447-qkfg9\" (UID: \"bdb07ae7-7207-4fbe-94b9-5f773d660ac5\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-f23d4bb3-6999885447-qkfg9"
Oct 30 22:58:13.731649 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:13.731628    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bdb07ae7-7207-4fbe-94b9-5f773d660ac5-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-f23d4bb3-6999885447-qkfg9\" (UID: \"bdb07ae7-7207-4fbe-94b9-5f773d660ac5\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-f23d4bb3-6999885447-qkfg9"
Oct 30 22:58:13.789625 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:13.789595    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hplcp\" (UniqueName: \"kubernetes.io/projected/bdb07ae7-7207-4fbe-94b9-5f773d660ac5-kube-api-access-hplcp\") pod \"payload-pre-processing-e2e-ait-f23d4bb3-6999885447-qkfg9\" (UID: \"bdb07ae7-7207-4fbe-94b9-5f773d660ac5\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-f23d4bb3-6999885447-qkfg9"
Oct 30 22:58:13.831880 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:13.831847    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d7629350-fdda-4ebb-b98f-618a1d30f88d-plugins-config-volume\") pod \"payload-processing-e2e-ait-f23d4bb3-78667f84d7-9hl2j\" (UID: \"d7629350-fdda-4ebb-b98f-618a1d30f88d\") " pod="openshift-ingress/payload-processing-e2e-ait-f23d4bb3-78667f84d7-9hl2j"
Oct 30 22:58:13.832041 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:13.831922    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dzz2s\" (UniqueName: \"kubernetes.io/projected/d7629350-fdda-4ebb-b98f-618a1d30f88d-kube-api-access-dzz2s\") pod \"payload-processing-e2e-ait-f23d4bb3-78667f84d7-9hl2j\" (UID: \"d7629350-fdda-4ebb-b98f-618a1d30f88d\") " pod="openshift-ingress/payload-processing-e2e-ait-f23d4bb3-78667f84d7-9hl2j"
Oct 30 22:58:13.832633 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:13.832611    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d7629350-fdda-4ebb-b98f-618a1d30f88d-plugins-config-volume\") pod \"payload-processing-e2e-ait-f23d4bb3-78667f84d7-9hl2j\" (UID: \"d7629350-fdda-4ebb-b98f-618a1d30f88d\") " pod="openshift-ingress/payload-processing-e2e-ait-f23d4bb3-78667f84d7-9hl2j"
Oct 30 22:58:13.890436 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:13.890405    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dzz2s\" (UniqueName: \"kubernetes.io/projected/d7629350-fdda-4ebb-b98f-618a1d30f88d-kube-api-access-dzz2s\") pod \"payload-processing-e2e-ait-f23d4bb3-78667f84d7-9hl2j\" (UID: \"d7629350-fdda-4ebb-b98f-618a1d30f88d\") " pod="openshift-ingress/payload-processing-e2e-ait-f23d4bb3-78667f84d7-9hl2j"
Oct 30 22:58:13.902275 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:13.902247    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-f23d4bb3-78667f84d7-9hl2j"
Oct 30 22:58:14.053375 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:14.053340    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-f23d4bb3-6999885447-qkfg9"
Oct 30 22:58:14.275096 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:14.275061    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-f23d4bb3-78667f84d7-9hl2j"]
Oct 30 22:58:14.282101 ip-10-0-136-210 kubenswrapper[5510]: W1030 22:58:14.282075    5510 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7629350_fdda_4ebb_b98f_618a1d30f88d.slice/crio-37bba20aac98127fae50ce3bba08ad3c07df45c73642e4aef38438e330a4c5a9 WatchSource:0}: Error finding container 37bba20aac98127fae50ce3bba08ad3c07df45c73642e4aef38438e330a4c5a9: Status 404 returned error can't find the container with id 37bba20aac98127fae50ce3bba08ad3c07df45c73642e4aef38438e330a4c5a9
Oct 30 22:58:14.424919 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:14.424883    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-f23d4bb3-6999885447-qkfg9"]
Oct 30 22:58:14.433129 ip-10-0-136-210 kubenswrapper[5510]: W1030 22:58:14.433104    5510 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbdb07ae7_7207_4fbe_94b9_5f773d660ac5.slice/crio-94dd428c78c887b57d3458507e180fb3ef338b128fe764866f110bf037174ca3 WatchSource:0}: Error finding container 94dd428c78c887b57d3458507e180fb3ef338b128fe764866f110bf037174ca3: Status 404 returned error can't find the container with id 94dd428c78c887b57d3458507e180fb3ef338b128fe764866f110bf037174ca3
Oct 30 22:58:14.473529 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:14.473499    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-f23d4bb3-78667f84d7-9hl2j" event={"ID":"d7629350-fdda-4ebb-b98f-618a1d30f88d","Type":"ContainerStarted","Data":"6aefdb56c6bf2437ecdd0a795abef821eb61269c80f210ce5172bceec7cd3a7e"}
Oct 30 22:58:14.473692 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:14.473559    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-f23d4bb3-78667f84d7-9hl2j"
Oct 30 22:58:14.473692 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:14.473585    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-f23d4bb3-78667f84d7-9hl2j" event={"ID":"d7629350-fdda-4ebb-b98f-618a1d30f88d","Type":"ContainerStarted","Data":"37bba20aac98127fae50ce3bba08ad3c07df45c73642e4aef38438e330a4c5a9"}
Oct 30 22:58:14.474806 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:14.474780    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-f23d4bb3-6999885447-qkfg9" event={"ID":"bdb07ae7-7207-4fbe-94b9-5f773d660ac5","Type":"ContainerStarted","Data":"94dd428c78c887b57d3458507e180fb3ef338b128fe764866f110bf037174ca3"}
Oct 30 22:58:14.586046 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:14.585945    5510 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-f23d4bb3-78667f84d7-9hl2j" podStartSLOduration=1.5859319429999998 podStartE2EDuration="1.585931943s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:58:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:58:14.584687174 +0000 UTC m=+4075.145497179" watchObservedRunningTime="2026-10-30 22:58:14.585931943 +0000 UTC m=+4075.146741942"
Oct 30 22:58:14.708810 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:14.708772    5510 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f23d4bb3-7847556659-2284l"]
Oct 30 22:58:14.713530 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:14.713500    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f23d4bb3-7847556659-2284l"
Oct 30 22:58:14.763207 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:14.763177    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-f23d4bb3-serving-cert\""
Oct 30 22:58:14.768752 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:14.768717    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f23d4bb3-7847556659-2284l"]
Oct 30 22:58:14.842224 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:14.842129    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a4dd1a9c-5d6d-48c5-addf-c094aa6b634b-openshift-service-ca\") pod \"maas-api-e2e-ait-f23d4bb3-7847556659-2284l\" (UID: \"a4dd1a9c-5d6d-48c5-addf-c094aa6b634b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f23d4bb3-7847556659-2284l"
Oct 30 22:58:14.842402 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:14.842230    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a4dd1a9c-5d6d-48c5-addf-c094aa6b634b-maas-api-tls\") pod \"maas-api-e2e-ait-f23d4bb3-7847556659-2284l\" (UID: \"a4dd1a9c-5d6d-48c5-addf-c094aa6b634b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f23d4bb3-7847556659-2284l"
Oct 30 22:58:14.842402 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:14.842330    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a4dd1a9c-5d6d-48c5-addf-c094aa6b634b-metrics-certs\") pod \"maas-api-e2e-ait-f23d4bb3-7847556659-2284l\" (UID: \"a4dd1a9c-5d6d-48c5-addf-c094aa6b634b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f23d4bb3-7847556659-2284l"
Oct 30 22:58:14.842532 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:14.842446    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x747f\" (UniqueName: \"kubernetes.io/projected/a4dd1a9c-5d6d-48c5-addf-c094aa6b634b-kube-api-access-x747f\") pod \"maas-api-e2e-ait-f23d4bb3-7847556659-2284l\" (UID: \"a4dd1a9c-5d6d-48c5-addf-c094aa6b634b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f23d4bb3-7847556659-2284l"
Oct 30 22:58:14.942597 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:14.942546    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:58:14.943097 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:14.943067    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a4dd1a9c-5d6d-48c5-addf-c094aa6b634b-openshift-service-ca\") pod \"maas-api-e2e-ait-f23d4bb3-7847556659-2284l\" (UID: \"a4dd1a9c-5d6d-48c5-addf-c094aa6b634b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f23d4bb3-7847556659-2284l"
Oct 30 22:58:14.943191 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:14.943130    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a4dd1a9c-5d6d-48c5-addf-c094aa6b634b-maas-api-tls\") pod \"maas-api-e2e-ait-f23d4bb3-7847556659-2284l\" (UID: \"a4dd1a9c-5d6d-48c5-addf-c094aa6b634b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f23d4bb3-7847556659-2284l"
Oct 30 22:58:14.943270 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:14.943251    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a4dd1a9c-5d6d-48c5-addf-c094aa6b634b-metrics-certs\") pod \"maas-api-e2e-ait-f23d4bb3-7847556659-2284l\" (UID: \"a4dd1a9c-5d6d-48c5-addf-c094aa6b634b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f23d4bb3-7847556659-2284l"
Oct 30 22:58:14.943334 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:58:14.943286    5510 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-f23d4bb3-serving-cert: secret "maas-api-e2e-ait-f23d4bb3-serving-cert" not found
Oct 30 22:58:14.943334 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:14.943303    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x747f\" (UniqueName: \"kubernetes.io/projected/a4dd1a9c-5d6d-48c5-addf-c094aa6b634b-kube-api-access-x747f\") pod \"maas-api-e2e-ait-f23d4bb3-7847556659-2284l\" (UID: \"a4dd1a9c-5d6d-48c5-addf-c094aa6b634b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f23d4bb3-7847556659-2284l"
Oct 30 22:58:14.943444 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:58:14.943353    5510 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a4dd1a9c-5d6d-48c5-addf-c094aa6b634b-maas-api-tls podName:a4dd1a9c-5d6d-48c5-addf-c094aa6b634b nodeName:}" failed. No retries permitted until 2026-10-30 22:58:15.443332383 +0000 UTC m=+4076.004142386 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/a4dd1a9c-5d6d-48c5-addf-c094aa6b634b-maas-api-tls") pod "maas-api-e2e-ait-f23d4bb3-7847556659-2284l" (UID: "a4dd1a9c-5d6d-48c5-addf-c094aa6b634b") : secret "maas-api-e2e-ait-f23d4bb3-serving-cert" not found
Oct 30 22:58:14.943878 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:14.943856    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a4dd1a9c-5d6d-48c5-addf-c094aa6b634b-openshift-service-ca\") pod \"maas-api-e2e-ait-f23d4bb3-7847556659-2284l\" (UID: \"a4dd1a9c-5d6d-48c5-addf-c094aa6b634b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f23d4bb3-7847556659-2284l"
Oct 30 22:58:14.946078 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:14.946056    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a4dd1a9c-5d6d-48c5-addf-c094aa6b634b-metrics-certs\") pod \"maas-api-e2e-ait-f23d4bb3-7847556659-2284l\" (UID: \"a4dd1a9c-5d6d-48c5-addf-c094aa6b634b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f23d4bb3-7847556659-2284l"
Oct 30 22:58:15.003960 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:15.003927    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x747f\" (UniqueName: \"kubernetes.io/projected/a4dd1a9c-5d6d-48c5-addf-c094aa6b634b-kube-api-access-x747f\") pod \"maas-api-e2e-ait-f23d4bb3-7847556659-2284l\" (UID: \"a4dd1a9c-5d6d-48c5-addf-c094aa6b634b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f23d4bb3-7847556659-2284l"
Oct 30 22:58:15.183053 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:15.182962    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:58:15.366917 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:15.366880    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:58:15.448800 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:15.448714    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a4dd1a9c-5d6d-48c5-addf-c094aa6b634b-maas-api-tls\") pod \"maas-api-e2e-ait-f23d4bb3-7847556659-2284l\" (UID: \"a4dd1a9c-5d6d-48c5-addf-c094aa6b634b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f23d4bb3-7847556659-2284l"
Oct 30 22:58:15.451170 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:15.451147    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a4dd1a9c-5d6d-48c5-addf-c094aa6b634b-maas-api-tls\") pod \"maas-api-e2e-ait-f23d4bb3-7847556659-2284l\" (UID: \"a4dd1a9c-5d6d-48c5-addf-c094aa6b634b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f23d4bb3-7847556659-2284l"
Oct 30 22:58:15.481044 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:15.481004    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-f23d4bb3-6999885447-qkfg9" event={"ID":"bdb07ae7-7207-4fbe-94b9-5f773d660ac5","Type":"ContainerStarted","Data":"e84e77c34220a9f02377b4620e67edfe22d56f4def40787f368c00bde3fce8b6"}
Oct 30 22:58:15.595902 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:15.595849    5510 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-f23d4bb3-6999885447-qkfg9" podStartSLOduration=2.59583614 podStartE2EDuration="2.59583614s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:58:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:58:15.594333389 +0000 UTC m=+4076.155143404" watchObservedRunningTime="2026-10-30 22:58:15.59583614 +0000 UTC m=+4076.156646147"
Oct 30 22:58:15.627384 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:15.627346    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f23d4bb3-7847556659-2284l"
Oct 30 22:58:16.052951 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:16.052916    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f23d4bb3-7847556659-2284l"]
Oct 30 22:58:16.060736 ip-10-0-136-210 kubenswrapper[5510]: W1030 22:58:16.060706    5510 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4dd1a9c_5d6d_48c5_addf_c094aa6b634b.slice/crio-6e175e4cdf134e87f9f3806231a5354216ed8a33921ab55ba268dc6b8d263237 WatchSource:0}: Error finding container 6e175e4cdf134e87f9f3806231a5354216ed8a33921ab55ba268dc6b8d263237: Status 404 returned error can't find the container with id 6e175e4cdf134e87f9f3806231a5354216ed8a33921ab55ba268dc6b8d263237
Oct 30 22:58:16.490083 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:16.489997    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f23d4bb3-7847556659-2284l" event={"ID":"a4dd1a9c-5d6d-48c5-addf-c094aa6b634b","Type":"ContainerStarted","Data":"6e175e4cdf134e87f9f3806231a5354216ed8a33921ab55ba268dc6b8d263237"}
Oct 30 22:58:16.490461 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:16.490236    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-f23d4bb3-6999885447-qkfg9"
Oct 30 22:58:17.496594 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:17.496562    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f23d4bb3-7847556659-2284l" event={"ID":"a4dd1a9c-5d6d-48c5-addf-c094aa6b634b","Type":"ContainerStarted","Data":"b33f22c2d8482875631f45f649cb37e27a8a5a8d3908c047dd9228cec73cf588"}
Oct 30 22:58:17.497045 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:17.496640    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f23d4bb3-7847556659-2284l"
Oct 30 22:58:17.614360 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:17.614308    5510 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f23d4bb3-7847556659-2284l" podStartSLOduration=2.7863861439999997 podStartE2EDuration="3.614283321s" totalImagesPullingTime="827.897177ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:58:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:58:17.613209633 +0000 UTC m=+4078.174019639" watchObservedRunningTime="2026-10-30 22:58:17.614283321 +0000 UTC m=+4078.175093343"
Oct 30 22:58:20.434111 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:20.434054    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:58:23.507215 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:23.507186    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f23d4bb3-7847556659-2284l"
Oct 30 22:58:25.467766 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:25.467727    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:58:25.484651 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:25.484620    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-f23d4bb3-78667f84d7-9hl2j"
Oct 30 22:58:27.499896 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:27.499865    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-f23d4bb3-6999885447-qkfg9"
Oct 30 22:58:30.415230 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:30.415182    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:58:32.592031 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:32.591996    5510 generic.go:356] "Generic (PLEG): container finished" podID="2f4c97c6-6598-4189-b7ca-0244b2799bf5" containerID="a2e32a6311b2d9833f56ece0bdbcc31bd2388c1228c7599aab8efc76e8d7a98b" exitCode=0
Oct 30 22:58:32.592527 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:32.592022    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f23d4bb3-openshift-default-865c59c95c-fzcbj" event={"ID":"2f4c97c6-6598-4189-b7ca-0244b2799bf5","Type":"ContainerDied","Data":"a2e32a6311b2d9833f56ece0bdbcc31bd2388c1228c7599aab8efc76e8d7a98b"}
Oct 30 22:58:32.592527 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:32.592437    5510 scope.go:122] "RemoveContainer" containerID="a2e32a6311b2d9833f56ece0bdbcc31bd2388c1228c7599aab8efc76e8d7a98b"
Oct 30 22:58:32.593736 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:32.593709    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:58:32.593808 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:32.593778    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:58:32.593858 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:32.593811    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:58:33.598466 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:33.598428    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f23d4bb3-openshift-default-865c59c95c-fzcbj" event={"ID":"2f4c97c6-6598-4189-b7ca-0244b2799bf5","Type":"ContainerStarted","Data":"539f204828ed8a77d5ee271d33463dc9259e6b77e5c512955267ded467a9c394"}
Oct 30 22:58:34.268853 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:34.268813    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-f23d4bb3-openshift-default-865c59c95c-fzcbj"
Oct 30 22:58:34.270136 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:34.270107    5510 patch_prober.go:29] interesting pod/e2e-ait-f23d4bb3-openshift-default-865c59c95c-fzcbj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.167:15021/healthz/ready\": dial tcp 10.133.0.167:15021: connect: connection refused" start-of-body=
Oct 30 22:58:34.270282 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:34.270172    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-f23d4bb3-openshift-default-865c59c95c-fzcbj" podUID="2f4c97c6-6598-4189-b7ca-0244b2799bf5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.167:15021/healthz/ready\": dial tcp 10.133.0.167:15021: connect: connection refused"
Oct 30 22:58:35.265233 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:35.265132    5510 patch_prober.go:29] interesting pod/e2e-ait-f23d4bb3-openshift-default-865c59c95c-fzcbj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.167:15021/healthz/ready\": dial tcp 10.133.0.167:15021: connect: connection refused" start-of-body=
Oct 30 22:58:35.265876 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:35.265840    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-f23d4bb3-openshift-default-865c59c95c-fzcbj" podUID="2f4c97c6-6598-4189-b7ca-0244b2799bf5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.167:15021/healthz/ready\": dial tcp 10.133.0.167:15021: connect: connection refused"
Oct 30 22:58:36.224728 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:36.224693    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:58:36.268684 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:36.268063    5510 patch_prober.go:29] interesting pod/e2e-ait-f23d4bb3-openshift-default-865c59c95c-fzcbj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.167:15021/healthz/ready\": dial tcp 10.133.0.167:15021: connect: connection refused" start-of-body=
Oct 30 22:58:36.268684 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:36.268143    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-f23d4bb3-openshift-default-865c59c95c-fzcbj" podUID="2f4c97c6-6598-4189-b7ca-0244b2799bf5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.167:15021/healthz/ready\": dial tcp 10.133.0.167:15021: connect: connection refused"
Oct 30 22:58:37.268684 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:37.265140    5510 patch_prober.go:29] interesting pod/e2e-ait-f23d4bb3-openshift-default-865c59c95c-fzcbj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.167:15021/healthz/ready\": dial tcp 10.133.0.167:15021: connect: connection refused" start-of-body=
Oct 30 22:58:37.268684 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:37.265204    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-f23d4bb3-openshift-default-865c59c95c-fzcbj" podUID="2f4c97c6-6598-4189-b7ca-0244b2799bf5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.167:15021/healthz/ready\": dial tcp 10.133.0.167:15021: connect: connection refused"
Oct 30 22:58:38.268129 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:38.268088    5510 patch_prober.go:29] interesting pod/e2e-ait-f23d4bb3-openshift-default-865c59c95c-fzcbj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.167:15021/healthz/ready\": dial tcp 10.133.0.167:15021: connect: connection refused" start-of-body=
Oct 30 22:58:38.268456 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:38.268423    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-f23d4bb3-openshift-default-865c59c95c-fzcbj" podUID="2f4c97c6-6598-4189-b7ca-0244b2799bf5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.167:15021/healthz/ready\": dial tcp 10.133.0.167:15021: connect: connection refused"
Oct 30 22:58:38.420890 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:58:38.420852    5510 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6gjbf" containerName="extractor"
Oct 30 22:58:38.440945 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:38.440913    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-f23d4bb3-openshift-default-865c59c95c-fzcbj"
Oct 30 22:58:39.268836 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:39.268804    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-f23d4bb3-openshift-default-865c59c95c-fzcbj"
Oct 30 22:58:39.269835 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:39.269814    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-f23d4bb3-openshift-default-865c59c95c-fzcbj"
Oct 30 22:58:41.466013 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:41.465956    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:58:41.789760 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:41.789722    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:58:46.512933 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:46.512897    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:58:51.439649 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:51.439611    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:58:56.422578 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:56.422539    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:58:56.694151 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:56.694062    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-f23d4bb3-78667f84d7-9hl2j"]
Oct 30 22:58:56.795514 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:56.795473    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-f23d4bb3-6999885447-qkfg9"]
Oct 30 22:58:57.101469 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:57.101437    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:58:57.288177 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:58:57.288143    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:59:01.327796 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:01.327758    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:59:01.695266 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:01.695166    5510 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-f23d4bb3-78667f84d7-9hl2j" podUID="d7629350-fdda-4ebb-b98f-618a1d30f88d" containerName="payload-processing" containerID="cri-o://6aefdb56c6bf2437ecdd0a795abef821eb61269c80f210ce5172bceec7cd3a7e" gracePeriod=25
Oct 30 22:59:01.796310 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:01.796271    5510 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-f23d4bb3-6999885447-qkfg9" podUID="bdb07ae7-7207-4fbe-94b9-5f773d660ac5" containerName="payload-pre-processing" containerID="cri-o://e84e77c34220a9f02377b4620e67edfe22d56f4def40787f368c00bde3fce8b6" gracePeriod=25
Oct 30 22:59:01.940753 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:01.940728    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-f23d4bb3-78667f84d7-9hl2j"
Oct 30 22:59:01.967786 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:01.967742    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d7629350-fdda-4ebb-b98f-618a1d30f88d-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d7629350-fdda-4ebb-b98f-618a1d30f88d-plugins-config-volume\") pod \"d7629350-fdda-4ebb-b98f-618a1d30f88d\" (UID: \"d7629350-fdda-4ebb-b98f-618a1d30f88d\") "
Oct 30 22:59:01.967870 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:01.967826    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d7629350-fdda-4ebb-b98f-618a1d30f88d-kube-api-access-dzz2s\" (UniqueName: \"kubernetes.io/projected/d7629350-fdda-4ebb-b98f-618a1d30f88d-kube-api-access-dzz2s\") pod \"d7629350-fdda-4ebb-b98f-618a1d30f88d\" (UID: \"d7629350-fdda-4ebb-b98f-618a1d30f88d\") "
Oct 30 22:59:01.968192 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:01.968163    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d7629350-fdda-4ebb-b98f-618a1d30f88d-plugins-config-volume" pod "d7629350-fdda-4ebb-b98f-618a1d30f88d" (UID: "d7629350-fdda-4ebb-b98f-618a1d30f88d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:59:01.972758 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:01.972716    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d7629350-fdda-4ebb-b98f-618a1d30f88d-kube-api-access-dzz2s" pod "d7629350-fdda-4ebb-b98f-618a1d30f88d" (UID: "d7629350-fdda-4ebb-b98f-618a1d30f88d"). InnerVolumeSpecName "kube-api-access-dzz2s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:59:02.034576 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:02.034552    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-f23d4bb3-6999885447-qkfg9"
Oct 30 22:59:02.069443 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:02.069419    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bdb07ae7-7207-4fbe-94b9-5f773d660ac5-kube-api-access-hplcp\" (UniqueName: \"kubernetes.io/projected/bdb07ae7-7207-4fbe-94b9-5f773d660ac5-kube-api-access-hplcp\") pod \"bdb07ae7-7207-4fbe-94b9-5f773d660ac5\" (UID: \"bdb07ae7-7207-4fbe-94b9-5f773d660ac5\") "
Oct 30 22:59:02.069577 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:02.069474    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/bdb07ae7-7207-4fbe-94b9-5f773d660ac5-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bdb07ae7-7207-4fbe-94b9-5f773d660ac5-plugins-config-volume\") pod \"bdb07ae7-7207-4fbe-94b9-5f773d660ac5\" (UID: \"bdb07ae7-7207-4fbe-94b9-5f773d660ac5\") "
Oct 30 22:59:02.069774 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:02.069762    5510 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d7629350-fdda-4ebb-b98f-618a1d30f88d-plugins-config-volume\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:59:02.069819 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:02.069776    5510 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dzz2s\" (UniqueName: \"kubernetes.io/projected/d7629350-fdda-4ebb-b98f-618a1d30f88d-kube-api-access-dzz2s\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:59:02.069886 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:02.069866    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bdb07ae7-7207-4fbe-94b9-5f773d660ac5-plugins-config-volume" pod "bdb07ae7-7207-4fbe-94b9-5f773d660ac5" (UID: "bdb07ae7-7207-4fbe-94b9-5f773d660ac5"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:59:02.071453 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:02.071436    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bdb07ae7-7207-4fbe-94b9-5f773d660ac5-kube-api-access-hplcp" pod "bdb07ae7-7207-4fbe-94b9-5f773d660ac5" (UID: "bdb07ae7-7207-4fbe-94b9-5f773d660ac5"). InnerVolumeSpecName "kube-api-access-hplcp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:59:02.171284 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:02.171246    5510 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hplcp\" (UniqueName: \"kubernetes.io/projected/bdb07ae7-7207-4fbe-94b9-5f773d660ac5-kube-api-access-hplcp\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:59:02.171284 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:02.171284    5510 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bdb07ae7-7207-4fbe-94b9-5f773d660ac5-plugins-config-volume\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:59:02.751567 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:02.751536    5510 generic.go:356] "Generic (PLEG): container finished" podID="bdb07ae7-7207-4fbe-94b9-5f773d660ac5" containerID="e84e77c34220a9f02377b4620e67edfe22d56f4def40787f368c00bde3fce8b6" exitCode=0
Oct 30 22:59:02.752055 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:02.751584    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-f23d4bb3-6999885447-qkfg9"
Oct 30 22:59:02.752055 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:02.751618    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-f23d4bb3-6999885447-qkfg9" event={"ID":"bdb07ae7-7207-4fbe-94b9-5f773d660ac5","Type":"ContainerDied","Data":"e84e77c34220a9f02377b4620e67edfe22d56f4def40787f368c00bde3fce8b6"}
Oct 30 22:59:02.752055 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:02.751678    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-f23d4bb3-6999885447-qkfg9" event={"ID":"bdb07ae7-7207-4fbe-94b9-5f773d660ac5","Type":"ContainerDied","Data":"94dd428c78c887b57d3458507e180fb3ef338b128fe764866f110bf037174ca3"}
Oct 30 22:59:02.752055 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:02.751705    5510 scope.go:122] "RemoveContainer" containerID="e84e77c34220a9f02377b4620e67edfe22d56f4def40787f368c00bde3fce8b6"
Oct 30 22:59:02.753065 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:02.753031    5510 generic.go:356] "Generic (PLEG): container finished" podID="d7629350-fdda-4ebb-b98f-618a1d30f88d" containerID="6aefdb56c6bf2437ecdd0a795abef821eb61269c80f210ce5172bceec7cd3a7e" exitCode=0
Oct 30 22:59:02.753188 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:02.753060    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-f23d4bb3-78667f84d7-9hl2j" event={"ID":"d7629350-fdda-4ebb-b98f-618a1d30f88d","Type":"ContainerDied","Data":"6aefdb56c6bf2437ecdd0a795abef821eb61269c80f210ce5172bceec7cd3a7e"}
Oct 30 22:59:02.753188 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:02.753114    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-f23d4bb3-78667f84d7-9hl2j" event={"ID":"d7629350-fdda-4ebb-b98f-618a1d30f88d","Type":"ContainerDied","Data":"37bba20aac98127fae50ce3bba08ad3c07df45c73642e4aef38438e330a4c5a9"}
Oct 30 22:59:02.753188 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:02.753082    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-f23d4bb3-78667f84d7-9hl2j"
Oct 30 22:59:02.760989 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:02.760971    5510 scope.go:122] "RemoveContainer" containerID="e84e77c34220a9f02377b4620e67edfe22d56f4def40787f368c00bde3fce8b6"
Oct 30 22:59:02.761261 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:59:02.761243    5510 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e84e77c34220a9f02377b4620e67edfe22d56f4def40787f368c00bde3fce8b6\": container with ID starting with e84e77c34220a9f02377b4620e67edfe22d56f4def40787f368c00bde3fce8b6 not found: ID does not exist" containerID="e84e77c34220a9f02377b4620e67edfe22d56f4def40787f368c00bde3fce8b6"
Oct 30 22:59:02.761389 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:02.761269    5510 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e84e77c34220a9f02377b4620e67edfe22d56f4def40787f368c00bde3fce8b6"} err="failed to get container status \"e84e77c34220a9f02377b4620e67edfe22d56f4def40787f368c00bde3fce8b6\": rpc error: code = NotFound desc = could not find container \"e84e77c34220a9f02377b4620e67edfe22d56f4def40787f368c00bde3fce8b6\": container with ID starting with e84e77c34220a9f02377b4620e67edfe22d56f4def40787f368c00bde3fce8b6 not found: ID does not exist"
Oct 30 22:59:02.761389 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:02.761286    5510 scope.go:122] "RemoveContainer" containerID="6aefdb56c6bf2437ecdd0a795abef821eb61269c80f210ce5172bceec7cd3a7e"
Oct 30 22:59:02.769158 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:02.769138    5510 scope.go:122] "RemoveContainer" containerID="6aefdb56c6bf2437ecdd0a795abef821eb61269c80f210ce5172bceec7cd3a7e"
Oct 30 22:59:02.769408 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:59:02.769387    5510 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6aefdb56c6bf2437ecdd0a795abef821eb61269c80f210ce5172bceec7cd3a7e\": container with ID starting with 6aefdb56c6bf2437ecdd0a795abef821eb61269c80f210ce5172bceec7cd3a7e not found: ID does not exist" containerID="6aefdb56c6bf2437ecdd0a795abef821eb61269c80f210ce5172bceec7cd3a7e"
Oct 30 22:59:02.769461 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:02.769415    5510 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6aefdb56c6bf2437ecdd0a795abef821eb61269c80f210ce5172bceec7cd3a7e"} err="failed to get container status \"6aefdb56c6bf2437ecdd0a795abef821eb61269c80f210ce5172bceec7cd3a7e\": rpc error: code = NotFound desc = could not find container \"6aefdb56c6bf2437ecdd0a795abef821eb61269c80f210ce5172bceec7cd3a7e\": container with ID starting with 6aefdb56c6bf2437ecdd0a795abef821eb61269c80f210ce5172bceec7cd3a7e not found: ID does not exist"
Oct 30 22:59:02.906096 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:02.906059    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-f23d4bb3-78667f84d7-9hl2j"]
Oct 30 22:59:02.953540 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:02.953504    5510 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-f23d4bb3-78667f84d7-9hl2j"]
Oct 30 22:59:03.015835 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:03.015757    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-f23d4bb3-6999885447-qkfg9"]
Oct 30 22:59:03.020553 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:03.020527    5510 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-f23d4bb3-6999885447-qkfg9"]
Oct 30 22:59:03.551587 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:03.551544    5510 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="bdb07ae7-7207-4fbe-94b9-5f773d660ac5" path="/var/lib/kubelet/pods/bdb07ae7-7207-4fbe-94b9-5f773d660ac5/volumes"
Oct 30 22:59:03.552079 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:03.552057    5510 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d7629350-fdda-4ebb-b98f-618a1d30f88d" path="/var/lib/kubelet/pods/d7629350-fdda-4ebb-b98f-618a1d30f88d/volumes"
Oct 30 22:59:05.492372 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:05.492334    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f23d4bb3-7847556659-2284l"]
Oct 30 22:59:05.492820 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:05.492545    5510 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f23d4bb3-7847556659-2284l" podUID="a4dd1a9c-5d6d-48c5-addf-c094aa6b634b" containerName="maas-api" containerID="cri-o://b33f22c2d8482875631f45f649cb37e27a8a5a8d3908c047dd9228cec73cf588" gracePeriod=30
Oct 30 22:59:05.646647 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:05.646540    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:59:05.749962 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:05.749889    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f23d4bb3-7847556659-2284l"
Oct 30 22:59:05.771252 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:05.771219    5510 generic.go:356] "Generic (PLEG): container finished" podID="a4dd1a9c-5d6d-48c5-addf-c094aa6b634b" containerID="b33f22c2d8482875631f45f649cb37e27a8a5a8d3908c047dd9228cec73cf588" exitCode=0
Oct 30 22:59:05.771440 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:05.771267    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f23d4bb3-7847556659-2284l"
Oct 30 22:59:05.771440 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:05.771298    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f23d4bb3-7847556659-2284l" event={"ID":"a4dd1a9c-5d6d-48c5-addf-c094aa6b634b","Type":"ContainerDied","Data":"b33f22c2d8482875631f45f649cb37e27a8a5a8d3908c047dd9228cec73cf588"}
Oct 30 22:59:05.771440 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:05.771358    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f23d4bb3-7847556659-2284l" event={"ID":"a4dd1a9c-5d6d-48c5-addf-c094aa6b634b","Type":"ContainerDied","Data":"6e175e4cdf134e87f9f3806231a5354216ed8a33921ab55ba268dc6b8d263237"}
Oct 30 22:59:05.771440 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:05.771373    5510 scope.go:122] "RemoveContainer" containerID="b33f22c2d8482875631f45f649cb37e27a8a5a8d3908c047dd9228cec73cf588"
Oct 30 22:59:05.780501 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:05.780482    5510 scope.go:122] "RemoveContainer" containerID="b33f22c2d8482875631f45f649cb37e27a8a5a8d3908c047dd9228cec73cf588"
Oct 30 22:59:05.780824 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:59:05.780801    5510 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b33f22c2d8482875631f45f649cb37e27a8a5a8d3908c047dd9228cec73cf588\": container with ID starting with b33f22c2d8482875631f45f649cb37e27a8a5a8d3908c047dd9228cec73cf588 not found: ID does not exist" containerID="b33f22c2d8482875631f45f649cb37e27a8a5a8d3908c047dd9228cec73cf588"
Oct 30 22:59:05.780886 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:05.780831    5510 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b33f22c2d8482875631f45f649cb37e27a8a5a8d3908c047dd9228cec73cf588"} err="failed to get container status \"b33f22c2d8482875631f45f649cb37e27a8a5a8d3908c047dd9228cec73cf588\": rpc error: code = NotFound desc = could not find container \"b33f22c2d8482875631f45f649cb37e27a8a5a8d3908c047dd9228cec73cf588\": container with ID starting with b33f22c2d8482875631f45f649cb37e27a8a5a8d3908c047dd9228cec73cf588 not found: ID does not exist"
Oct 30 22:59:05.806940 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:05.806907    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a4dd1a9c-5d6d-48c5-addf-c094aa6b634b-kube-api-access-x747f\" (UniqueName: \"kubernetes.io/projected/a4dd1a9c-5d6d-48c5-addf-c094aa6b634b-kube-api-access-x747f\") pod \"a4dd1a9c-5d6d-48c5-addf-c094aa6b634b\" (UID: \"a4dd1a9c-5d6d-48c5-addf-c094aa6b634b\") "
Oct 30 22:59:05.807076 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:05.806944    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a4dd1a9c-5d6d-48c5-addf-c094aa6b634b-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a4dd1a9c-5d6d-48c5-addf-c094aa6b634b-maas-api-tls\") pod \"a4dd1a9c-5d6d-48c5-addf-c094aa6b634b\" (UID: \"a4dd1a9c-5d6d-48c5-addf-c094aa6b634b\") "
Oct 30 22:59:05.807076 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:05.807019    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a4dd1a9c-5d6d-48c5-addf-c094aa6b634b-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a4dd1a9c-5d6d-48c5-addf-c094aa6b634b-openshift-service-ca\") pod \"a4dd1a9c-5d6d-48c5-addf-c094aa6b634b\" (UID: \"a4dd1a9c-5d6d-48c5-addf-c094aa6b634b\") "
Oct 30 22:59:05.807156 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:05.807080    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a4dd1a9c-5d6d-48c5-addf-c094aa6b634b-metrics-certs\" (UniqueName: \"kubernetes.io/secret/a4dd1a9c-5d6d-48c5-addf-c094aa6b634b-metrics-certs\") pod \"a4dd1a9c-5d6d-48c5-addf-c094aa6b634b\" (UID: \"a4dd1a9c-5d6d-48c5-addf-c094aa6b634b\") "
Oct 30 22:59:05.807470 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:05.807442    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a4dd1a9c-5d6d-48c5-addf-c094aa6b634b-openshift-service-ca" pod "a4dd1a9c-5d6d-48c5-addf-c094aa6b634b" (UID: "a4dd1a9c-5d6d-48c5-addf-c094aa6b634b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:59:05.809183 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:05.809150    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a4dd1a9c-5d6d-48c5-addf-c094aa6b634b-metrics-certs" pod "a4dd1a9c-5d6d-48c5-addf-c094aa6b634b" (UID: "a4dd1a9c-5d6d-48c5-addf-c094aa6b634b"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:59:05.809183 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:05.809166    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a4dd1a9c-5d6d-48c5-addf-c094aa6b634b-maas-api-tls" pod "a4dd1a9c-5d6d-48c5-addf-c094aa6b634b" (UID: "a4dd1a9c-5d6d-48c5-addf-c094aa6b634b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:59:05.809318 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:05.809220    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a4dd1a9c-5d6d-48c5-addf-c094aa6b634b-kube-api-access-x747f" pod "a4dd1a9c-5d6d-48c5-addf-c094aa6b634b" (UID: "a4dd1a9c-5d6d-48c5-addf-c094aa6b634b"). InnerVolumeSpecName "kube-api-access-x747f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:59:05.909019 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:05.908982    5510 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x747f\" (UniqueName: \"kubernetes.io/projected/a4dd1a9c-5d6d-48c5-addf-c094aa6b634b-kube-api-access-x747f\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:59:05.909019 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:05.909013    5510 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a4dd1a9c-5d6d-48c5-addf-c094aa6b634b-maas-api-tls\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:59:05.909019 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:05.909023    5510 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a4dd1a9c-5d6d-48c5-addf-c094aa6b634b-openshift-service-ca\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:59:05.909266 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:05.909032    5510 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a4dd1a9c-5d6d-48c5-addf-c094aa6b634b-metrics-certs\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:59:06.187451 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:06.187413    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f23d4bb3-7847556659-2284l"]
Oct 30 22:59:06.195357 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:06.195328    5510 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f23d4bb3-7847556659-2284l"]
Oct 30 22:59:07.176726 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:07.176684    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:59:07.465882 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:07.465786    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:59:07.551491 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:07.551461    5510 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a4dd1a9c-5d6d-48c5-addf-c094aa6b634b" path="/var/lib/kubelet/pods/a4dd1a9c-5d6d-48c5-addf-c094aa6b634b/volumes"
Oct 30 22:59:08.420325 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:59:08.420292    5510 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6gjbf" containerName="extractor"
Oct 30 22:59:12.490838 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:12.490798    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-f23d4bb3-openshift-default-865c59c95c-fzcbj"]
Oct 30 22:59:12.491392 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:12.491016    5510 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-f23d4bb3-openshift-default-865c59c95c-fzcbj" podUID="2f4c97c6-6598-4189-b7ca-0244b2799bf5" containerName="istio-proxy" containerID="cri-o://539f204828ed8a77d5ee271d33463dc9259e6b77e5c512955267ded467a9c394" gracePeriod=30
Oct 30 22:59:12.539585 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:12.539549    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:59:13.778984 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:13.778945    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:59:14.269680 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:14.269633    5510 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-authcfg-http-31179c-openshift-default-58959bdf7-x52mb"]
Oct 30 22:59:14.270121 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:14.270098    5510 cpu_manager.go:414] "containerMap: removing container" podUID="d7629350-fdda-4ebb-b98f-618a1d30f88d" containerName="payload-processing"
Oct 30 22:59:14.270121 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:14.270114    5510 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d7629350-fdda-4ebb-b98f-618a1d30f88d" containerName="payload-processing"
Oct 30 22:59:14.270214 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:14.270143    5510 cpu_manager.go:414] "containerMap: removing container" podUID="a4dd1a9c-5d6d-48c5-addf-c094aa6b634b" containerName="maas-api"
Oct 30 22:59:14.270214 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:14.270149    5510 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a4dd1a9c-5d6d-48c5-addf-c094aa6b634b" containerName="maas-api"
Oct 30 22:59:14.270214 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:14.270171    5510 cpu_manager.go:414] "containerMap: removing container" podUID="bdb07ae7-7207-4fbe-94b9-5f773d660ac5" containerName="payload-pre-processing"
Oct 30 22:59:14.270214 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:14.270176    5510 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bdb07ae7-7207-4fbe-94b9-5f773d660ac5" containerName="payload-pre-processing"
Oct 30 22:59:14.270350 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:14.270243    5510 memory_manager.go:365] "RemoveStaleState removing state" podUID="bdb07ae7-7207-4fbe-94b9-5f773d660ac5" containerName="payload-pre-processing"
Oct 30 22:59:14.270350 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:14.270254    5510 memory_manager.go:365] "RemoveStaleState removing state" podUID="d7629350-fdda-4ebb-b98f-618a1d30f88d" containerName="payload-processing"
Oct 30 22:59:14.270350 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:14.270263    5510 memory_manager.go:365] "RemoveStaleState removing state" podUID="a4dd1a9c-5d6d-48c5-addf-c094aa6b634b" containerName="maas-api"
Oct 30 22:59:14.273283 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:14.273264    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-authcfg-http-31179c-openshift-default-58959bdf7-x52mb"
Oct 30 22:59:14.337493 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:14.337441    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-authcfg-http-31179c-openshift-default-dockercfg-rt56n\""
Oct 30 22:59:14.376858 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:14.376823    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-authcfg-http-31179c-openshift-default-58959bdf7-x52mb"]
Oct 30 22:59:14.399683 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:14.399618    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fb6d6c9f-c79a-4cf8-b896-3cad3c60cbdf-workload-certs\") pod \"e2e-authcfg-http-31179c-openshift-default-58959bdf7-x52mb\" (UID: \"fb6d6c9f-c79a-4cf8-b896-3cad3c60cbdf\") " pod="openshift-ingress/e2e-authcfg-http-31179c-openshift-default-58959bdf7-x52mb"
Oct 30 22:59:14.399895 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:14.399733    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb6d6c9f-c79a-4cf8-b896-3cad3c60cbdf-credential-socket\") pod \"e2e-authcfg-http-31179c-openshift-default-58959bdf7-x52mb\" (UID: \"fb6d6c9f-c79a-4cf8-b896-3cad3c60cbdf\") " pod="openshift-ingress/e2e-authcfg-http-31179c-openshift-default-58959bdf7-x52mb"
Oct 30 22:59:14.399895 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:14.399786    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb6d6c9f-c79a-4cf8-b896-3cad3c60cbdf-workload-socket\") pod \"e2e-authcfg-http-31179c-openshift-default-58959bdf7-x52mb\" (UID: \"fb6d6c9f-c79a-4cf8-b896-3cad3c60cbdf\") " pod="openshift-ingress/e2e-authcfg-http-31179c-openshift-default-58959bdf7-x52mb"
Oct 30 22:59:14.399895 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:14.399816    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fb6d6c9f-c79a-4cf8-b896-3cad3c60cbdf-istiod-ca-cert\") pod \"e2e-authcfg-http-31179c-openshift-default-58959bdf7-x52mb\" (UID: \"fb6d6c9f-c79a-4cf8-b896-3cad3c60cbdf\") " pod="openshift-ingress/e2e-authcfg-http-31179c-openshift-default-58959bdf7-x52mb"
Oct 30 22:59:14.399895 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:14.399838    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fb6d6c9f-c79a-4cf8-b896-3cad3c60cbdf-istio-envoy\") pod \"e2e-authcfg-http-31179c-openshift-default-58959bdf7-x52mb\" (UID: \"fb6d6c9f-c79a-4cf8-b896-3cad3c60cbdf\") " pod="openshift-ingress/e2e-authcfg-http-31179c-openshift-default-58959bdf7-x52mb"
Oct 30 22:59:14.399895 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:14.399865    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fb6d6c9f-c79a-4cf8-b896-3cad3c60cbdf-istio-podinfo\") pod \"e2e-authcfg-http-31179c-openshift-default-58959bdf7-x52mb\" (UID: \"fb6d6c9f-c79a-4cf8-b896-3cad3c60cbdf\") " pod="openshift-ingress/e2e-authcfg-http-31179c-openshift-default-58959bdf7-x52mb"
Oct 30 22:59:14.400143 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:14.399972    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wrpcc\" (UniqueName: \"kubernetes.io/projected/fb6d6c9f-c79a-4cf8-b896-3cad3c60cbdf-kube-api-access-wrpcc\") pod \"e2e-authcfg-http-31179c-openshift-default-58959bdf7-x52mb\" (UID: \"fb6d6c9f-c79a-4cf8-b896-3cad3c60cbdf\") " pod="openshift-ingress/e2e-authcfg-http-31179c-openshift-default-58959bdf7-x52mb"
Oct 30 22:59:14.400143 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:14.400004    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fb6d6c9f-c79a-4cf8-b896-3cad3c60cbdf-istio-data\") pod \"e2e-authcfg-http-31179c-openshift-default-58959bdf7-x52mb\" (UID: \"fb6d6c9f-c79a-4cf8-b896-3cad3c60cbdf\") " pod="openshift-ingress/e2e-authcfg-http-31179c-openshift-default-58959bdf7-x52mb"
Oct 30 22:59:14.400143 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:14.400040    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fb6d6c9f-c79a-4cf8-b896-3cad3c60cbdf-istio-token\") pod \"e2e-authcfg-http-31179c-openshift-default-58959bdf7-x52mb\" (UID: \"fb6d6c9f-c79a-4cf8-b896-3cad3c60cbdf\") " pod="openshift-ingress/e2e-authcfg-http-31179c-openshift-default-58959bdf7-x52mb"
Oct 30 22:59:14.400377 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:14.400343    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:59:14.501078 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:14.501035    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fb6d6c9f-c79a-4cf8-b896-3cad3c60cbdf-workload-certs\") pod \"e2e-authcfg-http-31179c-openshift-default-58959bdf7-x52mb\" (UID: \"fb6d6c9f-c79a-4cf8-b896-3cad3c60cbdf\") " pod="openshift-ingress/e2e-authcfg-http-31179c-openshift-default-58959bdf7-x52mb"
Oct 30 22:59:14.501078 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:14.501083    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb6d6c9f-c79a-4cf8-b896-3cad3c60cbdf-credential-socket\") pod \"e2e-authcfg-http-31179c-openshift-default-58959bdf7-x52mb\" (UID: \"fb6d6c9f-c79a-4cf8-b896-3cad3c60cbdf\") " pod="openshift-ingress/e2e-authcfg-http-31179c-openshift-default-58959bdf7-x52mb"
Oct 30 22:59:14.501352 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:14.501126    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb6d6c9f-c79a-4cf8-b896-3cad3c60cbdf-workload-socket\") pod \"e2e-authcfg-http-31179c-openshift-default-58959bdf7-x52mb\" (UID: \"fb6d6c9f-c79a-4cf8-b896-3cad3c60cbdf\") " pod="openshift-ingress/e2e-authcfg-http-31179c-openshift-default-58959bdf7-x52mb"
Oct 30 22:59:14.501352 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:14.501147    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fb6d6c9f-c79a-4cf8-b896-3cad3c60cbdf-istiod-ca-cert\") pod \"e2e-authcfg-http-31179c-openshift-default-58959bdf7-x52mb\" (UID: \"fb6d6c9f-c79a-4cf8-b896-3cad3c60cbdf\") " pod="openshift-ingress/e2e-authcfg-http-31179c-openshift-default-58959bdf7-x52mb"
Oct 30 22:59:14.501352 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:14.501171    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fb6d6c9f-c79a-4cf8-b896-3cad3c60cbdf-istio-envoy\") pod \"e2e-authcfg-http-31179c-openshift-default-58959bdf7-x52mb\" (UID: \"fb6d6c9f-c79a-4cf8-b896-3cad3c60cbdf\") " pod="openshift-ingress/e2e-authcfg-http-31179c-openshift-default-58959bdf7-x52mb"
Oct 30 22:59:14.501352 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:14.501193    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fb6d6c9f-c79a-4cf8-b896-3cad3c60cbdf-istio-podinfo\") pod \"e2e-authcfg-http-31179c-openshift-default-58959bdf7-x52mb\" (UID: \"fb6d6c9f-c79a-4cf8-b896-3cad3c60cbdf\") " pod="openshift-ingress/e2e-authcfg-http-31179c-openshift-default-58959bdf7-x52mb"
Oct 30 22:59:14.501352 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:14.501248    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wrpcc\" (UniqueName: \"kubernetes.io/projected/fb6d6c9f-c79a-4cf8-b896-3cad3c60cbdf-kube-api-access-wrpcc\") pod \"e2e-authcfg-http-31179c-openshift-default-58959bdf7-x52mb\" (UID: \"fb6d6c9f-c79a-4cf8-b896-3cad3c60cbdf\") " pod="openshift-ingress/e2e-authcfg-http-31179c-openshift-default-58959bdf7-x52mb"
Oct 30 22:59:14.501352 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:14.501285    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fb6d6c9f-c79a-4cf8-b896-3cad3c60cbdf-istio-data\") pod \"e2e-authcfg-http-31179c-openshift-default-58959bdf7-x52mb\" (UID: \"fb6d6c9f-c79a-4cf8-b896-3cad3c60cbdf\") " pod="openshift-ingress/e2e-authcfg-http-31179c-openshift-default-58959bdf7-x52mb"
Oct 30 22:59:14.501352 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:14.501334    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fb6d6c9f-c79a-4cf8-b896-3cad3c60cbdf-istio-token\") pod \"e2e-authcfg-http-31179c-openshift-default-58959bdf7-x52mb\" (UID: \"fb6d6c9f-c79a-4cf8-b896-3cad3c60cbdf\") " pod="openshift-ingress/e2e-authcfg-http-31179c-openshift-default-58959bdf7-x52mb"
Oct 30 22:59:14.501856 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:14.501510    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fb6d6c9f-c79a-4cf8-b896-3cad3c60cbdf-workload-certs\") pod \"e2e-authcfg-http-31179c-openshift-default-58959bdf7-x52mb\" (UID: \"fb6d6c9f-c79a-4cf8-b896-3cad3c60cbdf\") " pod="openshift-ingress/e2e-authcfg-http-31179c-openshift-default-58959bdf7-x52mb"
Oct 30 22:59:14.501856 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:14.501550    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb6d6c9f-c79a-4cf8-b896-3cad3c60cbdf-credential-socket\") pod \"e2e-authcfg-http-31179c-openshift-default-58959bdf7-x52mb\" (UID: \"fb6d6c9f-c79a-4cf8-b896-3cad3c60cbdf\") " pod="openshift-ingress/e2e-authcfg-http-31179c-openshift-default-58959bdf7-x52mb"
Oct 30 22:59:14.501856 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:14.501607    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb6d6c9f-c79a-4cf8-b896-3cad3c60cbdf-workload-socket\") pod \"e2e-authcfg-http-31179c-openshift-default-58959bdf7-x52mb\" (UID: \"fb6d6c9f-c79a-4cf8-b896-3cad3c60cbdf\") " pod="openshift-ingress/e2e-authcfg-http-31179c-openshift-default-58959bdf7-x52mb"
Oct 30 22:59:14.501856 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:14.501786    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fb6d6c9f-c79a-4cf8-b896-3cad3c60cbdf-istio-data\") pod \"e2e-authcfg-http-31179c-openshift-default-58959bdf7-x52mb\" (UID: \"fb6d6c9f-c79a-4cf8-b896-3cad3c60cbdf\") " pod="openshift-ingress/e2e-authcfg-http-31179c-openshift-default-58959bdf7-x52mb"
Oct 30 22:59:14.502099 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:14.501903    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fb6d6c9f-c79a-4cf8-b896-3cad3c60cbdf-istiod-ca-cert\") pod \"e2e-authcfg-http-31179c-openshift-default-58959bdf7-x52mb\" (UID: \"fb6d6c9f-c79a-4cf8-b896-3cad3c60cbdf\") " pod="openshift-ingress/e2e-authcfg-http-31179c-openshift-default-58959bdf7-x52mb"
Oct 30 22:59:14.503734 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:14.503712    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fb6d6c9f-c79a-4cf8-b896-3cad3c60cbdf-istio-envoy\") pod \"e2e-authcfg-http-31179c-openshift-default-58959bdf7-x52mb\" (UID: \"fb6d6c9f-c79a-4cf8-b896-3cad3c60cbdf\") " pod="openshift-ingress/e2e-authcfg-http-31179c-openshift-default-58959bdf7-x52mb"
Oct 30 22:59:14.504049 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:14.504028    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fb6d6c9f-c79a-4cf8-b896-3cad3c60cbdf-istio-podinfo\") pod \"e2e-authcfg-http-31179c-openshift-default-58959bdf7-x52mb\" (UID: \"fb6d6c9f-c79a-4cf8-b896-3cad3c60cbdf\") " pod="openshift-ingress/e2e-authcfg-http-31179c-openshift-default-58959bdf7-x52mb"
Oct 30 22:59:14.599379 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:14.599305    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fb6d6c9f-c79a-4cf8-b896-3cad3c60cbdf-istio-token\") pod \"e2e-authcfg-http-31179c-openshift-default-58959bdf7-x52mb\" (UID: \"fb6d6c9f-c79a-4cf8-b896-3cad3c60cbdf\") " pod="openshift-ingress/e2e-authcfg-http-31179c-openshift-default-58959bdf7-x52mb"
Oct 30 22:59:14.599877 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:14.599858    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wrpcc\" (UniqueName: \"kubernetes.io/projected/fb6d6c9f-c79a-4cf8-b896-3cad3c60cbdf-kube-api-access-wrpcc\") pod \"e2e-authcfg-http-31179c-openshift-default-58959bdf7-x52mb\" (UID: \"fb6d6c9f-c79a-4cf8-b896-3cad3c60cbdf\") " pod="openshift-ingress/e2e-authcfg-http-31179c-openshift-default-58959bdf7-x52mb"
Oct 30 22:59:14.713268 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:14.713233    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:59:14.885409 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:14.885327    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-authcfg-http-31179c-openshift-default-58959bdf7-x52mb"
Oct 30 22:59:15.316952 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:15.316921    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-authcfg-http-31179c-openshift-default-58959bdf7-x52mb"]
Oct 30 22:59:15.324388 ip-10-0-136-210 kubenswrapper[5510]: W1030 22:59:15.324364    5510 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb6d6c9f_c79a_4cf8_b896_3cad3c60cbdf.slice/crio-c7c672ec9fa31b30e31ce288a0fe549271a637842b21d03d47969022e56d6bba WatchSource:0}: Error finding container c7c672ec9fa31b30e31ce288a0fe549271a637842b21d03d47969022e56d6bba: Status 404 returned error can't find the container with id c7c672ec9fa31b30e31ce288a0fe549271a637842b21d03d47969022e56d6bba
Oct 30 22:59:15.326630 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:15.326596    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:59:15.326728 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:15.326701    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:59:15.326775 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:15.326737    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:59:15.821683 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:15.821619    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-authcfg-http-31179c-openshift-default-58959bdf7-x52mb" event={"ID":"fb6d6c9f-c79a-4cf8-b896-3cad3c60cbdf","Type":"ContainerStarted","Data":"d78363b65c03f13953e3d76f977eda8123bc01479e43998cd110f0a87c80411a"}
Oct 30 22:59:15.821859 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:15.821690    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-authcfg-http-31179c-openshift-default-58959bdf7-x52mb" event={"ID":"fb6d6c9f-c79a-4cf8-b896-3cad3c60cbdf","Type":"ContainerStarted","Data":"c7c672ec9fa31b30e31ce288a0fe549271a637842b21d03d47969022e56d6bba"}
Oct 30 22:59:15.886295 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:15.886263    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-authcfg-http-31179c-openshift-default-58959bdf7-x52mb"
Oct 30 22:59:15.983623 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:15.983556    5510 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-authcfg-http-31179c-openshift-default-58959bdf7-x52mb" podStartSLOduration=1.983541569 podStartE2EDuration="1.983541569s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:59:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:59:15.982931309 +0000 UTC m=+4136.543741330" watchObservedRunningTime="2026-10-30 22:59:15.983541569 +0000 UTC m=+4136.544351573"
Oct 30 22:59:16.891979 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:16.891938    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-authcfg-http-31179c-openshift-default-58959bdf7-x52mb"
Oct 30 22:59:17.748550 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:17.748521    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-f23d4bb3-openshift-default-865c59c95c-fzcbj"
Oct 30 22:59:17.833515 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:17.833478    5510 generic.go:356] "Generic (PLEG): container finished" podID="2f4c97c6-6598-4189-b7ca-0244b2799bf5" containerID="539f204828ed8a77d5ee271d33463dc9259e6b77e5c512955267ded467a9c394" exitCode=0
Oct 30 22:59:17.833737 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:17.833529    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-f23d4bb3-openshift-default-865c59c95c-fzcbj"
Oct 30 22:59:17.833737 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:17.833561    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f23d4bb3-openshift-default-865c59c95c-fzcbj" event={"ID":"2f4c97c6-6598-4189-b7ca-0244b2799bf5","Type":"ContainerDied","Data":"539f204828ed8a77d5ee271d33463dc9259e6b77e5c512955267ded467a9c394"}
Oct 30 22:59:17.833737 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:17.833617    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f23d4bb3-openshift-default-865c59c95c-fzcbj" event={"ID":"2f4c97c6-6598-4189-b7ca-0244b2799bf5","Type":"ContainerDied","Data":"b24b5101d8335bc28e94bc15f50ebd7f542ce7d62eaca8358fd10f6f130dc986"}
Oct 30 22:59:17.833737 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:17.833640    5510 scope.go:122] "RemoveContainer" containerID="539f204828ed8a77d5ee271d33463dc9259e6b77e5c512955267ded467a9c394"
Oct 30 22:59:17.833980 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:17.833912    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-authcfg-http-31179c-openshift-default-58959bdf7-x52mb"
Oct 30 22:59:17.834988 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:17.834968    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-authcfg-http-31179c-openshift-default-58959bdf7-x52mb"
Oct 30 22:59:17.835102 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:17.835080    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2f4c97c6-6598-4189-b7ca-0244b2799bf5-istio-token\" (UniqueName: \"kubernetes.io/projected/2f4c97c6-6598-4189-b7ca-0244b2799bf5-istio-token\") pod \"2f4c97c6-6598-4189-b7ca-0244b2799bf5\" (UID: \"2f4c97c6-6598-4189-b7ca-0244b2799bf5\") "
Oct 30 22:59:17.835168 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:17.835134    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2f4c97c6-6598-4189-b7ca-0244b2799bf5-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2f4c97c6-6598-4189-b7ca-0244b2799bf5-istio-data\") pod \"2f4c97c6-6598-4189-b7ca-0244b2799bf5\" (UID: \"2f4c97c6-6598-4189-b7ca-0244b2799bf5\") "
Oct 30 22:59:17.835224 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:17.835168    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2f4c97c6-6598-4189-b7ca-0244b2799bf5-kube-api-access-c6bn2\" (UniqueName: \"kubernetes.io/projected/2f4c97c6-6598-4189-b7ca-0244b2799bf5-kube-api-access-c6bn2\") pod \"2f4c97c6-6598-4189-b7ca-0244b2799bf5\" (UID: \"2f4c97c6-6598-4189-b7ca-0244b2799bf5\") "
Oct 30 22:59:17.835299 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:17.835285    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2f4c97c6-6598-4189-b7ca-0244b2799bf5-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2f4c97c6-6598-4189-b7ca-0244b2799bf5-workload-certs\") pod \"2f4c97c6-6598-4189-b7ca-0244b2799bf5\" (UID: \"2f4c97c6-6598-4189-b7ca-0244b2799bf5\") "
Oct 30 22:59:17.835363 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:17.835326    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2f4c97c6-6598-4189-b7ca-0244b2799bf5-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2f4c97c6-6598-4189-b7ca-0244b2799bf5-istiod-ca-cert\") pod \"2f4c97c6-6598-4189-b7ca-0244b2799bf5\" (UID: \"2f4c97c6-6598-4189-b7ca-0244b2799bf5\") "
Oct 30 22:59:17.835363 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:17.835356    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2f4c97c6-6598-4189-b7ca-0244b2799bf5-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f4c97c6-6598-4189-b7ca-0244b2799bf5-workload-socket\") pod \"2f4c97c6-6598-4189-b7ca-0244b2799bf5\" (UID: \"2f4c97c6-6598-4189-b7ca-0244b2799bf5\") "
Oct 30 22:59:17.835473 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:17.835394    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2f4c97c6-6598-4189-b7ca-0244b2799bf5-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f4c97c6-6598-4189-b7ca-0244b2799bf5-credential-socket\") pod \"2f4c97c6-6598-4189-b7ca-0244b2799bf5\" (UID: \"2f4c97c6-6598-4189-b7ca-0244b2799bf5\") "
Oct 30 22:59:17.835532 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:17.835482    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2f4c97c6-6598-4189-b7ca-0244b2799bf5-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2f4c97c6-6598-4189-b7ca-0244b2799bf5-istio-envoy\") pod \"2f4c97c6-6598-4189-b7ca-0244b2799bf5\" (UID: \"2f4c97c6-6598-4189-b7ca-0244b2799bf5\") "
Oct 30 22:59:17.835588 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:17.835530    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/2f4c97c6-6598-4189-b7ca-0244b2799bf5-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2f4c97c6-6598-4189-b7ca-0244b2799bf5-istio-podinfo\") pod \"2f4c97c6-6598-4189-b7ca-0244b2799bf5\" (UID: \"2f4c97c6-6598-4189-b7ca-0244b2799bf5\") "
Oct 30 22:59:17.835644 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:17.835617    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f4c97c6-6598-4189-b7ca-0244b2799bf5-workload-certs" pod "2f4c97c6-6598-4189-b7ca-0244b2799bf5" (UID: "2f4c97c6-6598-4189-b7ca-0244b2799bf5"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:59:17.836224 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:17.835767    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f4c97c6-6598-4189-b7ca-0244b2799bf5-credential-socket" pod "2f4c97c6-6598-4189-b7ca-0244b2799bf5" (UID: "2f4c97c6-6598-4189-b7ca-0244b2799bf5"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:59:17.836224 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:17.835949    5510 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2f4c97c6-6598-4189-b7ca-0244b2799bf5-workload-certs\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:59:17.836224 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:17.835967    5510 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f4c97c6-6598-4189-b7ca-0244b2799bf5-credential-socket\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:59:17.836224 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:17.836191    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f4c97c6-6598-4189-b7ca-0244b2799bf5-workload-socket" pod "2f4c97c6-6598-4189-b7ca-0244b2799bf5" (UID: "2f4c97c6-6598-4189-b7ca-0244b2799bf5"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:59:17.836224 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:17.836203    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f4c97c6-6598-4189-b7ca-0244b2799bf5-istio-data" pod "2f4c97c6-6598-4189-b7ca-0244b2799bf5" (UID: "2f4c97c6-6598-4189-b7ca-0244b2799bf5"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:59:17.836563 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:17.836248    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2f4c97c6-6598-4189-b7ca-0244b2799bf5-istiod-ca-cert" pod "2f4c97c6-6598-4189-b7ca-0244b2799bf5" (UID: "2f4c97c6-6598-4189-b7ca-0244b2799bf5"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:59:17.837542 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:17.837518    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2f4c97c6-6598-4189-b7ca-0244b2799bf5-kube-api-access-c6bn2" pod "2f4c97c6-6598-4189-b7ca-0244b2799bf5" (UID: "2f4c97c6-6598-4189-b7ca-0244b2799bf5"). InnerVolumeSpecName "kube-api-access-c6bn2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:59:17.837755 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:17.837712    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2f4c97c6-6598-4189-b7ca-0244b2799bf5-istio-token" pod "2f4c97c6-6598-4189-b7ca-0244b2799bf5" (UID: "2f4c97c6-6598-4189-b7ca-0244b2799bf5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:59:17.838015 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:17.837988    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2f4c97c6-6598-4189-b7ca-0244b2799bf5-istio-podinfo" pod "2f4c97c6-6598-4189-b7ca-0244b2799bf5" (UID: "2f4c97c6-6598-4189-b7ca-0244b2799bf5"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:59:17.838420 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:17.838398    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f4c97c6-6598-4189-b7ca-0244b2799bf5-istio-envoy" pod "2f4c97c6-6598-4189-b7ca-0244b2799bf5" (UID: "2f4c97c6-6598-4189-b7ca-0244b2799bf5"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:59:17.856850 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:17.856828    5510 scope.go:122] "RemoveContainer" containerID="a2e32a6311b2d9833f56ece0bdbcc31bd2388c1228c7599aab8efc76e8d7a98b"
Oct 30 22:59:17.865544 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:17.865517    5510 scope.go:122] "RemoveContainer" containerID="539f204828ed8a77d5ee271d33463dc9259e6b77e5c512955267ded467a9c394"
Oct 30 22:59:17.865874 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:59:17.865850    5510 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"539f204828ed8a77d5ee271d33463dc9259e6b77e5c512955267ded467a9c394\": container with ID starting with 539f204828ed8a77d5ee271d33463dc9259e6b77e5c512955267ded467a9c394 not found: ID does not exist" containerID="539f204828ed8a77d5ee271d33463dc9259e6b77e5c512955267ded467a9c394"
Oct 30 22:59:17.865991 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:17.865880    5510 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"539f204828ed8a77d5ee271d33463dc9259e6b77e5c512955267ded467a9c394"} err="failed to get container status \"539f204828ed8a77d5ee271d33463dc9259e6b77e5c512955267ded467a9c394\": rpc error: code = NotFound desc = could not find container \"539f204828ed8a77d5ee271d33463dc9259e6b77e5c512955267ded467a9c394\": container with ID starting with 539f204828ed8a77d5ee271d33463dc9259e6b77e5c512955267ded467a9c394 not found: ID does not exist"
Oct 30 22:59:17.865991 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:17.865899    5510 scope.go:122] "RemoveContainer" containerID="a2e32a6311b2d9833f56ece0bdbcc31bd2388c1228c7599aab8efc76e8d7a98b"
Oct 30 22:59:17.866157 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:59:17.866137    5510 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a2e32a6311b2d9833f56ece0bdbcc31bd2388c1228c7599aab8efc76e8d7a98b\": container with ID starting with a2e32a6311b2d9833f56ece0bdbcc31bd2388c1228c7599aab8efc76e8d7a98b not found: ID does not exist" containerID="a2e32a6311b2d9833f56ece0bdbcc31bd2388c1228c7599aab8efc76e8d7a98b"
Oct 30 22:59:17.866287 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:17.866165    5510 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a2e32a6311b2d9833f56ece0bdbcc31bd2388c1228c7599aab8efc76e8d7a98b"} err="failed to get container status \"a2e32a6311b2d9833f56ece0bdbcc31bd2388c1228c7599aab8efc76e8d7a98b\": rpc error: code = NotFound desc = could not find container \"a2e32a6311b2d9833f56ece0bdbcc31bd2388c1228c7599aab8efc76e8d7a98b\": container with ID starting with a2e32a6311b2d9833f56ece0bdbcc31bd2388c1228c7599aab8efc76e8d7a98b not found: ID does not exist"
Oct 30 22:59:17.936916 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:17.936879    5510 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2f4c97c6-6598-4189-b7ca-0244b2799bf5-istio-data\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:59:17.936916 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:17.936912    5510 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c6bn2\" (UniqueName: \"kubernetes.io/projected/2f4c97c6-6598-4189-b7ca-0244b2799bf5-kube-api-access-c6bn2\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:59:17.936916 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:17.936922    5510 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2f4c97c6-6598-4189-b7ca-0244b2799bf5-istiod-ca-cert\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:59:17.936916 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:17.936931    5510 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f4c97c6-6598-4189-b7ca-0244b2799bf5-workload-socket\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:59:17.937454 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:17.936942    5510 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2f4c97c6-6598-4189-b7ca-0244b2799bf5-istio-envoy\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:59:17.937454 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:17.936955    5510 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2f4c97c6-6598-4189-b7ca-0244b2799bf5-istio-podinfo\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:59:17.937454 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:17.936966    5510 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2f4c97c6-6598-4189-b7ca-0244b2799bf5-istio-token\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:59:18.250271 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:18.250227    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-f23d4bb3-openshift-default-865c59c95c-fzcbj"]
Oct 30 22:59:18.257758 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:18.257722    5510 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-f23d4bb3-openshift-default-865c59c95c-fzcbj"]
Oct 30 22:59:18.788043 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:18.788013    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:59:18.995606 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:18.995568    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:59:19.185552 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:19.185449    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:59:19.553348 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:19.553311    5510 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2f4c97c6-6598-4189-b7ca-0244b2799bf5" path="/var/lib/kubelet/pods/2f4c97c6-6598-4189-b7ca-0244b2799bf5/volumes"
Oct 30 22:59:22.100843 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:22.100806    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:59:22.291900 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:22.291857    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:59:22.357850 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:22.357758    5510 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-authcfg-http-31179c-6ccff9b9c5-flqbb"]
Oct 30 22:59:22.358282 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:22.358263    5510 cpu_manager.go:414] "containerMap: removing container" podUID="2f4c97c6-6598-4189-b7ca-0244b2799bf5" containerName="istio-proxy"
Oct 30 22:59:22.358376 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:22.358285    5510 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2f4c97c6-6598-4189-b7ca-0244b2799bf5" containerName="istio-proxy"
Oct 30 22:59:22.358461 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:22.358447    5510 memory_manager.go:365] "RemoveStaleState removing state" podUID="2f4c97c6-6598-4189-b7ca-0244b2799bf5" containerName="istio-proxy"
Oct 30 22:59:22.358524 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:22.358467    5510 memory_manager.go:365] "RemoveStaleState removing state" podUID="2f4c97c6-6598-4189-b7ca-0244b2799bf5" containerName="istio-proxy"
Oct 30 22:59:22.361975 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:22.361952    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-31179c-6ccff9b9c5-flqbb"
Oct 30 22:59:22.416226 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:22.416186    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-authcfg-http-31179c-6ccff9b9c5-flqbb"]
Oct 30 22:59:22.458037 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:22.458003    5510 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-authcfg-http-31179c\""
Oct 30 22:59:22.458246 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:22.458097    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-authcfg-http-31179c-dockercfg-rv5wn\""
Oct 30 22:59:22.486109 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:22.486076    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dm9xv\" (UniqueName: \"kubernetes.io/projected/a01d85f8-b761-4c55-aaa4-bae0fd901a65-kube-api-access-dm9xv\") pod \"payload-pre-processing-e2e-authcfg-http-31179c-6ccff9b9c5-flqbb\" (UID: \"a01d85f8-b761-4c55-aaa4-bae0fd901a65\") " pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-31179c-6ccff9b9c5-flqbb"
Oct 30 22:59:22.486282 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:22.486253    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a01d85f8-b761-4c55-aaa4-bae0fd901a65-plugins-config-volume\") pod \"payload-pre-processing-e2e-authcfg-http-31179c-6ccff9b9c5-flqbb\" (UID: \"a01d85f8-b761-4c55-aaa4-bae0fd901a65\") " pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-31179c-6ccff9b9c5-flqbb"
Oct 30 22:59:22.533382 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:22.533352    5510 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-authcfg-http-31179c-6667b5cb67-qrrpw"]
Oct 30 22:59:22.533837 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:22.533822    5510 cpu_manager.go:414] "containerMap: removing container" podUID="2f4c97c6-6598-4189-b7ca-0244b2799bf5" containerName="istio-proxy"
Oct 30 22:59:22.533896 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:22.533838    5510 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2f4c97c6-6598-4189-b7ca-0244b2799bf5" containerName="istio-proxy"
Oct 30 22:59:22.536279 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:22.536258    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-authcfg-http-31179c-6667b5cb67-qrrpw"
Oct 30 22:59:22.587808 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:22.587777    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a01d85f8-b761-4c55-aaa4-bae0fd901a65-plugins-config-volume\") pod \"payload-pre-processing-e2e-authcfg-http-31179c-6ccff9b9c5-flqbb\" (UID: \"a01d85f8-b761-4c55-aaa4-bae0fd901a65\") " pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-31179c-6ccff9b9c5-flqbb"
Oct 30 22:59:22.587992 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:22.587818    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dm9xv\" (UniqueName: \"kubernetes.io/projected/a01d85f8-b761-4c55-aaa4-bae0fd901a65-kube-api-access-dm9xv\") pod \"payload-pre-processing-e2e-authcfg-http-31179c-6ccff9b9c5-flqbb\" (UID: \"a01d85f8-b761-4c55-aaa4-bae0fd901a65\") " pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-31179c-6ccff9b9c5-flqbb"
Oct 30 22:59:22.588446 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:22.588425    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a01d85f8-b761-4c55-aaa4-bae0fd901a65-plugins-config-volume\") pod \"payload-pre-processing-e2e-authcfg-http-31179c-6ccff9b9c5-flqbb\" (UID: \"a01d85f8-b761-4c55-aaa4-bae0fd901a65\") " pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-31179c-6ccff9b9c5-flqbb"
Oct 30 22:59:22.594009 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:22.593982    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-authcfg-http-31179c-6667b5cb67-qrrpw"]
Oct 30 22:59:22.646124 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:22.646047    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dm9xv\" (UniqueName: \"kubernetes.io/projected/a01d85f8-b761-4c55-aaa4-bae0fd901a65-kube-api-access-dm9xv\") pod \"payload-pre-processing-e2e-authcfg-http-31179c-6ccff9b9c5-flqbb\" (UID: \"a01d85f8-b761-4c55-aaa4-bae0fd901a65\") " pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-31179c-6ccff9b9c5-flqbb"
Oct 30 22:59:22.675678 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:22.675622    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-31179c-6ccff9b9c5-flqbb"
Oct 30 22:59:22.688506 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:22.688477    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ms5cj\" (UniqueName: \"kubernetes.io/projected/8c127dae-831c-4a86-b68d-11a8091e8853-kube-api-access-ms5cj\") pod \"payload-processing-e2e-authcfg-http-31179c-6667b5cb67-qrrpw\" (UID: \"8c127dae-831c-4a86-b68d-11a8091e8853\") " pod="openshift-ingress/payload-processing-e2e-authcfg-http-31179c-6667b5cb67-qrrpw"
Oct 30 22:59:22.688762 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:22.688738    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8c127dae-831c-4a86-b68d-11a8091e8853-plugins-config-volume\") pod \"payload-processing-e2e-authcfg-http-31179c-6667b5cb67-qrrpw\" (UID: \"8c127dae-831c-4a86-b68d-11a8091e8853\") " pod="openshift-ingress/payload-processing-e2e-authcfg-http-31179c-6667b5cb67-qrrpw"
Oct 30 22:59:22.789902 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:22.789852    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8c127dae-831c-4a86-b68d-11a8091e8853-plugins-config-volume\") pod \"payload-processing-e2e-authcfg-http-31179c-6667b5cb67-qrrpw\" (UID: \"8c127dae-831c-4a86-b68d-11a8091e8853\") " pod="openshift-ingress/payload-processing-e2e-authcfg-http-31179c-6667b5cb67-qrrpw"
Oct 30 22:59:22.790122 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:22.789934    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ms5cj\" (UniqueName: \"kubernetes.io/projected/8c127dae-831c-4a86-b68d-11a8091e8853-kube-api-access-ms5cj\") pod \"payload-processing-e2e-authcfg-http-31179c-6667b5cb67-qrrpw\" (UID: \"8c127dae-831c-4a86-b68d-11a8091e8853\") " pod="openshift-ingress/payload-processing-e2e-authcfg-http-31179c-6667b5cb67-qrrpw"
Oct 30 22:59:22.790535 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:22.790514    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8c127dae-831c-4a86-b68d-11a8091e8853-plugins-config-volume\") pod \"payload-processing-e2e-authcfg-http-31179c-6667b5cb67-qrrpw\" (UID: \"8c127dae-831c-4a86-b68d-11a8091e8853\") " pod="openshift-ingress/payload-processing-e2e-authcfg-http-31179c-6667b5cb67-qrrpw"
Oct 30 22:59:22.861276 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:22.861247    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ms5cj\" (UniqueName: \"kubernetes.io/projected/8c127dae-831c-4a86-b68d-11a8091e8853-kube-api-access-ms5cj\") pod \"payload-processing-e2e-authcfg-http-31179c-6667b5cb67-qrrpw\" (UID: \"8c127dae-831c-4a86-b68d-11a8091e8853\") " pod="openshift-ingress/payload-processing-e2e-authcfg-http-31179c-6667b5cb67-qrrpw"
Oct 30 22:59:23.046592 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:23.046556    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-authcfg-http-31179c-6ccff9b9c5-flqbb"]
Oct 30 22:59:23.054156 ip-10-0-136-210 kubenswrapper[5510]: W1030 22:59:23.054124    5510 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda01d85f8_b761_4c55_aaa4_bae0fd901a65.slice/crio-12bacb66b20133c7f2dcebcdce76530b5d88464fb891e4e2a27606814739d401 WatchSource:0}: Error finding container 12bacb66b20133c7f2dcebcdce76530b5d88464fb891e4e2a27606814739d401: Status 404 returned error can't find the container with id 12bacb66b20133c7f2dcebcdce76530b5d88464fb891e4e2a27606814739d401
Oct 30 22:59:23.148411 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:23.148369    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-authcfg-http-31179c-6667b5cb67-qrrpw"
Oct 30 22:59:23.520122 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:23.520079    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-authcfg-http-31179c-6667b5cb67-qrrpw"]
Oct 30 22:59:23.528097 ip-10-0-136-210 kubenswrapper[5510]: W1030 22:59:23.528071    5510 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c127dae_831c_4a86_b68d_11a8091e8853.slice/crio-388cba2b016b068fc2f6d3cf8d51548ae9be91a1da94981be62dc61612dcf5ba WatchSource:0}: Error finding container 388cba2b016b068fc2f6d3cf8d51548ae9be91a1da94981be62dc61612dcf5ba: Status 404 returned error can't find the container with id 388cba2b016b068fc2f6d3cf8d51548ae9be91a1da94981be62dc61612dcf5ba
Oct 30 22:59:23.867014 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:23.866906    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-31179c-6ccff9b9c5-flqbb" event={"ID":"a01d85f8-b761-4c55-aaa4-bae0fd901a65","Type":"ContainerStarted","Data":"0c4fdb1421ffb9966edeab536b9338625f7431fb4a127636ac6cf044da7d06bd"}
Oct 30 22:59:23.867014 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:23.866976    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-31179c-6ccff9b9c5-flqbb"
Oct 30 22:59:23.867014 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:23.866997    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-31179c-6ccff9b9c5-flqbb" event={"ID":"a01d85f8-b761-4c55-aaa4-bae0fd901a65","Type":"ContainerStarted","Data":"12bacb66b20133c7f2dcebcdce76530b5d88464fb891e4e2a27606814739d401"}
Oct 30 22:59:23.868466 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:23.868438    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-authcfg-http-31179c-6667b5cb67-qrrpw" event={"ID":"8c127dae-831c-4a86-b68d-11a8091e8853","Type":"ContainerStarted","Data":"e43152a1a70df03ce4453a03183504fcc0bb8c9daf6dc6745db0a16eceafca9a"}
Oct 30 22:59:23.868598 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:23.868481    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-authcfg-http-31179c-6667b5cb67-qrrpw" event={"ID":"8c127dae-831c-4a86-b68d-11a8091e8853","Type":"ContainerStarted","Data":"388cba2b016b068fc2f6d3cf8d51548ae9be91a1da94981be62dc61612dcf5ba"}
Oct 30 22:59:23.868598 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:23.868502    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-authcfg-http-31179c-6667b5cb67-qrrpw"
Oct 30 22:59:23.977910 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:23.977855    5510 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-31179c-6ccff9b9c5-flqbb" podStartSLOduration=1.97784117 podStartE2EDuration="1.97784117s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:59:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:59:23.97622882 +0000 UTC m=+4144.537038836" watchObservedRunningTime="2026-10-30 22:59:23.97784117 +0000 UTC m=+4144.538651175"
Oct 30 22:59:24.087864 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:24.087797    5510 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-authcfg-http-31179c-6667b5cb67-qrrpw" podStartSLOduration=2.087784717 podStartE2EDuration="2.087784717s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:59:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:59:24.0861486 +0000 UTC m=+4144.646958607" watchObservedRunningTime="2026-10-30 22:59:24.087784717 +0000 UTC m=+4144.648594722"
Oct 30 22:59:25.022670 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:25.022628    5510 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-authcfg-http-31179c-58f6548f68-x75x2"]
Oct 30 22:59:25.026425 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:25.026401    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-31179c-58f6548f68-x75x2"
Oct 30 22:59:25.075814 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:25.075779    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-authcfg-http-31179c-serving-cert\""
Oct 30 22:59:25.123929 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:25.123895    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-authcfg-http-31179c-58f6548f68-x75x2"]
Oct 30 22:59:25.216499 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:25.216462    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jjhc5\" (UniqueName: \"kubernetes.io/projected/f9f2defa-179f-4fd6-a9ad-ae7588c1a4da-kube-api-access-jjhc5\") pod \"maas-api-e2e-authcfg-http-31179c-58f6548f68-x75x2\" (UID: \"f9f2defa-179f-4fd6-a9ad-ae7588c1a4da\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-31179c-58f6548f68-x75x2"
Oct 30 22:59:25.216499 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:25.216504    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f9f2defa-179f-4fd6-a9ad-ae7588c1a4da-maas-api-tls\") pod \"maas-api-e2e-authcfg-http-31179c-58f6548f68-x75x2\" (UID: \"f9f2defa-179f-4fd6-a9ad-ae7588c1a4da\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-31179c-58f6548f68-x75x2"
Oct 30 22:59:25.216736 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:25.216594    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f9f2defa-179f-4fd6-a9ad-ae7588c1a4da-metrics-certs\") pod \"maas-api-e2e-authcfg-http-31179c-58f6548f68-x75x2\" (UID: \"f9f2defa-179f-4fd6-a9ad-ae7588c1a4da\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-31179c-58f6548f68-x75x2"
Oct 30 22:59:25.216736 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:25.216708    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f9f2defa-179f-4fd6-a9ad-ae7588c1a4da-openshift-service-ca\") pod \"maas-api-e2e-authcfg-http-31179c-58f6548f68-x75x2\" (UID: \"f9f2defa-179f-4fd6-a9ad-ae7588c1a4da\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-31179c-58f6548f68-x75x2"
Oct 30 22:59:25.268727 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:25.268650    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:59:25.318136 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:25.318034    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f9f2defa-179f-4fd6-a9ad-ae7588c1a4da-openshift-service-ca\") pod \"maas-api-e2e-authcfg-http-31179c-58f6548f68-x75x2\" (UID: \"f9f2defa-179f-4fd6-a9ad-ae7588c1a4da\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-31179c-58f6548f68-x75x2"
Oct 30 22:59:25.318333 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:25.318175    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jjhc5\" (UniqueName: \"kubernetes.io/projected/f9f2defa-179f-4fd6-a9ad-ae7588c1a4da-kube-api-access-jjhc5\") pod \"maas-api-e2e-authcfg-http-31179c-58f6548f68-x75x2\" (UID: \"f9f2defa-179f-4fd6-a9ad-ae7588c1a4da\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-31179c-58f6548f68-x75x2"
Oct 30 22:59:25.318333 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:25.318210    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f9f2defa-179f-4fd6-a9ad-ae7588c1a4da-maas-api-tls\") pod \"maas-api-e2e-authcfg-http-31179c-58f6548f68-x75x2\" (UID: \"f9f2defa-179f-4fd6-a9ad-ae7588c1a4da\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-31179c-58f6548f68-x75x2"
Oct 30 22:59:25.318333 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:25.318266    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f9f2defa-179f-4fd6-a9ad-ae7588c1a4da-metrics-certs\") pod \"maas-api-e2e-authcfg-http-31179c-58f6548f68-x75x2\" (UID: \"f9f2defa-179f-4fd6-a9ad-ae7588c1a4da\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-31179c-58f6548f68-x75x2"
Oct 30 22:59:25.318508 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:59:25.318420    5510 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-authcfg-http-31179c-serving-cert: secret "maas-api-e2e-authcfg-http-31179c-serving-cert" not found
Oct 30 22:59:25.318508 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:59:25.318495    5510 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f9f2defa-179f-4fd6-a9ad-ae7588c1a4da-maas-api-tls podName:f9f2defa-179f-4fd6-a9ad-ae7588c1a4da nodeName:}" failed. No retries permitted until 2026-10-30 22:59:25.818472408 +0000 UTC m=+4146.379282403 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/f9f2defa-179f-4fd6-a9ad-ae7588c1a4da-maas-api-tls") pod "maas-api-e2e-authcfg-http-31179c-58f6548f68-x75x2" (UID: "f9f2defa-179f-4fd6-a9ad-ae7588c1a4da") : secret "maas-api-e2e-authcfg-http-31179c-serving-cert" not found
Oct 30 22:59:25.318804 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:25.318783    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f9f2defa-179f-4fd6-a9ad-ae7588c1a4da-openshift-service-ca\") pod \"maas-api-e2e-authcfg-http-31179c-58f6548f68-x75x2\" (UID: \"f9f2defa-179f-4fd6-a9ad-ae7588c1a4da\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-31179c-58f6548f68-x75x2"
Oct 30 22:59:25.320887 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:25.320862    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f9f2defa-179f-4fd6-a9ad-ae7588c1a4da-metrics-certs\") pod \"maas-api-e2e-authcfg-http-31179c-58f6548f68-x75x2\" (UID: \"f9f2defa-179f-4fd6-a9ad-ae7588c1a4da\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-31179c-58f6548f68-x75x2"
Oct 30 22:59:25.376710 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:25.376680    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jjhc5\" (UniqueName: \"kubernetes.io/projected/f9f2defa-179f-4fd6-a9ad-ae7588c1a4da-kube-api-access-jjhc5\") pod \"maas-api-e2e-authcfg-http-31179c-58f6548f68-x75x2\" (UID: \"f9f2defa-179f-4fd6-a9ad-ae7588c1a4da\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-31179c-58f6548f68-x75x2"
Oct 30 22:59:25.460086 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:25.460049    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:59:25.649330 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:25.649244    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:59:25.823307 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:25.823268    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f9f2defa-179f-4fd6-a9ad-ae7588c1a4da-maas-api-tls\") pod \"maas-api-e2e-authcfg-http-31179c-58f6548f68-x75x2\" (UID: \"f9f2defa-179f-4fd6-a9ad-ae7588c1a4da\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-31179c-58f6548f68-x75x2"
Oct 30 22:59:25.825680 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:25.825644    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f9f2defa-179f-4fd6-a9ad-ae7588c1a4da-maas-api-tls\") pod \"maas-api-e2e-authcfg-http-31179c-58f6548f68-x75x2\" (UID: \"f9f2defa-179f-4fd6-a9ad-ae7588c1a4da\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-31179c-58f6548f68-x75x2"
Oct 30 22:59:25.938794 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:25.938718    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-31179c-58f6548f68-x75x2"
Oct 30 22:59:26.310436 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:26.310394    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-authcfg-http-31179c-58f6548f68-x75x2"]
Oct 30 22:59:26.317283 ip-10-0-136-210 kubenswrapper[5510]: W1030 22:59:26.317255    5510 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf9f2defa_179f_4fd6_a9ad_ae7588c1a4da.slice/crio-d5bbb93d27adbb819dea6170b2a9af7128a18e0ff587bc132c5413bc4acc00fd WatchSource:0}: Error finding container d5bbb93d27adbb819dea6170b2a9af7128a18e0ff587bc132c5413bc4acc00fd: Status 404 returned error can't find the container with id d5bbb93d27adbb819dea6170b2a9af7128a18e0ff587bc132c5413bc4acc00fd
Oct 30 22:59:26.884429 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:26.884391    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-31179c-58f6548f68-x75x2" event={"ID":"f9f2defa-179f-4fd6-a9ad-ae7588c1a4da","Type":"ContainerStarted","Data":"d5bbb93d27adbb819dea6170b2a9af7128a18e0ff587bc132c5413bc4acc00fd"}
Oct 30 22:59:27.890857 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:27.890820    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-31179c-58f6548f68-x75x2" event={"ID":"f9f2defa-179f-4fd6-a9ad-ae7588c1a4da","Type":"ContainerStarted","Data":"f819ef06563e538fa23326f93bd88d8a0d4734c8b0e0a22cec6511553351b778"}
Oct 30 22:59:27.891248 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:27.890945    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-31179c-58f6548f68-x75x2"
Oct 30 22:59:28.003205 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:28.003148    5510 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-31179c-58f6548f68-x75x2" podStartSLOduration=3.154363271 podStartE2EDuration="4.003133033s" totalImagesPullingTime="848.769762ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:59:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:59:28.002235089 +0000 UTC m=+4148.563045095" watchObservedRunningTime="2026-10-30 22:59:28.003133033 +0000 UTC m=+4148.563943035"
Oct 30 22:59:30.286597 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:30.286501    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:59:33.909208 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:33.909177    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-31179c-58f6548f68-x75x2"
Oct 30 22:59:34.876189 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:34.876156    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-31179c-6ccff9b9c5-flqbb"
Oct 30 22:59:34.876366 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:34.876229    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-authcfg-http-31179c-6667b5cb67-qrrpw"
Oct 30 22:59:35.426968 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:35.426890    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:59:38.419678 ip-10-0-136-210 kubenswrapper[5510]: E1030 22:59:38.419628    5510 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6gjbf" containerName="extractor"
Oct 30 22:59:40.430625 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:40.430588    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:59:41.919609 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:41.919568    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:59:42.997228 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:42.997197    5510 generic.go:356] "Generic (PLEG): container finished" podID="fb6d6c9f-c79a-4cf8-b896-3cad3c60cbdf" containerID="d78363b65c03f13953e3d76f977eda8123bc01479e43998cd110f0a87c80411a" exitCode=0
Oct 30 22:59:42.997764 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:42.997267    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-authcfg-http-31179c-openshift-default-58959bdf7-x52mb" event={"ID":"fb6d6c9f-c79a-4cf8-b896-3cad3c60cbdf","Type":"ContainerDied","Data":"d78363b65c03f13953e3d76f977eda8123bc01479e43998cd110f0a87c80411a"}
Oct 30 22:59:42.997823 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:42.997784    5510 scope.go:122] "RemoveContainer" containerID="d78363b65c03f13953e3d76f977eda8123bc01479e43998cd110f0a87c80411a"
Oct 30 22:59:42.998976 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:42.998949    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:59:42.999054 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:42.999040    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:59:42.999094 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:42.999075    5510 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:59:43.570219 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:43.570168    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:59:44.004226 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:44.004131    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-authcfg-http-31179c-openshift-default-58959bdf7-x52mb" event={"ID":"fb6d6c9f-c79a-4cf8-b896-3cad3c60cbdf","Type":"ContainerStarted","Data":"b078321501fc3745db0bb96e45554ddfa4eeb409c1165527523ed8d468ac2971"}
Oct 30 22:59:44.885880 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:44.885848    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-authcfg-http-31179c-openshift-default-58959bdf7-x52mb"
Oct 30 22:59:44.887258 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:44.887221    5510 patch_prober.go:29] interesting pod/e2e-authcfg-http-31179c-openshift-default-58959bdf7-x52mb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.171:15021/healthz/ready\": dial tcp 10.133.0.171:15021: connect: connection refused" start-of-body=
Oct 30 22:59:44.887405 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:44.887271    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-authcfg-http-31179c-openshift-default-58959bdf7-x52mb" podUID="fb6d6c9f-c79a-4cf8-b896-3cad3c60cbdf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.171:15021/healthz/ready\": dial tcp 10.133.0.171:15021: connect: connection refused"
Oct 30 22:59:45.885930 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:45.885886    5510 patch_prober.go:29] interesting pod/e2e-authcfg-http-31179c-openshift-default-58959bdf7-x52mb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.171:15021/healthz/ready\": dial tcp 10.133.0.171:15021: connect: connection refused" start-of-body=
Oct 30 22:59:45.886456 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:45.885946    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-authcfg-http-31179c-openshift-default-58959bdf7-x52mb" podUID="fb6d6c9f-c79a-4cf8-b896-3cad3c60cbdf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.171:15021/healthz/ready\": dial tcp 10.133.0.171:15021: connect: connection refused"
Oct 30 22:59:45.892755 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:45.890063    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:59:46.886358 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:46.886314    5510 patch_prober.go:29] interesting pod/e2e-authcfg-http-31179c-openshift-default-58959bdf7-x52mb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.171:15021/healthz/ready\": dial tcp 10.133.0.171:15021: connect: connection refused" start-of-body=
Oct 30 22:59:46.886905 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:46.886381    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-authcfg-http-31179c-openshift-default-58959bdf7-x52mb" podUID="fb6d6c9f-c79a-4cf8-b896-3cad3c60cbdf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.171:15021/healthz/ready\": dial tcp 10.133.0.171:15021: connect: connection refused"
Oct 30 22:59:47.834786 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:47.834747    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-authcfg-http-31179c-openshift-default-58959bdf7-x52mb"
Oct 30 22:59:47.886286 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:47.886248    5510 patch_prober.go:29] interesting pod/e2e-authcfg-http-31179c-openshift-default-58959bdf7-x52mb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.171:15021/healthz/ready\": dial tcp 10.133.0.171:15021: connect: connection refused" start-of-body=
Oct 30 22:59:47.886464 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:47.886318    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-authcfg-http-31179c-openshift-default-58959bdf7-x52mb" podUID="fb6d6c9f-c79a-4cf8-b896-3cad3c60cbdf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.171:15021/healthz/ready\": dial tcp 10.133.0.171:15021: connect: connection refused"
Oct 30 22:59:48.886107 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:48.886058    5510 patch_prober.go:29] interesting pod/e2e-authcfg-http-31179c-openshift-default-58959bdf7-x52mb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.171:15021/healthz/ready\": dial tcp 10.133.0.171:15021: connect: connection refused" start-of-body=
Oct 30 22:59:48.886341 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:48.886132    5510 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-authcfg-http-31179c-openshift-default-58959bdf7-x52mb" podUID="fb6d6c9f-c79a-4cf8-b896-3cad3c60cbdf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.171:15021/healthz/ready\": dial tcp 10.133.0.171:15021: connect: connection refused"
Oct 30 22:59:49.890750 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:49.890712    5510 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-authcfg-http-31179c-openshift-default-58959bdf7-x52mb"
Oct 30 22:59:49.891742 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:49.891724    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-authcfg-http-31179c-openshift-default-58959bdf7-x52mb"
Oct 30 22:59:50.816100 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:50.816062    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 22:59:55.809443 ip-10-0-136-210 kubenswrapper[5510]: I1030 22:59:55.809405    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 23:00:00.713585 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:00:00.713544    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 23:00:05.610135 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:00:05.610091    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 23:00:08.419746 ip-10-0-136-210 kubenswrapper[5510]: E1030 23:00:08.419709    5510 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6gjbf" containerName="extractor"
Oct 30 23:00:10.507573 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:00:10.507525    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 23:00:15.419997 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:00:15.419930    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 23:00:20.306347 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:00:20.306305    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 23:00:25.224734 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:00:25.224696    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 23:00:30.122341 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:00:30.122300    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 23:00:35.098125 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:00:35.098002    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 23:00:38.418955 ip-10-0-136-210 kubenswrapper[5510]: E1030 23:00:38.418921    5510 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6gjbf" containerName="extractor"
Oct 30 23:00:40.017867 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:00:40.017824    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 23:00:42.335075 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:00:42.335035    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 23:00:42.577385 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:00:42.577345    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 23:00:45.030011 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:00:45.029973    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 23:00:46.286113 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:00:46.286074    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 23:00:49.815480 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:00:49.815437    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 23:00:54.719527 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:00:54.719491    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 23:00:59.615363 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:00:59.615310    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 23:01:04.654543 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:01:04.654483    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 23:01:08.422935 ip-10-0-136-210 kubenswrapper[5510]: E1030 23:01:08.422866    5510 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6gjbf" containerName="extractor"
Oct 30 23:01:09.936513 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:01:09.935991    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 23:01:14.626553 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:01:14.626515    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-authcfg-http-31179c-6667b5cb67-qrrpw"]
Oct 30 23:01:14.724526 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:01:14.724493    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-authcfg-http-31179c-6ccff9b9c5-flqbb"]
Oct 30 23:01:14.931650 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:01:14.931558    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 23:01:15.166452 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:01:15.166414    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 23:01:19.627243 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:01:19.627202    5510 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-authcfg-http-31179c-6667b5cb67-qrrpw" podUID="8c127dae-831c-4a86-b68d-11a8091e8853" containerName="payload-processing" containerID="cri-o://e43152a1a70df03ce4453a03183504fcc0bb8c9daf6dc6745db0a16eceafca9a" gracePeriod=25
Oct 30 23:01:19.725269 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:01:19.725204    5510 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-31179c-6ccff9b9c5-flqbb" podUID="a01d85f8-b761-4c55-aaa4-bae0fd901a65" containerName="payload-pre-processing" containerID="cri-o://0c4fdb1421ffb9966edeab536b9338625f7431fb4a127636ac6cf044da7d06bd" gracePeriod=25
Oct 30 23:01:19.763962 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:01:19.763917    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 23:01:19.883304 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:01:19.883233    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-authcfg-http-31179c-6667b5cb67-qrrpw"
Oct 30 23:01:19.925248 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:01:19.925216    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8c127dae-831c-4a86-b68d-11a8091e8853-kube-api-access-ms5cj\" (UniqueName: \"kubernetes.io/projected/8c127dae-831c-4a86-b68d-11a8091e8853-kube-api-access-ms5cj\") pod \"8c127dae-831c-4a86-b68d-11a8091e8853\" (UID: \"8c127dae-831c-4a86-b68d-11a8091e8853\") "
Oct 30 23:01:19.925458 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:01:19.925320    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8c127dae-831c-4a86-b68d-11a8091e8853-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8c127dae-831c-4a86-b68d-11a8091e8853-plugins-config-volume\") pod \"8c127dae-831c-4a86-b68d-11a8091e8853\" (UID: \"8c127dae-831c-4a86-b68d-11a8091e8853\") "
Oct 30 23:01:19.925784 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:01:19.925761    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8c127dae-831c-4a86-b68d-11a8091e8853-plugins-config-volume" pod "8c127dae-831c-4a86-b68d-11a8091e8853" (UID: "8c127dae-831c-4a86-b68d-11a8091e8853"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 23:01:19.927461 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:01:19.927436    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8c127dae-831c-4a86-b68d-11a8091e8853-kube-api-access-ms5cj" pod "8c127dae-831c-4a86-b68d-11a8091e8853" (UID: "8c127dae-831c-4a86-b68d-11a8091e8853"). InnerVolumeSpecName "kube-api-access-ms5cj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 23:01:19.987572 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:01:19.987542    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-31179c-6ccff9b9c5-flqbb"
Oct 30 23:01:20.026066 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:01:20.026028    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a01d85f8-b761-4c55-aaa4-bae0fd901a65-kube-api-access-dm9xv\" (UniqueName: \"kubernetes.io/projected/a01d85f8-b761-4c55-aaa4-bae0fd901a65-kube-api-access-dm9xv\") pod \"a01d85f8-b761-4c55-aaa4-bae0fd901a65\" (UID: \"a01d85f8-b761-4c55-aaa4-bae0fd901a65\") "
Oct 30 23:01:20.026285 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:01:20.026137    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a01d85f8-b761-4c55-aaa4-bae0fd901a65-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a01d85f8-b761-4c55-aaa4-bae0fd901a65-plugins-config-volume\") pod \"a01d85f8-b761-4c55-aaa4-bae0fd901a65\" (UID: \"a01d85f8-b761-4c55-aaa4-bae0fd901a65\") "
Oct 30 23:01:20.026517 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:01:20.026492    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a01d85f8-b761-4c55-aaa4-bae0fd901a65-plugins-config-volume" pod "a01d85f8-b761-4c55-aaa4-bae0fd901a65" (UID: "a01d85f8-b761-4c55-aaa4-bae0fd901a65"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 23:01:20.026591 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:01:20.026561    5510 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ms5cj\" (UniqueName: \"kubernetes.io/projected/8c127dae-831c-4a86-b68d-11a8091e8853-kube-api-access-ms5cj\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 23:01:20.026591 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:01:20.026585    5510 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8c127dae-831c-4a86-b68d-11a8091e8853-plugins-config-volume\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 23:01:20.028460 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:01:20.028434    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a01d85f8-b761-4c55-aaa4-bae0fd901a65-kube-api-access-dm9xv" pod "a01d85f8-b761-4c55-aaa4-bae0fd901a65" (UID: "a01d85f8-b761-4c55-aaa4-bae0fd901a65"). InnerVolumeSpecName "kube-api-access-dm9xv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 23:01:20.127453 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:01:20.127414    5510 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dm9xv\" (UniqueName: \"kubernetes.io/projected/a01d85f8-b761-4c55-aaa4-bae0fd901a65-kube-api-access-dm9xv\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 23:01:20.127453 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:01:20.127446    5510 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a01d85f8-b761-4c55-aaa4-bae0fd901a65-plugins-config-volume\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 23:01:20.507125 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:01:20.507086    5510 generic.go:356] "Generic (PLEG): container finished" podID="a01d85f8-b761-4c55-aaa4-bae0fd901a65" containerID="0c4fdb1421ffb9966edeab536b9338625f7431fb4a127636ac6cf044da7d06bd" exitCode=0
Oct 30 23:01:20.507350 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:01:20.507144    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-31179c-6ccff9b9c5-flqbb"
Oct 30 23:01:20.507350 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:01:20.507161    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-31179c-6ccff9b9c5-flqbb" event={"ID":"a01d85f8-b761-4c55-aaa4-bae0fd901a65","Type":"ContainerDied","Data":"0c4fdb1421ffb9966edeab536b9338625f7431fb4a127636ac6cf044da7d06bd"}
Oct 30 23:01:20.507350 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:01:20.507245    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-31179c-6ccff9b9c5-flqbb" event={"ID":"a01d85f8-b761-4c55-aaa4-bae0fd901a65","Type":"ContainerDied","Data":"12bacb66b20133c7f2dcebcdce76530b5d88464fb891e4e2a27606814739d401"}
Oct 30 23:01:20.507350 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:01:20.507267    5510 scope.go:122] "RemoveContainer" containerID="0c4fdb1421ffb9966edeab536b9338625f7431fb4a127636ac6cf044da7d06bd"
Oct 30 23:01:20.508877 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:01:20.508851    5510 generic.go:356] "Generic (PLEG): container finished" podID="8c127dae-831c-4a86-b68d-11a8091e8853" containerID="e43152a1a70df03ce4453a03183504fcc0bb8c9daf6dc6745db0a16eceafca9a" exitCode=0
Oct 30 23:01:20.509074 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:01:20.508908    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-authcfg-http-31179c-6667b5cb67-qrrpw"
Oct 30 23:01:20.509167 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:01:20.508926    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-authcfg-http-31179c-6667b5cb67-qrrpw" event={"ID":"8c127dae-831c-4a86-b68d-11a8091e8853","Type":"ContainerDied","Data":"e43152a1a70df03ce4453a03183504fcc0bb8c9daf6dc6745db0a16eceafca9a"}
Oct 30 23:01:20.509296 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:01:20.509207    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-authcfg-http-31179c-6667b5cb67-qrrpw" event={"ID":"8c127dae-831c-4a86-b68d-11a8091e8853","Type":"ContainerDied","Data":"388cba2b016b068fc2f6d3cf8d51548ae9be91a1da94981be62dc61612dcf5ba"}
Oct 30 23:01:20.517880 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:01:20.517860    5510 scope.go:122] "RemoveContainer" containerID="0c4fdb1421ffb9966edeab536b9338625f7431fb4a127636ac6cf044da7d06bd"
Oct 30 23:01:20.518222 ip-10-0-136-210 kubenswrapper[5510]: E1030 23:01:20.518193    5510 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0c4fdb1421ffb9966edeab536b9338625f7431fb4a127636ac6cf044da7d06bd\": container with ID starting with 0c4fdb1421ffb9966edeab536b9338625f7431fb4a127636ac6cf044da7d06bd not found: ID does not exist" containerID="0c4fdb1421ffb9966edeab536b9338625f7431fb4a127636ac6cf044da7d06bd"
Oct 30 23:01:20.518363 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:01:20.518232    5510 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0c4fdb1421ffb9966edeab536b9338625f7431fb4a127636ac6cf044da7d06bd"} err="failed to get container status \"0c4fdb1421ffb9966edeab536b9338625f7431fb4a127636ac6cf044da7d06bd\": rpc error: code = NotFound desc = could not find container \"0c4fdb1421ffb9966edeab536b9338625f7431fb4a127636ac6cf044da7d06bd\": container with ID starting with 0c4fdb1421ffb9966edeab536b9338625f7431fb4a127636ac6cf044da7d06bd not found: ID does not exist"
Oct 30 23:01:20.518363 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:01:20.518254    5510 scope.go:122] "RemoveContainer" containerID="e43152a1a70df03ce4453a03183504fcc0bb8c9daf6dc6745db0a16eceafca9a"
Oct 30 23:01:20.526669 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:01:20.526637    5510 scope.go:122] "RemoveContainer" containerID="e43152a1a70df03ce4453a03183504fcc0bb8c9daf6dc6745db0a16eceafca9a"
Oct 30 23:01:20.527343 ip-10-0-136-210 kubenswrapper[5510]: E1030 23:01:20.527324    5510 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e43152a1a70df03ce4453a03183504fcc0bb8c9daf6dc6745db0a16eceafca9a\": container with ID starting with e43152a1a70df03ce4453a03183504fcc0bb8c9daf6dc6745db0a16eceafca9a not found: ID does not exist" containerID="e43152a1a70df03ce4453a03183504fcc0bb8c9daf6dc6745db0a16eceafca9a"
Oct 30 23:01:20.527438 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:01:20.527350    5510 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e43152a1a70df03ce4453a03183504fcc0bb8c9daf6dc6745db0a16eceafca9a"} err="failed to get container status \"e43152a1a70df03ce4453a03183504fcc0bb8c9daf6dc6745db0a16eceafca9a\": rpc error: code = NotFound desc = could not find container \"e43152a1a70df03ce4453a03183504fcc0bb8c9daf6dc6745db0a16eceafca9a\": container with ID starting with e43152a1a70df03ce4453a03183504fcc0bb8c9daf6dc6745db0a16eceafca9a not found: ID does not exist"
Oct 30 23:01:20.661085 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:01:20.661053    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-authcfg-http-31179c-6ccff9b9c5-flqbb"]
Oct 30 23:01:20.707837 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:01:20.707800    5510 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-authcfg-http-31179c-6ccff9b9c5-flqbb"]
Oct 30 23:01:20.770390 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:01:20.770362    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-authcfg-http-31179c-6667b5cb67-qrrpw"]
Oct 30 23:01:20.774303 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:01:20.774280    5510 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-authcfg-http-31179c-6667b5cb67-qrrpw"]
Oct 30 23:01:21.551563 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:01:21.551530    5510 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8c127dae-831c-4a86-b68d-11a8091e8853" path="/var/lib/kubelet/pods/8c127dae-831c-4a86-b68d-11a8091e8853/volumes"
Oct 30 23:01:21.551955 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:01:21.551941    5510 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a01d85f8-b761-4c55-aaa4-bae0fd901a65" path="/var/lib/kubelet/pods/a01d85f8-b761-4c55-aaa4-bae0fd901a65/volumes"
Oct 30 23:01:29.019114 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:01:29.019075    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-authcfg-http-31179c-58f6548f68-x75x2"]
Oct 30 23:01:29.019608 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:01:29.019322    5510 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-31179c-58f6548f68-x75x2" podUID="f9f2defa-179f-4fd6-a9ad-ae7588c1a4da" containerName="maas-api" containerID="cri-o://f819ef06563e538fa23326f93bd88d8a0d4734c8b0e0a22cec6511553351b778" gracePeriod=30
Oct 30 23:01:29.173240 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:01:29.173203    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 23:01:29.275870 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:01:29.275808    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-31179c-58f6548f68-x75x2"
Oct 30 23:01:29.427461 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:01:29.427431    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f9f2defa-179f-4fd6-a9ad-ae7588c1a4da-metrics-certs\" (UniqueName: \"kubernetes.io/secret/f9f2defa-179f-4fd6-a9ad-ae7588c1a4da-metrics-certs\") pod \"f9f2defa-179f-4fd6-a9ad-ae7588c1a4da\" (UID: \"f9f2defa-179f-4fd6-a9ad-ae7588c1a4da\") "
Oct 30 23:01:29.427634 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:01:29.427471    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f9f2defa-179f-4fd6-a9ad-ae7588c1a4da-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f9f2defa-179f-4fd6-a9ad-ae7588c1a4da-maas-api-tls\") pod \"f9f2defa-179f-4fd6-a9ad-ae7588c1a4da\" (UID: \"f9f2defa-179f-4fd6-a9ad-ae7588c1a4da\") "
Oct 30 23:01:29.427634 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:01:29.427504    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f9f2defa-179f-4fd6-a9ad-ae7588c1a4da-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f9f2defa-179f-4fd6-a9ad-ae7588c1a4da-openshift-service-ca\") pod \"f9f2defa-179f-4fd6-a9ad-ae7588c1a4da\" (UID: \"f9f2defa-179f-4fd6-a9ad-ae7588c1a4da\") "
Oct 30 23:01:29.427634 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:01:29.427567    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f9f2defa-179f-4fd6-a9ad-ae7588c1a4da-kube-api-access-jjhc5\" (UniqueName: \"kubernetes.io/projected/f9f2defa-179f-4fd6-a9ad-ae7588c1a4da-kube-api-access-jjhc5\") pod \"f9f2defa-179f-4fd6-a9ad-ae7588c1a4da\" (UID: \"f9f2defa-179f-4fd6-a9ad-ae7588c1a4da\") "
Oct 30 23:01:29.427917 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:01:29.427890    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f9f2defa-179f-4fd6-a9ad-ae7588c1a4da-openshift-service-ca" pod "f9f2defa-179f-4fd6-a9ad-ae7588c1a4da" (UID: "f9f2defa-179f-4fd6-a9ad-ae7588c1a4da"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 23:01:29.429868 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:01:29.429841    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f9f2defa-179f-4fd6-a9ad-ae7588c1a4da-kube-api-access-jjhc5" pod "f9f2defa-179f-4fd6-a9ad-ae7588c1a4da" (UID: "f9f2defa-179f-4fd6-a9ad-ae7588c1a4da"). InnerVolumeSpecName "kube-api-access-jjhc5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 23:01:29.430000 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:01:29.429873    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f9f2defa-179f-4fd6-a9ad-ae7588c1a4da-maas-api-tls" pod "f9f2defa-179f-4fd6-a9ad-ae7588c1a4da" (UID: "f9f2defa-179f-4fd6-a9ad-ae7588c1a4da"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 23:01:29.430000 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:01:29.429895    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f9f2defa-179f-4fd6-a9ad-ae7588c1a4da-metrics-certs" pod "f9f2defa-179f-4fd6-a9ad-ae7588c1a4da" (UID: "f9f2defa-179f-4fd6-a9ad-ae7588c1a4da"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 23:01:29.529304 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:01:29.529212    5510 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f9f2defa-179f-4fd6-a9ad-ae7588c1a4da-metrics-certs\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 23:01:29.529304 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:01:29.529247    5510 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f9f2defa-179f-4fd6-a9ad-ae7588c1a4da-maas-api-tls\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 23:01:29.529304 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:01:29.529259    5510 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f9f2defa-179f-4fd6-a9ad-ae7588c1a4da-openshift-service-ca\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 23:01:29.529304 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:01:29.529270    5510 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jjhc5\" (UniqueName: \"kubernetes.io/projected/f9f2defa-179f-4fd6-a9ad-ae7588c1a4da-kube-api-access-jjhc5\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 23:01:29.561363 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:01:29.561327    5510 generic.go:356] "Generic (PLEG): container finished" podID="f9f2defa-179f-4fd6-a9ad-ae7588c1a4da" containerID="f819ef06563e538fa23326f93bd88d8a0d4734c8b0e0a22cec6511553351b778" exitCode=0
Oct 30 23:01:29.561528 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:01:29.561371    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-31179c-58f6548f68-x75x2"
Oct 30 23:01:29.561528 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:01:29.561408    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-31179c-58f6548f68-x75x2" event={"ID":"f9f2defa-179f-4fd6-a9ad-ae7588c1a4da","Type":"ContainerDied","Data":"f819ef06563e538fa23326f93bd88d8a0d4734c8b0e0a22cec6511553351b778"}
Oct 30 23:01:29.561528 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:01:29.561463    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-31179c-58f6548f68-x75x2" event={"ID":"f9f2defa-179f-4fd6-a9ad-ae7588c1a4da","Type":"ContainerDied","Data":"d5bbb93d27adbb819dea6170b2a9af7128a18e0ff587bc132c5413bc4acc00fd"}
Oct 30 23:01:29.561528 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:01:29.561497    5510 scope.go:122] "RemoveContainer" containerID="f819ef06563e538fa23326f93bd88d8a0d4734c8b0e0a22cec6511553351b778"
Oct 30 23:01:29.571905 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:01:29.571875    5510 scope.go:122] "RemoveContainer" containerID="f819ef06563e538fa23326f93bd88d8a0d4734c8b0e0a22cec6511553351b778"
Oct 30 23:01:29.572231 ip-10-0-136-210 kubenswrapper[5510]: E1030 23:01:29.572209    5510 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f819ef06563e538fa23326f93bd88d8a0d4734c8b0e0a22cec6511553351b778\": container with ID starting with f819ef06563e538fa23326f93bd88d8a0d4734c8b0e0a22cec6511553351b778 not found: ID does not exist" containerID="f819ef06563e538fa23326f93bd88d8a0d4734c8b0e0a22cec6511553351b778"
Oct 30 23:01:29.572308 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:01:29.572242    5510 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f819ef06563e538fa23326f93bd88d8a0d4734c8b0e0a22cec6511553351b778"} err="failed to get container status \"f819ef06563e538fa23326f93bd88d8a0d4734c8b0e0a22cec6511553351b778\": rpc error: code = NotFound desc = could not find container \"f819ef06563e538fa23326f93bd88d8a0d4734c8b0e0a22cec6511553351b778\": container with ID starting with f819ef06563e538fa23326f93bd88d8a0d4734c8b0e0a22cec6511553351b778 not found: ID does not exist"
Oct 30 23:01:29.718277 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:01:29.718242    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-authcfg-http-31179c-58f6548f68-x75x2"]
Oct 30 23:01:29.721464 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:01:29.721438    5510 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-authcfg-http-31179c-58f6548f68-x75x2"]
Oct 30 23:01:30.697300 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:01:30.697257    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 23:01:30.974739 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:01:30.974578    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 23:01:31.552569 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:01:31.552532    5510 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f9f2defa-179f-4fd6-a9ad-ae7588c1a4da" path="/var/lib/kubelet/pods/f9f2defa-179f-4fd6-a9ad-ae7588c1a4da/volumes"
Oct 30 23:01:31.902915 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:01:31.902827    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 23:01:36.823081 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:01:36.823029    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 23:01:37.003923 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:01:37.003892    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 23:01:37.831576 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:01:37.831543    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 23:01:38.119083 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:01:38.119000    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 23:01:38.203598 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:01:38.203562    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-authcfg-http-31179c-openshift-default-58959bdf7-x52mb"]
Oct 30 23:01:38.203865 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:01:38.203841    5510 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-authcfg-http-31179c-openshift-default-58959bdf7-x52mb" podUID="fb6d6c9f-c79a-4cf8-b896-3cad3c60cbdf" containerName="istio-proxy" containerID="cri-o://b078321501fc3745db0bb96e45554ddfa4eeb409c1165527523ed8d468ac2971" gracePeriod=30
Oct 30 23:01:38.324786 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:01:38.324596    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 23:01:38.420877 ip-10-0-136-210 kubenswrapper[5510]: E1030 23:01:38.420785    5510 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6gjbf" containerName="extractor"
Oct 30 23:01:42.644192 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:01:42.644153    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 23:01:43.459220 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:01:43.459194    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-authcfg-http-31179c-openshift-default-58959bdf7-x52mb"
Oct 30 23:01:43.469387 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:01:43.469335    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fb6d6c9f-c79a-4cf8-b896-3cad3c60cbdf-kube-api-access-wrpcc\" (UniqueName: \"kubernetes.io/projected/fb6d6c9f-c79a-4cf8-b896-3cad3c60cbdf-kube-api-access-wrpcc\") pod \"fb6d6c9f-c79a-4cf8-b896-3cad3c60cbdf\" (UID: \"fb6d6c9f-c79a-4cf8-b896-3cad3c60cbdf\") "
Oct 30 23:01:43.469468 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:01:43.469386    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/fb6d6c9f-c79a-4cf8-b896-3cad3c60cbdf-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fb6d6c9f-c79a-4cf8-b896-3cad3c60cbdf-istio-podinfo\") pod \"fb6d6c9f-c79a-4cf8-b896-3cad3c60cbdf\" (UID: \"fb6d6c9f-c79a-4cf8-b896-3cad3c60cbdf\") "
Oct 30 23:01:43.469468 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:01:43.469451    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fb6d6c9f-c79a-4cf8-b896-3cad3c60cbdf-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb6d6c9f-c79a-4cf8-b896-3cad3c60cbdf-workload-socket\") pod \"fb6d6c9f-c79a-4cf8-b896-3cad3c60cbdf\" (UID: \"fb6d6c9f-c79a-4cf8-b896-3cad3c60cbdf\") "
Oct 30 23:01:43.469550 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:01:43.469534    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fb6d6c9f-c79a-4cf8-b896-3cad3c60cbdf-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fb6d6c9f-c79a-4cf8-b896-3cad3c60cbdf-istio-envoy\") pod \"fb6d6c9f-c79a-4cf8-b896-3cad3c60cbdf\" (UID: \"fb6d6c9f-c79a-4cf8-b896-3cad3c60cbdf\") "
Oct 30 23:01:43.469741 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:01:43.469584    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fb6d6c9f-c79a-4cf8-b896-3cad3c60cbdf-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb6d6c9f-c79a-4cf8-b896-3cad3c60cbdf-credential-socket\") pod \"fb6d6c9f-c79a-4cf8-b896-3cad3c60cbdf\" (UID: \"fb6d6c9f-c79a-4cf8-b896-3cad3c60cbdf\") "
Oct 30 23:01:43.469741 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:01:43.469617    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fb6d6c9f-c79a-4cf8-b896-3cad3c60cbdf-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fb6d6c9f-c79a-4cf8-b896-3cad3c60cbdf-workload-certs\") pod \"fb6d6c9f-c79a-4cf8-b896-3cad3c60cbdf\" (UID: \"fb6d6c9f-c79a-4cf8-b896-3cad3c60cbdf\") "
Oct 30 23:01:43.469741 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:01:43.469686    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fb6d6c9f-c79a-4cf8-b896-3cad3c60cbdf-istio-token\" (UniqueName: \"kubernetes.io/projected/fb6d6c9f-c79a-4cf8-b896-3cad3c60cbdf-istio-token\") pod \"fb6d6c9f-c79a-4cf8-b896-3cad3c60cbdf\" (UID: \"fb6d6c9f-c79a-4cf8-b896-3cad3c60cbdf\") "
Oct 30 23:01:43.469923 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:01:43.469799    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fb6d6c9f-c79a-4cf8-b896-3cad3c60cbdf-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fb6d6c9f-c79a-4cf8-b896-3cad3c60cbdf-istiod-ca-cert\") pod \"fb6d6c9f-c79a-4cf8-b896-3cad3c60cbdf\" (UID: \"fb6d6c9f-c79a-4cf8-b896-3cad3c60cbdf\") "
Oct 30 23:01:43.469923 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:01:43.469781    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb6d6c9f-c79a-4cf8-b896-3cad3c60cbdf-workload-socket" pod "fb6d6c9f-c79a-4cf8-b896-3cad3c60cbdf" (UID: "fb6d6c9f-c79a-4cf8-b896-3cad3c60cbdf"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 23:01:43.469923 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:01:43.469842    5510 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fb6d6c9f-c79a-4cf8-b896-3cad3c60cbdf-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fb6d6c9f-c79a-4cf8-b896-3cad3c60cbdf-istio-data\") pod \"fb6d6c9f-c79a-4cf8-b896-3cad3c60cbdf\" (UID: \"fb6d6c9f-c79a-4cf8-b896-3cad3c60cbdf\") "
Oct 30 23:01:43.469923 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:01:43.469888    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb6d6c9f-c79a-4cf8-b896-3cad3c60cbdf-credential-socket" pod "fb6d6c9f-c79a-4cf8-b896-3cad3c60cbdf" (UID: "fb6d6c9f-c79a-4cf8-b896-3cad3c60cbdf"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 23:01:43.470152 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:01:43.469937    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb6d6c9f-c79a-4cf8-b896-3cad3c60cbdf-workload-certs" pod "fb6d6c9f-c79a-4cf8-b896-3cad3c60cbdf" (UID: "fb6d6c9f-c79a-4cf8-b896-3cad3c60cbdf"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 23:01:43.470152 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:01:43.470129    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb6d6c9f-c79a-4cf8-b896-3cad3c60cbdf-istio-data" pod "fb6d6c9f-c79a-4cf8-b896-3cad3c60cbdf" (UID: "fb6d6c9f-c79a-4cf8-b896-3cad3c60cbdf"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 23:01:43.470268 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:01:43.470198    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fb6d6c9f-c79a-4cf8-b896-3cad3c60cbdf-istiod-ca-cert" pod "fb6d6c9f-c79a-4cf8-b896-3cad3c60cbdf" (UID: "fb6d6c9f-c79a-4cf8-b896-3cad3c60cbdf"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 23:01:43.470336 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:01:43.470289    5510 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb6d6c9f-c79a-4cf8-b896-3cad3c60cbdf-credential-socket\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 23:01:43.470336 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:01:43.470312    5510 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fb6d6c9f-c79a-4cf8-b896-3cad3c60cbdf-workload-certs\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 23:01:43.470336 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:01:43.470328    5510 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fb6d6c9f-c79a-4cf8-b896-3cad3c60cbdf-istiod-ca-cert\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 23:01:43.470440 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:01:43.470341    5510 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fb6d6c9f-c79a-4cf8-b896-3cad3c60cbdf-istio-data\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 23:01:43.470440 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:01:43.470354    5510 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb6d6c9f-c79a-4cf8-b896-3cad3c60cbdf-workload-socket\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 23:01:43.471789 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:01:43.471766    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fb6d6c9f-c79a-4cf8-b896-3cad3c60cbdf-istio-token" pod "fb6d6c9f-c79a-4cf8-b896-3cad3c60cbdf" (UID: "fb6d6c9f-c79a-4cf8-b896-3cad3c60cbdf"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 23:01:43.471877 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:01:43.471785    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fb6d6c9f-c79a-4cf8-b896-3cad3c60cbdf-kube-api-access-wrpcc" pod "fb6d6c9f-c79a-4cf8-b896-3cad3c60cbdf" (UID: "fb6d6c9f-c79a-4cf8-b896-3cad3c60cbdf"). InnerVolumeSpecName "kube-api-access-wrpcc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 23:01:43.472017 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:01:43.471991    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/fb6d6c9f-c79a-4cf8-b896-3cad3c60cbdf-istio-podinfo" pod "fb6d6c9f-c79a-4cf8-b896-3cad3c60cbdf" (UID: "fb6d6c9f-c79a-4cf8-b896-3cad3c60cbdf"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 23:01:43.472017 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:01:43.472003    5510 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb6d6c9f-c79a-4cf8-b896-3cad3c60cbdf-istio-envoy" pod "fb6d6c9f-c79a-4cf8-b896-3cad3c60cbdf" (UID: "fb6d6c9f-c79a-4cf8-b896-3cad3c60cbdf"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 23:01:43.571037 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:01:43.571010    5510 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fb6d6c9f-c79a-4cf8-b896-3cad3c60cbdf-istio-envoy\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 23:01:43.571037 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:01:43.571034    5510 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fb6d6c9f-c79a-4cf8-b896-3cad3c60cbdf-istio-token\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 23:01:43.571189 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:01:43.571045    5510 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wrpcc\" (UniqueName: \"kubernetes.io/projected/fb6d6c9f-c79a-4cf8-b896-3cad3c60cbdf-kube-api-access-wrpcc\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 23:01:43.571189 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:01:43.571054    5510 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fb6d6c9f-c79a-4cf8-b896-3cad3c60cbdf-istio-podinfo\") on node \"ip-10-0-136-210.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 23:01:43.634995 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:01:43.634962    5510 generic.go:356] "Generic (PLEG): container finished" podID="fb6d6c9f-c79a-4cf8-b896-3cad3c60cbdf" containerID="b078321501fc3745db0bb96e45554ddfa4eeb409c1165527523ed8d468ac2971" exitCode=0
Oct 30 23:01:43.635173 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:01:43.635017    5510 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-authcfg-http-31179c-openshift-default-58959bdf7-x52mb"
Oct 30 23:01:43.635173 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:01:43.635024    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-authcfg-http-31179c-openshift-default-58959bdf7-x52mb" event={"ID":"fb6d6c9f-c79a-4cf8-b896-3cad3c60cbdf","Type":"ContainerDied","Data":"b078321501fc3745db0bb96e45554ddfa4eeb409c1165527523ed8d468ac2971"}
Oct 30 23:01:43.635173 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:01:43.635066    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-authcfg-http-31179c-openshift-default-58959bdf7-x52mb" event={"ID":"fb6d6c9f-c79a-4cf8-b896-3cad3c60cbdf","Type":"ContainerDied","Data":"c7c672ec9fa31b30e31ce288a0fe549271a637842b21d03d47969022e56d6bba"}
Oct 30 23:01:43.635173 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:01:43.635089    5510 scope.go:122] "RemoveContainer" containerID="b078321501fc3745db0bb96e45554ddfa4eeb409c1165527523ed8d468ac2971"
Oct 30 23:01:43.644387 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:01:43.644364    5510 scope.go:122] "RemoveContainer" containerID="d78363b65c03f13953e3d76f977eda8123bc01479e43998cd110f0a87c80411a"
Oct 30 23:01:43.652437 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:01:43.652418    5510 scope.go:122] "RemoveContainer" containerID="b078321501fc3745db0bb96e45554ddfa4eeb409c1165527523ed8d468ac2971"
Oct 30 23:01:43.652727 ip-10-0-136-210 kubenswrapper[5510]: E1030 23:01:43.652709    5510 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b078321501fc3745db0bb96e45554ddfa4eeb409c1165527523ed8d468ac2971\": container with ID starting with b078321501fc3745db0bb96e45554ddfa4eeb409c1165527523ed8d468ac2971 not found: ID does not exist" containerID="b078321501fc3745db0bb96e45554ddfa4eeb409c1165527523ed8d468ac2971"
Oct 30 23:01:43.652807 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:01:43.652735    5510 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b078321501fc3745db0bb96e45554ddfa4eeb409c1165527523ed8d468ac2971"} err="failed to get container status \"b078321501fc3745db0bb96e45554ddfa4eeb409c1165527523ed8d468ac2971\": rpc error: code = NotFound desc = could not find container \"b078321501fc3745db0bb96e45554ddfa4eeb409c1165527523ed8d468ac2971\": container with ID starting with b078321501fc3745db0bb96e45554ddfa4eeb409c1165527523ed8d468ac2971 not found: ID does not exist"
Oct 30 23:01:43.652807 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:01:43.652755    5510 scope.go:122] "RemoveContainer" containerID="d78363b65c03f13953e3d76f977eda8123bc01479e43998cd110f0a87c80411a"
Oct 30 23:01:43.653016 ip-10-0-136-210 kubenswrapper[5510]: E1030 23:01:43.652997    5510 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d78363b65c03f13953e3d76f977eda8123bc01479e43998cd110f0a87c80411a\": container with ID starting with d78363b65c03f13953e3d76f977eda8123bc01479e43998cd110f0a87c80411a not found: ID does not exist" containerID="d78363b65c03f13953e3d76f977eda8123bc01479e43998cd110f0a87c80411a"
Oct 30 23:01:43.653060 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:01:43.653019    5510 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d78363b65c03f13953e3d76f977eda8123bc01479e43998cd110f0a87c80411a"} err="failed to get container status \"d78363b65c03f13953e3d76f977eda8123bc01479e43998cd110f0a87c80411a\": rpc error: code = NotFound desc = could not find container \"d78363b65c03f13953e3d76f977eda8123bc01479e43998cd110f0a87c80411a\": container with ID starting with d78363b65c03f13953e3d76f977eda8123bc01479e43998cd110f0a87c80411a not found: ID does not exist"
Oct 30 23:01:43.795003 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:01:43.794969    5510 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-authcfg-http-31179c-openshift-default-58959bdf7-x52mb"]
Oct 30 23:01:43.799120 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:01:43.799092    5510 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-authcfg-http-31179c-openshift-default-58959bdf7-x52mb"]
Oct 30 23:01:45.552063 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:01:45.552029    5510 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fb6d6c9f-c79a-4cf8-b896-3cad3c60cbdf" path="/var/lib/kubelet/pods/fb6d6c9f-c79a-4cf8-b896-3cad3c60cbdf/volumes"
Oct 30 23:02:08.419599 ip-10-0-136-210 kubenswrapper[5510]: E1030 23:02:08.419557    5510 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6gjbf" containerName="extractor"
Oct 30 23:02:38.419967 ip-10-0-136-210 kubenswrapper[5510]: E1030 23:02:38.419929    5510 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6gjbf" containerName="extractor"
Oct 30 23:02:42.941406 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:02:42.941362    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 23:03:01.840698 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:03:01.840638    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 23:03:02.975101 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:03:02.975055    5510 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-869c494b8f-27qzn_41b8b39f-9e91-463b-bc79-330e7d62d699/manager/0.log"
Oct 30 23:03:04.130690 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:03:04.130635    5510 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_usage-logs-tenancy-proxy-9c5d49bcd-zxct4_24ea2ad0-bd0f-4e1c-806f-b6962996beba/proxy/0.log"
Oct 30 23:03:05.197380 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:03:05.197351    5510 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-57df996b79-zq2fb_8f990085-4005-4a21-b7b4-91ce25c74761/maas-api/0.log"
Oct 30 23:03:06.149551 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:03:06.149516    5510 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-7cdbf6dc44-s67k8_03f1ca8f-6c09-4771-aff9-f217eddb47c0/postgres/0.log"
Oct 30 23:03:08.420093 ip-10-0-136-210 kubenswrapper[5510]: E1030 23:03:08.420056    5510 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6gjbf" containerName="extractor"
Oct 30 23:03:09.065836 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:03:09.065806    5510 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-678d87766f-p74zh_e36f91b0-e733-4cf5-91da-f805f35f56ba/manager/0.log"
Oct 30 23:03:11.477553 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:03:11.477503    5510 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-84877cfc-8nxg5_bae2a39a-a18e-4b74-b6ef-3879493afb2a/kube-auth-proxy/0.log"
Oct 30 23:03:12.865358 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:03:12.865325    5510 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-85544d4cd7-hfrbk_474fc2c2-03ae-420b-a831-c8e98a3e5ed8/istio-proxy/0.log"
Oct 30 23:03:13.111561 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:03:13.111495    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 23:03:13.304276 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:03:13.304241    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 23:03:13.434172 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:03:13.434132    5510 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-85544d4cd7-llmh8_1123817a-b255-47e2-9188-91650a8b4378/istio-proxy/0.log"
Oct 30 23:03:13.710298 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:03:13.710216    5510 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-85544d4cd7-m5nkk_6fb59fa4-0f8d-4111-87e3-e0a2e29982e4/istio-proxy/0.log"
Oct 30 23:03:13.933488 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:03:13.933461    5510 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-85544d4cd7-q6cg4_46c98868-6890-4614-93b4-55493087df27/istio-proxy/0.log"
Oct 30 23:03:14.686651 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:03:14.686617    5510 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7b98bb678c-slj64_6a38438c-669a-4bb0-af27-0d7b661d5ba0/payload-pre-processing/0.log"
Oct 30 23:03:14.898570 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:03:14.898539    5510 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-55fbbbf9dc-d72tk_c08b8a69-8878-4042-82ee-d9eed9bcb960/payload-processing/0.log"
Oct 30 23:03:16.040855 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:03:16.040823    5510 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-cbd4f5586-42mzp_9f899391-6025-4f44-9553-a82cd5e25e69/main/0.log"
Oct 30 23:03:16.536558 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:03:16.536523    5510 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-embedding-simulated-kserve-67bf594cd6-zz2gk_cb6133b6-d8dc-4687-9974-68874f9a34d4/main/0.log"
Oct 30 23:03:16.991965 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:03:16.991865    5510 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8kpvjt_fb0f223b-851b-438d-b981-4d0d79ccfe64/main/0.log"
Oct 30 23:03:17.218600 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:03:17.218566    5510 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_echo-simulator-kserve-64b6f8f96c-b62z6_558165fb-690a-4dbf-8362-defbce94ea9b/main/0.log"
Oct 30 23:03:17.448377 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:03:17.448345    5510 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-84b4ff656c-vrvcr_0cf74802-5189-453b-8b55-801a5f442939/main/0.log"
Oct 30 23:03:21.623160 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:03:21.623129    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 23:03:29.398634 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:03:29.398600    5510 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-869c494b8f-27qzn_41b8b39f-9e91-463b-bc79-330e7d62d699/manager/0.log"
Oct 30 23:03:29.810117 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:03:29.810076    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 23:03:30.002078 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:03:30.002042    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 23:03:30.358594 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:03:30.358557    5510 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_usage-logs-tenancy-proxy-9c5d49bcd-zxct4_24ea2ad0-bd0f-4e1c-806f-b6962996beba/proxy/0.log"
Oct 30 23:03:31.233716 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:03:31.233678    5510 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-57df996b79-zq2fb_8f990085-4005-4a21-b7b4-91ce25c74761/maas-api/0.log"
Oct 30 23:03:32.138504 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:03:32.138474    5510 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-7cdbf6dc44-s67k8_03f1ca8f-6c09-4771-aff9-f217eddb47c0/postgres/0.log"
Oct 30 23:03:34.593221 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:03:34.593187    5510 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-678d87766f-p74zh_e36f91b0-e733-4cf5-91da-f805f35f56ba/manager/0.log"
Oct 30 23:03:36.476167 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:03:36.476130    5510 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-84877cfc-8nxg5_bae2a39a-a18e-4b74-b6ef-3879493afb2a/kube-auth-proxy/0.log"
Oct 30 23:03:37.821765 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:03:37.821726    5510 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-85544d4cd7-hfrbk_474fc2c2-03ae-420b-a831-c8e98a3e5ed8/istio-proxy/0.log"
Oct 30 23:03:38.367099 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:03:38.367068    5510 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-85544d4cd7-llmh8_1123817a-b255-47e2-9188-91650a8b4378/istio-proxy/0.log"
Oct 30 23:03:38.419568 ip-10-0-136-210 kubenswrapper[5510]: E1030 23:03:38.419530    5510 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6gjbf" containerName="extractor"
Oct 30 23:03:38.643590 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:03:38.643498    5510 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-85544d4cd7-m5nkk_6fb59fa4-0f8d-4111-87e3-e0a2e29982e4/istio-proxy/0.log"
Oct 30 23:03:38.877412 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:03:38.877379    5510 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-85544d4cd7-q6cg4_46c98868-6890-4614-93b4-55493087df27/istio-proxy/0.log"
Oct 30 23:03:39.621082 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:03:39.621050    5510 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7b98bb678c-slj64_6a38438c-669a-4bb0-af27-0d7b661d5ba0/payload-pre-processing/0.log"
Oct 30 23:03:39.840396 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:03:39.840363    5510 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-55fbbbf9dc-d72tk_c08b8a69-8878-4042-82ee-d9eed9bcb960/payload-processing/0.log"
Oct 30 23:03:40.742698 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:03:40.742666    5510 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-cbd4f5586-42mzp_9f899391-6025-4f44-9553-a82cd5e25e69/main/0.log"
Oct 30 23:03:41.233775 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:03:41.233745    5510 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-embedding-simulated-kserve-67bf594cd6-zz2gk_cb6133b6-d8dc-4687-9974-68874f9a34d4/main/0.log"
Oct 30 23:03:41.671876 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:03:41.671838    5510 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8kpvjt_fb0f223b-851b-438d-b981-4d0d79ccfe64/main/0.log"
Oct 30 23:03:41.887218 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:03:41.887184    5510 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_echo-simulator-kserve-64b6f8f96c-b62z6_558165fb-690a-4dbf-8362-defbce94ea9b/main/0.log"
Oct 30 23:03:42.110448 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:03:42.110417    5510 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-84b4ff656c-vrvcr_0cf74802-5189-453b-8b55-801a5f442939/main/0.log"
Oct 30 23:03:43.283311 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:03:43.283277    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 23:03:54.358143 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:03:54.358104    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 23:03:54.554283 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:03:54.554240    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 23:04:08.419172 ip-10-0-136-210 kubenswrapper[5510]: E1030 23:04:08.419079    5510 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6gjbf" containerName="extractor"
Oct 30 23:04:12.446944 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:04:12.446911    5510 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-869c494b8f-27qzn_41b8b39f-9e91-463b-bc79-330e7d62d699/manager/0.log"
Oct 30 23:04:12.998665 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:04:12.998623    5510 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-869c494b8f-27qzn_41b8b39f-9e91-463b-bc79-330e7d62d699/manager/0.log"
Oct 30 23:04:13.770429 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:04:13.770396    5510 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-55fbbbf9dc-d72tk_c08b8a69-8878-4042-82ee-d9eed9bcb960/payload-processing/0.log"
Oct 30 23:04:13.989833 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:04:13.989807    5510 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-55fbbbf9dc-d72tk_c08b8a69-8878-4042-82ee-d9eed9bcb960/payload-processing/0.log"
Oct 30 23:04:38.419759 ip-10-0-136-210 kubenswrapper[5510]: E1030 23:04:38.419713    5510 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6gjbf" containerName="extractor"
Oct 30 23:04:43.621711 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:04:43.621670    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 23:04:53.751669 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:04:53.751620    5510 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-869c494b8f-27qzn_41b8b39f-9e91-463b-bc79-330e7d62d699/manager/0.log"
Oct 30 23:04:54.312904 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:04:54.312868    5510 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-869c494b8f-27qzn_41b8b39f-9e91-463b-bc79-330e7d62d699/manager/0.log"
Oct 30 23:04:55.040958 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:04:55.040929    5510 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-55fbbbf9dc-d72tk_c08b8a69-8878-4042-82ee-d9eed9bcb960/payload-processing/0.log"
Oct 30 23:04:55.275082 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:04:55.275051    5510 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-55fbbbf9dc-d72tk_c08b8a69-8878-4042-82ee-d9eed9bcb960/payload-processing/0.log"
Oct 30 23:04:55.627715 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:04:55.627680    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 23:04:55.632265 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:04:55.632234    5510 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-must-gather-2cfz8/must-gather-hdc4z"]
Oct 30 23:04:55.632689 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:04:55.632676    5510 cpu_manager.go:414] "containerMap: removing container" podUID="fb6d6c9f-c79a-4cf8-b896-3cad3c60cbdf" containerName="istio-proxy"
Oct 30 23:04:55.632744 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:04:55.632690    5510 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fb6d6c9f-c79a-4cf8-b896-3cad3c60cbdf" containerName="istio-proxy"
Oct 30 23:04:55.632744 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:04:55.632713    5510 cpu_manager.go:414] "containerMap: removing container" podUID="a01d85f8-b761-4c55-aaa4-bae0fd901a65" containerName="payload-pre-processing"
Oct 30 23:04:55.632744 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:04:55.632719    5510 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a01d85f8-b761-4c55-aaa4-bae0fd901a65" containerName="payload-pre-processing"
Oct 30 23:04:55.632744 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:04:55.632728    5510 cpu_manager.go:414] "containerMap: removing container" podUID="f9f2defa-179f-4fd6-a9ad-ae7588c1a4da" containerName="maas-api"
Oct 30 23:04:55.632744 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:04:55.632733    5510 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f9f2defa-179f-4fd6-a9ad-ae7588c1a4da" containerName="maas-api"
Oct 30 23:04:55.632914 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:04:55.632748    5510 cpu_manager.go:414] "containerMap: removing container" podUID="8c127dae-831c-4a86-b68d-11a8091e8853" containerName="payload-processing"
Oct 30 23:04:55.632914 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:04:55.632753    5510 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8c127dae-831c-4a86-b68d-11a8091e8853" containerName="payload-processing"
Oct 30 23:04:55.632914 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:04:55.632818    5510 memory_manager.go:365] "RemoveStaleState removing state" podUID="a01d85f8-b761-4c55-aaa4-bae0fd901a65" containerName="payload-pre-processing"
Oct 30 23:04:55.632914 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:04:55.632828    5510 memory_manager.go:365] "RemoveStaleState removing state" podUID="f9f2defa-179f-4fd6-a9ad-ae7588c1a4da" containerName="maas-api"
Oct 30 23:04:55.632914 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:04:55.632833    5510 memory_manager.go:365] "RemoveStaleState removing state" podUID="fb6d6c9f-c79a-4cf8-b896-3cad3c60cbdf" containerName="istio-proxy"
Oct 30 23:04:55.632914 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:04:55.632840    5510 memory_manager.go:365] "RemoveStaleState removing state" podUID="8c127dae-831c-4a86-b68d-11a8091e8853" containerName="payload-processing"
Oct 30 23:04:55.632914 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:04:55.632848    5510 memory_manager.go:365] "RemoveStaleState removing state" podUID="fb6d6c9f-c79a-4cf8-b896-3cad3c60cbdf" containerName="istio-proxy"
Oct 30 23:04:55.633143 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:04:55.632917    5510 cpu_manager.go:414] "containerMap: removing container" podUID="fb6d6c9f-c79a-4cf8-b896-3cad3c60cbdf" containerName="istio-proxy"
Oct 30 23:04:55.633143 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:04:55.632922    5510 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fb6d6c9f-c79a-4cf8-b896-3cad3c60cbdf" containerName="istio-proxy"
Oct 30 23:04:55.636067 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:04:55.636048    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-2cfz8/must-gather-hdc4z"
Oct 30 23:04:55.731758 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:04:55.731726    5510 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-2cfz8\"/\"openshift-service-ca.crt\""
Oct 30 23:04:55.731758 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:04:55.731764    5510 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-2cfz8\"/\"default-dockercfg-4lpsw\""
Oct 30 23:04:55.731983 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:04:55.731795    5510 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-2cfz8\"/\"kube-root-ca.crt\""
Oct 30 23:04:55.733451 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:04:55.733430    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-2cfz8/must-gather-hdc4z"]
Oct 30 23:04:55.799672 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:04:55.799621    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-btj4n\" (UniqueName: \"kubernetes.io/projected/e60b68d6-cbd2-4d82-9740-0d42cea8eea2-kube-api-access-btj4n\") pod \"must-gather-hdc4z\" (UID: \"e60b68d6-cbd2-4d82-9740-0d42cea8eea2\") " pod="openshift-must-gather-2cfz8/must-gather-hdc4z"
Oct 30 23:04:55.799853 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:04:55.799746    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/e60b68d6-cbd2-4d82-9740-0d42cea8eea2-must-gather-output\") pod \"must-gather-hdc4z\" (UID: \"e60b68d6-cbd2-4d82-9740-0d42cea8eea2\") " pod="openshift-must-gather-2cfz8/must-gather-hdc4z"
Oct 30 23:04:55.822961 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:04:55.822923    5510 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log" limit=1024
Oct 30 23:04:55.900895 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:04:55.900805    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-btj4n\" (UniqueName: \"kubernetes.io/projected/e60b68d6-cbd2-4d82-9740-0d42cea8eea2-kube-api-access-btj4n\") pod \"must-gather-hdc4z\" (UID: \"e60b68d6-cbd2-4d82-9740-0d42cea8eea2\") " pod="openshift-must-gather-2cfz8/must-gather-hdc4z"
Oct 30 23:04:55.900895 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:04:55.900853    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/e60b68d6-cbd2-4d82-9740-0d42cea8eea2-must-gather-output\") pod \"must-gather-hdc4z\" (UID: \"e60b68d6-cbd2-4d82-9740-0d42cea8eea2\") " pod="openshift-must-gather-2cfz8/must-gather-hdc4z"
Oct 30 23:04:55.901198 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:04:55.901181    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/e60b68d6-cbd2-4d82-9740-0d42cea8eea2-must-gather-output\") pod \"must-gather-hdc4z\" (UID: \"e60b68d6-cbd2-4d82-9740-0d42cea8eea2\") " pod="openshift-must-gather-2cfz8/must-gather-hdc4z"
Oct 30 23:04:55.958713 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:04:55.958678    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-btj4n\" (UniqueName: \"kubernetes.io/projected/e60b68d6-cbd2-4d82-9740-0d42cea8eea2-kube-api-access-btj4n\") pod \"must-gather-hdc4z\" (UID: \"e60b68d6-cbd2-4d82-9740-0d42cea8eea2\") " pod="openshift-must-gather-2cfz8/must-gather-hdc4z"
Oct 30 23:04:56.250005 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:04:56.249904    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-2cfz8/must-gather-hdc4z"
Oct 30 23:04:56.616704 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:04:56.616667    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-2cfz8/must-gather-hdc4z"]
Oct 30 23:04:56.620177 ip-10-0-136-210 kubenswrapper[5510]: W1030 23:04:56.620146    5510 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode60b68d6_cbd2_4d82_9740_0d42cea8eea2.slice/crio-d3436e2eccb40cc31c0cf2ec0981bb37328803ff1231bf200ace072a8629078d WatchSource:0}: Error finding container d3436e2eccb40cc31c0cf2ec0981bb37328803ff1231bf200ace072a8629078d: Status 404 returned error can't find the container with id d3436e2eccb40cc31c0cf2ec0981bb37328803ff1231bf200ace072a8629078d
Oct 30 23:04:56.622046 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:04:56.622026    5510 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 23:04:56.687030 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:04:56.686994    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-2cfz8/must-gather-hdc4z" event={"ID":"e60b68d6-cbd2-4d82-9740-0d42cea8eea2","Type":"ContainerStarted","Data":"d3436e2eccb40cc31c0cf2ec0981bb37328803ff1231bf200ace072a8629078d"}
Oct 30 23:04:58.699636 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:04:58.699601    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-2cfz8/must-gather-hdc4z" event={"ID":"e60b68d6-cbd2-4d82-9740-0d42cea8eea2","Type":"ContainerStarted","Data":"be9aa64f23c51de2d25017f1f38afff5b508e1be022b9eb17220e9ab18c9c53e"}
Oct 30 23:04:58.700149 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:04:58.699645    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-2cfz8/must-gather-hdc4z" event={"ID":"e60b68d6-cbd2-4d82-9740-0d42cea8eea2","Type":"ContainerStarted","Data":"74e7eb7f96fe2c85a6b94f91affaf38aaae1c13a7e800f19fdd062225d7d4884"}
Oct 30 23:04:58.812817 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:04:58.812752    5510 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-must-gather-2cfz8/must-gather-hdc4z" podStartSLOduration=2.4008042720000002 podStartE2EDuration="3.812734658s" totalImagesPullingTime="1.411930386s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 23:04:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 23:04:58.812141237 +0000 UTC m=+4479.372951239" watchObservedRunningTime="2026-10-30 23:04:58.812734658 +0000 UTC m=+4479.373544664"
Oct 30 23:05:04.602621 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:05:04.602586    5510 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-68jrf_e810bcd1-78da-409f-b86e-08c47031939e/global-pull-secret-syncer/0.log"
Oct 30 23:05:07.727645 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:05:07.727610    5510 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-5mw6n_f69b6a2e-d33f-4cc8-8e0a-598b20948f38/konnectivity-agent/0.log"
Oct 30 23:05:08.457367 ip-10-0-136-210 kubenswrapper[5510]: E1030 23:05:08.457318    5510 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6gjbf" containerName="extractor"
Oct 30 23:05:09.839415 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:05:09.839381    5510 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-136-210.us-west-2.compute.internal_4492449458f4d650448405f5044e8d24/haproxy/0.log"
Oct 30 23:05:14.982855 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:05:14.982816    5510 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-must-gather-2cfz8/perf-node-gather-daemonset-s8n9k"]
Oct 30 23:05:14.988551 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:05:14.988522    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-2cfz8/perf-node-gather-daemonset-s8n9k"
Oct 30 23:05:15.043863 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:05:15.043024    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-2cfz8/perf-node-gather-daemonset-s8n9k"]
Oct 30 23:05:15.113877 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:05:15.113841    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/fa9f6061-288e-40e5-bfa0-26fff17a0a69-lib-modules\") pod \"perf-node-gather-daemonset-s8n9k\" (UID: \"fa9f6061-288e-40e5-bfa0-26fff17a0a69\") " pod="openshift-must-gather-2cfz8/perf-node-gather-daemonset-s8n9k"
Oct 30 23:05:15.114058 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:05:15.113915    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ttpvx\" (UniqueName: \"kubernetes.io/projected/fa9f6061-288e-40e5-bfa0-26fff17a0a69-kube-api-access-ttpvx\") pod \"perf-node-gather-daemonset-s8n9k\" (UID: \"fa9f6061-288e-40e5-bfa0-26fff17a0a69\") " pod="openshift-must-gather-2cfz8/perf-node-gather-daemonset-s8n9k"
Oct 30 23:05:15.114058 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:05:15.113983    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fa9f6061-288e-40e5-bfa0-26fff17a0a69-sys\") pod \"perf-node-gather-daemonset-s8n9k\" (UID: \"fa9f6061-288e-40e5-bfa0-26fff17a0a69\") " pod="openshift-must-gather-2cfz8/perf-node-gather-daemonset-s8n9k"
Oct 30 23:05:15.114058 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:05:15.114021    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/fa9f6061-288e-40e5-bfa0-26fff17a0a69-proc\") pod \"perf-node-gather-daemonset-s8n9k\" (UID: \"fa9f6061-288e-40e5-bfa0-26fff17a0a69\") " pod="openshift-must-gather-2cfz8/perf-node-gather-daemonset-s8n9k"
Oct 30 23:05:15.114176 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:05:15.114136    5510 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/fa9f6061-288e-40e5-bfa0-26fff17a0a69-podres\") pod \"perf-node-gather-daemonset-s8n9k\" (UID: \"fa9f6061-288e-40e5-bfa0-26fff17a0a69\") " pod="openshift-must-gather-2cfz8/perf-node-gather-daemonset-s8n9k"
Oct 30 23:05:15.215765 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:05:15.215724    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/fa9f6061-288e-40e5-bfa0-26fff17a0a69-proc\") pod \"perf-node-gather-daemonset-s8n9k\" (UID: \"fa9f6061-288e-40e5-bfa0-26fff17a0a69\") " pod="openshift-must-gather-2cfz8/perf-node-gather-daemonset-s8n9k"
Oct 30 23:05:15.216188 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:05:15.216167    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/fa9f6061-288e-40e5-bfa0-26fff17a0a69-podres\") pod \"perf-node-gather-daemonset-s8n9k\" (UID: \"fa9f6061-288e-40e5-bfa0-26fff17a0a69\") " pod="openshift-must-gather-2cfz8/perf-node-gather-daemonset-s8n9k"
Oct 30 23:05:15.216443 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:05:15.216428    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/fa9f6061-288e-40e5-bfa0-26fff17a0a69-lib-modules\") pod \"perf-node-gather-daemonset-s8n9k\" (UID: \"fa9f6061-288e-40e5-bfa0-26fff17a0a69\") " pod="openshift-must-gather-2cfz8/perf-node-gather-daemonset-s8n9k"
Oct 30 23:05:15.216760 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:05:15.216728    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ttpvx\" (UniqueName: \"kubernetes.io/projected/fa9f6061-288e-40e5-bfa0-26fff17a0a69-kube-api-access-ttpvx\") pod \"perf-node-gather-daemonset-s8n9k\" (UID: \"fa9f6061-288e-40e5-bfa0-26fff17a0a69\") " pod="openshift-must-gather-2cfz8/perf-node-gather-daemonset-s8n9k"
Oct 30 23:05:15.216878 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:05:15.216792    5510 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fa9f6061-288e-40e5-bfa0-26fff17a0a69-sys\") pod \"perf-node-gather-daemonset-s8n9k\" (UID: \"fa9f6061-288e-40e5-bfa0-26fff17a0a69\") " pod="openshift-must-gather-2cfz8/perf-node-gather-daemonset-s8n9k"
Oct 30 23:05:15.216878 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:05:15.216597    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/fa9f6061-288e-40e5-bfa0-26fff17a0a69-lib-modules\") pod \"perf-node-gather-daemonset-s8n9k\" (UID: \"fa9f6061-288e-40e5-bfa0-26fff17a0a69\") " pod="openshift-must-gather-2cfz8/perf-node-gather-daemonset-s8n9k"
Oct 30 23:05:15.216878 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:05:15.216353    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/fa9f6061-288e-40e5-bfa0-26fff17a0a69-podres\") pod \"perf-node-gather-daemonset-s8n9k\" (UID: \"fa9f6061-288e-40e5-bfa0-26fff17a0a69\") " pod="openshift-must-gather-2cfz8/perf-node-gather-daemonset-s8n9k"
Oct 30 23:05:15.217177 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:05:15.216056    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/fa9f6061-288e-40e5-bfa0-26fff17a0a69-proc\") pod \"perf-node-gather-daemonset-s8n9k\" (UID: \"fa9f6061-288e-40e5-bfa0-26fff17a0a69\") " pod="openshift-must-gather-2cfz8/perf-node-gather-daemonset-s8n9k"
Oct 30 23:05:15.217240 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:05:15.217223    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fa9f6061-288e-40e5-bfa0-26fff17a0a69-sys\") pod \"perf-node-gather-daemonset-s8n9k\" (UID: \"fa9f6061-288e-40e5-bfa0-26fff17a0a69\") " pod="openshift-must-gather-2cfz8/perf-node-gather-daemonset-s8n9k"
Oct 30 23:05:15.275188 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:05:15.275155    5510 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ttpvx\" (UniqueName: \"kubernetes.io/projected/fa9f6061-288e-40e5-bfa0-26fff17a0a69-kube-api-access-ttpvx\") pod \"perf-node-gather-daemonset-s8n9k\" (UID: \"fa9f6061-288e-40e5-bfa0-26fff17a0a69\") " pod="openshift-must-gather-2cfz8/perf-node-gather-daemonset-s8n9k"
Oct 30 23:05:15.302328 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:05:15.302302    5510 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-2cfz8/perf-node-gather-daemonset-s8n9k"
Oct 30 23:05:15.669325 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:05:15.669239    5510 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-2cfz8/perf-node-gather-daemonset-s8n9k"]
Oct 30 23:05:15.672494 ip-10-0-136-210 kubenswrapper[5510]: W1030 23:05:15.672467    5510 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podfa9f6061_288e_40e5_bfa0_26fff17a0a69.slice/crio-d8b0f9a52785f1eb74e5b92053f01d2a01d9ca532a1b9c3c5ec1decc638a3e79 WatchSource:0}: Error finding container d8b0f9a52785f1eb74e5b92053f01d2a01d9ca532a1b9c3c5ec1decc638a3e79: Status 404 returned error can't find the container with id d8b0f9a52785f1eb74e5b92053f01d2a01d9ca532a1b9c3c5ec1decc638a3e79
Oct 30 23:05:15.826706 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:05:15.826637    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-2cfz8/perf-node-gather-daemonset-s8n9k" event={"ID":"fa9f6061-288e-40e5-bfa0-26fff17a0a69","Type":"ContainerStarted","Data":"0fb9c0585fef3b4b566a56e3b18e58564e027aaa5ce13b3e1e1451cdeda3e227"}
Oct 30 23:05:15.826871 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:05:15.826727    5510 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-2cfz8/perf-node-gather-daemonset-s8n9k" event={"ID":"fa9f6061-288e-40e5-bfa0-26fff17a0a69","Type":"ContainerStarted","Data":"d8b0f9a52785f1eb74e5b92053f01d2a01d9ca532a1b9c3c5ec1decc638a3e79"}
Oct 30 23:05:15.826871 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:05:15.826771    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-2cfz8/perf-node-gather-daemonset-s8n9k"
Oct 30 23:05:15.938724 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:05:15.938607    5510 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-must-gather-2cfz8/perf-node-gather-daemonset-s8n9k" podStartSLOduration=1.9385929960000001 podStartE2EDuration="1.938592996s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 23:05:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 23:05:15.936750297 +0000 UTC m=+4496.497560302" watchObservedRunningTime="2026-10-30 23:05:15.938592996 +0000 UTC m=+4496.499403001"
Oct 30 23:05:16.225307 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:05:16.225225    5510 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-678d87766f-p74zh_e36f91b0-e733-4cf5-91da-f805f35f56ba/manager/0.log"
Oct 30 23:05:19.671894 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:05:19.671862    5510 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_c59dcc75-2e4e-4ece-ae39-2e6c5f693942/alertmanager/0.log"
Oct 30 23:05:20.069034 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:05:20.069001    5510 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_c59dcc75-2e4e-4ece-ae39-2e6c5f693942/config-reloader/0.log"
Oct 30 23:05:20.467316 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:05:20.467232    5510 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_c59dcc75-2e4e-4ece-ae39-2e6c5f693942/kube-rbac-proxy-web/0.log"
Oct 30 23:05:20.865699 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:05:20.865667    5510 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_c59dcc75-2e4e-4ece-ae39-2e6c5f693942/kube-rbac-proxy/0.log"
Oct 30 23:05:21.264565 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:05:21.264472    5510 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_c59dcc75-2e4e-4ece-ae39-2e6c5f693942/kube-rbac-proxy-metric/0.log"
Oct 30 23:05:21.664387 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:05:21.664355    5510 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_c59dcc75-2e4e-4ece-ae39-2e6c5f693942/prom-label-proxy/0.log"
Oct 30 23:05:21.842819 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:05:21.842788    5510 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-2cfz8/perf-node-gather-daemonset-s8n9k"
Oct 30 23:05:22.063494 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:05:22.063455    5510 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_c59dcc75-2e4e-4ece-ae39-2e6c5f693942/init-config-reloader/0.log"
Oct 30 23:05:24.586444 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:05:24.586408    5510 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-77f4f76594-bfnw9_8883cb69-54c2-437c-a92e-3bc38547027f/monitoring-plugin/0.log"
Oct 30 23:05:26.277903 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:05:26.277867    5510 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-mpsnt_d414339d-1a64-4a2a-ad15-805b71ebfd1b/node-exporter/0.log"
Oct 30 23:05:26.686161 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:05:26.686079    5510 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-mpsnt_d414339d-1a64-4a2a-ad15-805b71ebfd1b/kube-rbac-proxy/0.log"
Oct 30 23:05:27.089477 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:05:27.089448    5510 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-mpsnt_d414339d-1a64-4a2a-ad15-805b71ebfd1b/init-textfile/0.log"
Oct 30 23:05:28.737310 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:05:28.737265    5510 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-7bd7489957-h95jn_3abd55eb-d1d8-4fdf-b82c-d35894b652ac/kube-rbac-proxy-main/0.log"
Oct 30 23:05:29.134067 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:05:29.134039    5510 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-7bd7489957-h95jn_3abd55eb-d1d8-4fdf-b82c-d35894b652ac/kube-rbac-proxy-self/0.log"
Oct 30 23:05:29.535574 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:05:29.535538    5510 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-7bd7489957-h95jn_3abd55eb-d1d8-4fdf-b82c-d35894b652ac/openshift-state-metrics/0.log"
Oct 30 23:05:34.047000 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:05:34.046970    5510 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-bb95b88bf-g9f94_0e0d17a1-7742-42b0-9d9b-740bb945e454/telemeter-client/0.log"
Oct 30 23:05:34.442641 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:05:34.442556    5510 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-bb95b88bf-g9f94_0e0d17a1-7742-42b0-9d9b-740bb945e454/reload/0.log"
Oct 30 23:05:34.840681 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:05:34.840625    5510 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-bb95b88bf-g9f94_0e0d17a1-7742-42b0-9d9b-740bb945e454/kube-rbac-proxy/0.log"
Oct 30 23:05:35.244829 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:05:35.244758    5510 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7c848fbc95-8jm99_8c88b730-5cf1-4434-950f-4a3aa8f739cd/thanos-query/0.log"
Oct 30 23:05:35.642243 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:05:35.642210    5510 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7c848fbc95-8jm99_8c88b730-5cf1-4434-950f-4a3aa8f739cd/kube-rbac-proxy-web/0.log"
Oct 30 23:05:36.042159 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:05:36.042126    5510 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7c848fbc95-8jm99_8c88b730-5cf1-4434-950f-4a3aa8f739cd/kube-rbac-proxy/0.log"
Oct 30 23:05:36.445855 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:05:36.445767    5510 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7c848fbc95-8jm99_8c88b730-5cf1-4434-950f-4a3aa8f739cd/prom-label-proxy/0.log"
Oct 30 23:05:36.843932 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:05:36.843895    5510 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7c848fbc95-8jm99_8c88b730-5cf1-4434-950f-4a3aa8f739cd/kube-rbac-proxy-rules/0.log"
Oct 30 23:05:37.241365 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:05:37.241277    5510 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7c848fbc95-8jm99_8c88b730-5cf1-4434-950f-4a3aa8f739cd/kube-rbac-proxy-metrics/0.log"
Oct 30 23:05:38.357238 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:05:38.357212    5510 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-84877cfc-8nxg5_bae2a39a-a18e-4b74-b6ef-3879493afb2a/kube-auth-proxy/0.log"
Oct 30 23:05:38.420567 ip-10-0-136-210 kubenswrapper[5510]: E1030 23:05:38.420534    5510 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6gjbf" containerName="extractor"
Oct 30 23:05:39.403382 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:05:39.403351    5510 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-85544d4cd7-hfrbk_474fc2c2-03ae-420b-a831-c8e98a3e5ed8/istio-proxy/0.log"
Oct 30 23:05:39.821482 ip-10-0-136-210 kubenswrapper[5510]: I1030 23:05:39.821447    5510 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-85544d4cd7-llmh8_1123817a-b255-47e2-9188-91650a8b4378/istio-proxy/0.log"